How russow10/capuchin-bird-audio-recognizer works
Project Type & Purpose
• This is a full-stack (web) application built using Streamlit that processes audio recordings to detect and analyze Capuchin bird calls.
• It integrates deep learning (via TensorFlow and pre-trained model weights) for detection, uses audio processing libraries (Librosa, Matplotlib, and Pandas) to generate visualizations/analyses, and leverages a RAG-powered chatbot (using LangChain and Chromadb) for interactive questions.
Main Components
a. Frontend/Presentation Layer:
• Streamlit Application: Represented by the main entry point (app.py) and the interactive UI pages located in the pages/ directory.
• Pages: Upload_Audio, Audio_Results, Chat_Assistant, and About pages each handling specific tasks (audio upload, results visualization, interactive chatbot, and project information respectively).
b. Backend/Processing Layer:
• Audio Processing Module: Logic that extracts features (spectrograms, MFCCs, etc.) from uploaded audio using Librosa.
• Deep Learning Module: Utilizes TensorFlow along with the pre-trained weights (in the weights directory) to detect and count bird calls.
• Analysis Components: Generate visual analytics using Matplotlib and Pandas.
c. External/Supporting Services:
• RAG-Chatbot & Vector Databases: Components powered by LangChain for the chatbot functionality, interfacing with a vector database such as Chromadb (mentioned in the README as chroma_db / temp_chroma_db).
• Environment Configuration: Configured via environment variables (e.g., GROQ_API_KEY in the .env file).
d. Auxiliary Components:
• Documentation (docs folder) and Notebooks: For experiments, model training, and exploration.
• Assets: Image assets (images folder), which enhance the UI or documentation.
Relationships & Interactions
• The user interacts with the application’s UI (built with Streamlit), which routes requests (clicks, file uploads) to the appropriate page handlers.
• The Upload_Audio page sends data (audio files) into the audio processing module, which extracts features and passes them on to the deep learning model.
• The deep learning module evaluates audio data using pre-trained TensorFlow models (using weights from the weights folder) and returns detection results.
• The Audio_Results page then displays visualizations (waveforms, spectrograms, etc.) based on processed results.
• For conversational queries, the Chat_Assistant page sends analysis context to the LangChain-based RAG chatbot, which in turn queries the Chromadb vector database to retrieve and present relevant answers.
• The main app.py acts as the entry point tying all these components together.
Architectural Patterns & Design Principles
• Modularity: Each page and processing module is separated into distinct components, which aids in maintainability and scalability.
• Layered Architecture: Clear separation between UI (Streamlit pages), processing (audio and analysis logic), and data (model weights, external DB).
• Extensibility: The architecture allows for additional pages or processing modules (e.g., for new audio analyses or integrations) to be easily added.