How harry0703/audionotes works
Project Type and Purpose
• AudioNotes is a backend application that processes multimedia inputs to generate structured, readable notes.
• Its purpose is to extract content from audio/video files using FunASR and then enhance and structure that content using a language model (Qwen2 through Ollama).
• The project supports two modes of deployment: Docker-based (recommended) and local deployment with a PostgreSQL database.
File Structure
• Top-Level Files:
• .env.example, Dockerfile, docker-compose.yml, main.py, and requirements.txt indicate environmental configuration, containerization, and the entry point.
• The app Directory:
• Contains the core business logic.
• app/services holds modules that interface with external services:
• asr_funasr.py: Likely handles communication with the FunASR service for audio/video recognition.
• data_layer.py: Manages interactions with the database (PostgreSQL), ensuring that processed notes are stored.
• ollama.py: Interfaces with the Ollama service to access Qwen2 for processing the recognized content.
• app/utils holds utility functions (utils.py) used across modules.
• docs Directory:
• Contains illustrations and sample images demonstrating the system’s output.
README for Additional Insights
• The README clearly outlines how the system interacts with external dependencies (FunASR and Qwen2) and describes both deployment methods.
• It provides key URLs and commands, which indicate the need to manage dependencies and configurations (e.g., modifying .env, running Docker containers or a local server).
a. Main Components to Include:
• Client/User Interface: Represents the end-user accessing the service via a web-based UI (e.g., provided by chainlit as hinted by the 'chainlit run main.py' command).
• API/Application Server: The server that runs main.py — this acts as the central hub orchestrating service interactions.
• Service Layer:
• ASR Module (app/services/asr_funasr.py): For audio/video recognition via FunASR.
• Data Layer Module (app/services/data_layer.py): For interactions with the PostgreSQL database.
• Ollama Module (app/services/ollama.py): For interfacing with the Qwen2 model.
• Utilities: Represent the shared functions in app/utils/utils.py that support service operations.
• External Services:
• FunASR (external open-source ASR tool).
• Ollama’s Qwen2 (the language model).
• Database:
• PostgreSQL: Used in the local deployment setup for storing structured notes and related data.
b. Relationships and Interactions:
• How the client sends requests to the API/Application server.
• That the ASR module sends audio/video files to FunASR and receives recognized text.
• Then, this text is passed to the Ollama module, which calls the Qwen2 model to transform or structure the content.
• The structured notes are stored/retrieved from the PostgreSQL database (via the Data module).
• Interactions back to the client, e.g., delivering the structured markdown notes via the UI.
c. Architectural Patterns & Design Principles:
• Separation of Concerns: The division into services (ASR, Data, and Ollama) ensures each module focuses on a specific task.
• Modularity: Use of utility functions and clear service modules benefits maintainability and future enhancements.
• Deployment Flexibility: The presence of both Docker and local deployment setups indicates the need to clearly delineate the infrastructure components and environment variables/configurations.
• Extensibility: With a clear service and data layer, incorporating additional services or switching between external providers can be easily visualized.
• From Client → Application Server
• From Application Server → ASR Service → Application Server → Language Processing Service → Application Server → Database
• Additional Annotations: Label key protocols or interfaces if known (e.g., REST API calls, gRPC, or direct function calls) and note container-specific details if illustrating Docker elements.