How prachirathi07/cogvoice-poc works
Project Type and Purpose
• This is a Python-based Proof-of-Concept voice-analysis web application built with Streamlit. It ingests short audio samples, extracts acoustic and linguistic features (via Whisper), applies basic unsupervised ML to highlight potential signs of cognitive decline, and presents the results in a simple UI. It also includes an offline training pipeline to build the model from training data.
File Structure Analysis
Top-level files:
• Dockerfile – containerizes the app and its dependencies
• app.py – Streamlit front-end for real-time inference
• config.py – central configuration for paths, model parameters, external tools
• requirements.txt – Python libraries, including Streamlit, Whisper, ML packages
• start.ipynb – exploratory notebook
• README.md
Data and models:
• data/Control, data/Dementia – raw WAV files
• data/features_and_transcripts.csv – precomputed features/transcripts from training set
• data/models/models.pkl – serialized ML model
Processing and ML:
• feature_extractor.py – functions to extract acoustic features (pauses, pitch) and invoke Whisper for transcripts, compute linguistic metrics
• train_pipeline.py – batch pipeline: reads raw data, calls feature_extractor, trains unsupervised model, saves models.pkl
• prediction.py – loads models.pkl, given a new audio file runs feature_extractor and returns scores/class labels
• main.py – orchestrates end-to-end inference logic (could be reused by both CLI and app.py)
Key Components
a. Data Layer (raw audio & model storage)
• Raw audio folder (data/Control, data/Dementia)
• Features/Transcripts CSV (intermediate store)
• Model artifact (data/models/models.pkl)
b. Offline Training Pipeline (batch)
• train_pipeline.py
• feature_extractor.py
• External: Whisper STT & ffmpeg
• Output: serialized model
c. Online Inference Service
• Streamlit UI (app.py)
• main.py / prediction.py
• feature_extractor.py
• Model loader (models.pkl)
• External: Whisper STT & ffmpeg
d. Configuration & Environment
• config.py for paths & hyperparameters
• Docker container (Dockerfile) encapsulating all services/dependencies
Relationships and Data Flows
• Offline path: Raw audio → feature_extractor → features.csv → train_pipeline → models.pkl
• Online path: User uploads audio via Streamlit → main.py/prediction.py invokes feature_extractor → model.predict → results returned to UI
Architectural Patterns & Design Principles
• Separation of concerns:
• Data Extraction (feature_extractor)
• Model Training (train_pipeline)
• Inference (prediction/main + UI)
• Batch vs. real-time: clearly delineate the offline training workflow vs. the real-time inference workflow
• Modular reuse: feature_extractor is shared by both training and inference
• Containerization: Docker ensures consistent environment