How charishmav/audio-context-layer works
Audio Context Layer is a Streamlit proof of concept for turning an uploaded recording into timestamped activity context and answering questions grounded in that context. The main path loads audio, detects active regions, optionally labels them with a synthetic-tone classifier, and routes questions to counting, temporal, perceptual, environment, or causal rules. Users can also replace inferred events with known annotations. The README documents a separate ESC-50 clip-classification experiment; it does not add timestamped events to the app. App excerpts are partial, so uncertain wiring is omitted.
Main entry points: app.py Streamlit app · audio_engine.py Context builder · classifier.py Tone classifier · synthetic_data.py Synthetic data generator
Components 12
User interface
- Streamlit app
app.py
Audio context
- Audio loading
src/audio_engine.py - Activity detection
src/audio_engine.py - Context builder
src/audio_engine.py
Question answering
- Rule-based QA
src/audio_engine.py - QA annotations
data/annotations/qa_dataset.jsonl
Classification experiments
- Tone classifier
src/classifier.py - Tone model
models - Synthetic data generator
src/synthetic_data.py - ESC-50 audio
data/esc50 - ESC-50 classifier
scripts/train_esc50.py
Other components
- Recording user
Connections 14
- Recording user → Streamlit app: uploads audio
- Recording user → Streamlit app: asks question
- Streamlit app → Audio loading: loads audio
- Streamlit app → Context builder: builds context
- Audio loading → Context builder: provides samples
- Context builder → Activity detection: detects regions
- Context builder → Tone classifier: classifies regions
- Tone classifier → Tone model: loads model
- Streamlit app → Rule-based QA: asks question
- Context builder → Streamlit app: returns events
- Rule-based QA → Streamlit app: returns answer
- Recording user → Streamlit app: enters annotations
- Synthetic data generator → QA annotations: generates QA rows
- ESC-50 classifier → ESC-50 audio: reads clips