How sathyapanneer/audiocontextlayer works
Audio Context Layer is a proof-of-concept for answering natural-language questions about audio. Its main workflow is: a user supplies a clip, the demo analyzes it into detected event intervals and structured context, then routes a question to a deterministic handler that answers from that context. The handlers cover perception, verification, counting, temporal order, and evidence-based reasoning. The graph focuses on runtime behavior; training and dataset-generation scripts are omitted. Source evidence is sampled, so uncertain internal wiring is left out.
Main entry points: pipeline.py Inference pipeline · app.py Streamlit demo · features.py AST features · context.py Context builder
Components 15
Demo interface
- Streamlit demo
app/app.py - Audio loading
app/app.py - Timeline visualization
acl/viz.py
Audio perception
- Inference pipeline
acl/pipeline.py - AST features
acl/features.py - Event detector
acl/detector.py - Event decoder
acl/decode.py
Context and reasoning
- Context builder
acl/context.py - Question router
acl/intent.py - Entity extraction
acl/intent.py - Answer handlers
acl/answer.py
Other components
- User
- Audio clip
- Question
- Pretrained AST
Connections 25
- User → Streamlit demo: uses
- User → Audio clip: selects or uploads
- User → Question: asks
- Audio clip → Streamlit demo: supplies clip
- Streamlit demo → Audio loading: loads audio
- Audio loading → Streamlit demo: returns waveform
- Streamlit demo → Inference pipeline: requests analysis
- Inference pipeline → AST features: extracts cells
- AST features → Pretrained AST: runs backbone
- AST features → Inference pipeline: returns features
- Inference pipeline → Event detector: predicts events
- Inference pipeline → Event decoder: decodes probabilities
- Inference pipeline → Context builder: builds context
- Context builder → Inference pipeline: returns context
- Inference pipeline → Streamlit demo: returns context
- Streamlit demo → Timeline visualization: plots events
- Streamlit demo → Inference pipeline: asks question
- Inference pipeline → Question router: routes question
- Inference pipeline → Entity extraction: extracts entities
- Inference pipeline → Answer handlers: requests answer
- Question → Streamlit demo: enters question
- Question router → Inference pipeline: returns subtype
- Entity extraction → Inference pipeline: returns entities
- Answer handlers → Inference pipeline: returns answer
- Inference pipeline → Streamlit demo: returns answer