# sathyapanneer/audiocontextlayer architecture

> Architecture diagram of the GitHub repository sathyapanneer/audiocontextlayer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sathyapanneer/audiocontextlayer
- Repository: https://github.com/sathyapanneer/audiocontextlayer

## Overview

Audio Context Layer is a proof-of-concept for answering natural-language questions about audio. Its main workflow is: a user supplies a clip, the demo analyzes it into detected event intervals and structured context, then routes a question to a deterministic handler that answers from that context. The handlers cover perception, verification, counting, temporal order, and evidence-based reasoning. The graph focuses on runtime behavior; training and dataset-generation scripts are omitted. Source evidence is sampled, so uncertain internal wiring is left out.

Main entry points: [`acl/pipeline.py`](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/pipeline.py) (Inference pipeline), [`app/app.py`](https://github.com/sathyapanneer/audiocontextlayer/blob/main/app/app.py) (Streamlit demo), [`acl/features.py`](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/features.py) (AST features), [`acl/context.py`](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/context.py) (Context builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Demo interface"]
  node_demo["Streamlit demo<br/>[app.py]"]
  node_audio_loader["Audio loading<br/>[app.py]"]
  node_timeline["Timeline visualization<br/>[viz.py]"]
end

subgraph group_perception["Audio perception"]
  node_pipeline["Inference pipeline<br/>[pipeline.py]"]
  node_features["AST features<br/>[features.py]"]
  node_event_head["Event detector<br/>[detector.py]"]
  node_decoder["Event decoder<br/>[decode.py]"]
end

subgraph group_context["Context and reasoning"]
  node_context_builder["Context builder<br/>[context.py]"]
  node_intent["Question router<br/>[intent.py]"]
  node_entities["Entity extraction<br/>[intent.py]"]
  node_handlers["Answer handlers<br/>[answer.py]"]
end

node_user(("User"))
node_audio["Audio clip"]
node_question["Question"]
node_ast["Pretrained AST"]

node_user -->|"uses"| node_demo
node_user -->|"selects or uploads"| node_audio
node_user -->|"asks"| node_question
node_audio -->|"supplies clip"| node_demo
node_demo -->|"loads audio"| node_audio_loader
node_audio_loader -->|"returns waveform"| node_demo
node_demo -->|"requests analysis"| node_pipeline
node_pipeline -->|"extracts cells"| node_features
node_features -->|"runs backbone"| node_ast
node_features -->|"returns features"| node_pipeline
node_pipeline -->|"predicts events"| node_event_head
node_pipeline -->|"decodes probabilities"| node_decoder
node_pipeline -->|"builds context"| node_context_builder
node_context_builder -->|"returns context"| node_pipeline
node_pipeline -->|"returns context"| node_demo
node_demo -->|"plots events"| node_timeline
node_demo -->|"asks question"| node_pipeline
node_pipeline -->|"routes question"| node_intent
node_pipeline -->|"extracts entities"| node_entities
node_pipeline -->|"requests answer"| node_handlers
node_question -->|"enters question"| node_demo
node_intent -->|"returns subtype"| node_pipeline
node_entities -->|"returns entities"| node_pipeline
node_handlers -->|"returns answer"| node_pipeline
node_pipeline -->|"returns answer"| node_demo

click node_demo "https://github.com/sathyapanneer/audiocontextlayer/blob/main/app/app.py"
click node_audio_loader "https://github.com/sathyapanneer/audiocontextlayer/blob/main/app/app.py"
click node_pipeline "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/pipeline.py"
click node_features "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/features.py"
click node_event_head "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/detector.py"
click node_decoder "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/decode.py"
click node_context_builder "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/context.py"
click node_intent "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/intent.py"
click node_entities "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/intent.py"
click node_handlers "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/answer.py"
click node_timeline "https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/viz.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_audio_loader,node_timeline,node_user toneBlue
class node_pipeline,node_features,node_event_head,node_decoder toneAmber
class node_context_builder,node_intent,node_entities,node_handlers toneMint
class node_audio,node_question,node_ast toneIndigo
```

## Components

### Demo interface

- [Streamlit demo](https://github.com/sathyapanneer/audiocontextlayer/blob/main/app/app.py): `app/app.py`
- [Audio loading](https://github.com/sathyapanneer/audiocontextlayer/blob/main/app/app.py): `app/app.py`
- [Timeline visualization](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/viz.py): `acl/viz.py`

### Audio perception

- [Inference pipeline](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/pipeline.py): `acl/pipeline.py`
- [AST features](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/features.py): `acl/features.py`
- [Event detector](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/detector.py): `acl/detector.py`
- [Event decoder](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/decode.py): `acl/decode.py`

### Context and reasoning

- [Context builder](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/context.py): `acl/context.py`
- [Question router](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/intent.py): `acl/intent.py`
- [Entity extraction](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/intent.py): `acl/intent.py`
- [Answer handlers](https://github.com/sathyapanneer/audiocontextlayer/blob/main/acl/answer.py): `acl/answer.py`

### Other components

- **User**
- **Audio clip**
- **Question**
- **Pretrained AST**

## Connections

- User → Streamlit demo: uses
- User → Audio clip: selects or uploads
- User → Question: asks
- Audio clip → Streamlit demo: supplies clip
- Streamlit demo → Audio loading: loads audio
- Audio loading → Streamlit demo: returns waveform
- Streamlit demo → Inference pipeline: requests analysis
- Inference pipeline → AST features: extracts cells
- AST features → Pretrained AST: runs backbone
- AST features → Inference pipeline: returns features
- Inference pipeline → Event detector: predicts events
- Inference pipeline → Event decoder: decodes probabilities
- Inference pipeline → Context builder: builds context
- Context builder → Inference pipeline: returns context
- Inference pipeline → Streamlit demo: returns context
- Streamlit demo → Timeline visualization: plots events
- Streamlit demo → Inference pipeline: asks question
- Inference pipeline → Question router: routes question
- Inference pipeline → Entity extraction: extracts entities
- Inference pipeline → Answer handlers: requests answer
- Question → Streamlit demo: enters question
- Question router → Inference pipeline: returns subtype
- Entity extraction → Inference pipeline: returns entities
- Answer handlers → Inference pipeline: returns answer
- Inference pipeline → Streamlit demo: returns answer