# charishmav/audio-context-layer architecture

> Architecture diagram of the GitHub repository charishmav/audio-context-layer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/charishmav/audio-context-layer
- Repository: https://github.com/charishmav/audio-context-layer

## Overview

Audio Context Layer is a Streamlit proof of concept for turning an uploaded recording into timestamped activity context and answering questions grounded in that context. The main path loads audio, detects active regions, optionally labels them with a synthetic-tone classifier, and routes questions to counting, temporal, perceptual, environment, or causal rules. Users can also replace inferred events with known annotations. The README documents a separate ESC-50 clip-classification experiment; it does not add timestamped events to the app. App excerpts are partial, so uncertain wiring is omitted.

Main entry points: [`app.py`](https://github.com/charishmav/audio-context-layer/blob/main/app.py) (Streamlit app), [`src/audio_engine.py`](https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py) (Context builder), [`src/classifier.py`](https://github.com/charishmav/audio-context-layer/blob/main/src/classifier.py) (Tone classifier), [`src/synthetic_data.py`](https://github.com/charishmav/audio-context-layer/blob/main/src/synthetic_data.py) (Synthetic data generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interface"]
  node_app["Streamlit app<br/>[app.py]"]
end

subgraph group_context["Audio context"]
  node_audio["Audio loading<br/>[audio_engine.py]"]
  node_activity["Activity detection<br/>[audio_engine.py]"]
  node_context_builder["Context builder<br/>[audio_engine.py]"]
end

subgraph group_qa["Question answering"]
  node_qa_rules["Rule-based QA<br/>[audio_engine.py]"]
  node_annotation_data["QA annotations<br/>[qa_dataset.jsonl]"]
end

subgraph group_experiments["Classification experiments"]
  node_tone_classifier["Tone classifier<br/>[classifier.py]"]
  node_model_artifact[("Tone model")]
  node_synthetic_generator["Synthetic data generator<br/>[synthetic_data.py]"]
  node_esc50_data["ESC-50 audio"]
  node_esc50_experiment["ESC-50 classifier<br/>[train_esc50.py]"]
end

node_user(("Recording user"))

node_user -->|"uploads audio"| node_app
node_user -->|"asks question"| node_app
node_app -->|"loads audio"| node_audio
node_app -->|"builds context"| node_context_builder
node_audio -->|"provides samples"| node_context_builder
node_context_builder -->|"detects regions"| node_activity
node_context_builder -.->|"classifies regions"| node_tone_classifier
node_tone_classifier -.->|"loads model"| node_model_artifact
node_app -->|"asks question"| node_qa_rules
node_context_builder -->|"returns events"| node_app
node_qa_rules -->|"returns answer"| node_app
node_user -->|"enters annotations"| node_app
node_synthetic_generator -->|"generates QA rows"| node_annotation_data
node_esc50_experiment -->|"reads clips"| node_esc50_data

click node_app "https://github.com/charishmav/audio-context-layer/blob/main/app.py"
click node_audio "https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py"
click node_activity "https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py"
click node_context_builder "https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py"
click node_tone_classifier "https://github.com/charishmav/audio-context-layer/blob/main/src/classifier.py"
click node_model_artifact "https://github.com/charishmav/audio-context-layer/tree/main/models"
click node_qa_rules "https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py"
click node_annotation_data "https://github.com/charishmav/audio-context-layer/blob/main/data/annotations/qa_dataset.jsonl"
click node_synthetic_generator "https://github.com/charishmav/audio-context-layer/blob/main/src/synthetic_data.py"
click node_esc50_data "https://github.com/charishmav/audio-context-layer/tree/main/data/esc50"
click node_esc50_experiment "https://github.com/charishmav/audio-context-layer/blob/main/scripts/train_esc50.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_user toneBlue
class node_audio,node_activity,node_context_builder toneAmber
class node_qa_rules,node_annotation_data toneMint
class node_tone_classifier,node_model_artifact,node_synthetic_generator,node_esc50_data,node_esc50_experiment toneRose
```

## Components

### User interface

- [Streamlit app](https://github.com/charishmav/audio-context-layer/blob/main/app.py): `app.py`

### Audio context

- [Audio loading](https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py): `src/audio_engine.py`
- [Activity detection](https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py): `src/audio_engine.py`
- [Context builder](https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py): `src/audio_engine.py`

### Question answering

- [Rule-based QA](https://github.com/charishmav/audio-context-layer/blob/main/src/audio_engine.py): `src/audio_engine.py`
- [QA annotations](https://github.com/charishmav/audio-context-layer/blob/main/data/annotations/qa_dataset.jsonl): `data/annotations/qa_dataset.jsonl`

### Classification experiments

- [Tone classifier](https://github.com/charishmav/audio-context-layer/blob/main/src/classifier.py): `src/classifier.py`
- [Tone model](https://github.com/charishmav/audio-context-layer/tree/main/models): `models`
- [Synthetic data generator](https://github.com/charishmav/audio-context-layer/blob/main/src/synthetic_data.py): `src/synthetic_data.py`
- [ESC-50 audio](https://github.com/charishmav/audio-context-layer/tree/main/data/esc50): `data/esc50`
- [ESC-50 classifier](https://github.com/charishmav/audio-context-layer/blob/main/scripts/train_esc50.py): `scripts/train_esc50.py`

### Other components

- **Recording user**

## Connections

- Recording user → Streamlit app: uploads audio
- Recording user → Streamlit app: asks question
- Streamlit app → Audio loading: loads audio
- Streamlit app → Context builder: builds context
- Audio loading → Context builder: provides samples
- Context builder → Activity detection: detects regions
- Context builder → Tone classifier: classifies regions
- Tone classifier → Tone model: loads model
- Streamlit app → Rule-based QA: asks question
- Context builder → Streamlit app: returns events
- Rule-based QA → Streamlit app: returns answer
- Recording user → Streamlit app: enters annotations
- Synthetic data generator → QA annotations: generates QA rows
- ESC-50 classifier → ESC-50 audio: reads clips