# bavithaxa/audio-context-layer architecture

> Architecture diagram of the GitHub repository bavithaxa/audio-context-layer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bavithaxa/audio-context-layer
- Repository: https://github.com/bavithaxa/audio-context-layer

## Overview

This repository explores an interpretable audio question-answering pipeline: synthetic scenes provide audio and ground-truth event manifests; audio perception produces timestamped context; a language model answers questions from that context. The graph also preserves the documented QA-generation and model-evaluation branches. Source excerpts are partial, so links between unsampled modules are included only where README documentation or sampled code supports them.

Main entry points: [`data/context`](https://github.com/bavithaxa/audio-context-layer/tree/main/data/context) (Timestamped context), [`results`](https://github.com/bavithaxa/audio-context-layer/tree/main/results) (Evaluation results), [`scripts/answer_question.py`](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/answer_question.py) (Answer question), [`scripts/evaluate_qa.py`](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_qa.py) (QA evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dataset["Scene dataset"]
  node_composer["Compose scenes<br/>[compose_scenes.py]"]
  node_audio["Scene audio"]
  node_manifest["Event manifests"]
  node_qa_pairs["QA pairs"]
  node_qa_generator["Generate questions<br/>[generate_qa.py]"]
end

subgraph group_perception["Audio perception"]
  node_ast_detector["AST event detector<br/>[detect_events.py]"]
  node_yamnet_context["YAMNet context builder<br/>[build_context.py]"]
  node_context["Timestamped context"]
  node_window_index["Window index<br/>[window_index.json]"]
end

subgraph group_qa["Question answering"]
  node_answerer["Answer question<br/>[answer_question.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_qa_evaluator["QA evaluation<br/>[evaluate_qa.py]"]
  node_qa_scorer["Score answers<br/>[score_qa.py]"]
  node_error_analysis["Error attribution"]
  node_perception_eval["Perception evaluation<br/>[evaluate_ast.py]"]
  node_panns_eval["PANNs evaluation<br/>[evaluate_panns.py]"]
  node_yamnet_eval["YAMNet evaluation<br/>[evaluate_yamnet.py]"]
  node_predictions["QA predictions"]
  node_metrics["Evaluation results"]
end

node_user(("Questioner"))
node_groq{{"Groq language model"}}

node_composer -->|"writes scenes"| node_audio
node_composer -->|"writes events"| node_manifest
node_manifest -->|"supplies events"| node_qa_generator
node_qa_generator -->|"writes questions"| node_qa_pairs
node_audio -->|"processes audio"| node_ast_detector
node_ast_detector -->|"writes detections"| node_context
node_audio -->|"processes audio"| node_yamnet_context
node_yamnet_context -->|"writes context"| node_context
node_user -->|"asks question"| node_answerer
node_context -->|"provides evidence"| node_answerer
node_answerer -.->|"requests answer"| node_groq
node_answerer -->|"returns answer"| node_user
node_qa_pairs -->|"supplies questions"| node_qa_evaluator
node_context -->|"supplies evidence"| node_qa_evaluator
node_qa_evaluator -->|"requests predictions"| node_groq
node_qa_evaluator -->|"writes predictions"| node_predictions
node_predictions -->|"supplies answers"| node_qa_scorer
node_qa_scorer -->|"writes scores"| node_metrics
node_metrics -->|"supplies scores"| node_error_analysis
node_manifest -->|"supplies ground truth"| node_error_analysis
node_context -->|"supplies detections"| node_error_analysis
node_window_index -->|"supplies windows"| node_perception_eval
node_window_index -->|"supplies windows"| node_panns_eval
node_window_index -->|"supplies windows"| node_yamnet_eval
node_perception_eval -->|"writes results"| node_metrics
node_panns_eval -->|"writes results"| node_metrics
node_yamnet_eval -->|"writes results"| node_metrics

click node_composer "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/compose_scenes.py"
click node_audio "https://github.com/bavithaxa/audio-context-layer/tree/main/data"
click node_manifest "https://github.com/bavithaxa/audio-context-layer/tree/main/data/manifests"
click node_qa_pairs "https://github.com/bavithaxa/audio-context-layer/tree/main/data/qa"
click node_qa_generator "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/generate_qa.py"
click node_ast_detector "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/detect_events.py"
click node_yamnet_context "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/build_context.py"
click node_context "https://github.com/bavithaxa/audio-context-layer/tree/main/data/context"
click node_answerer "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/answer_question.py"
click node_qa_evaluator "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_qa.py"
click node_qa_scorer "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/score_qa.py"
click node_error_analysis "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/error_attribution.py"
click node_perception_eval "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_ast.py"
click node_panns_eval "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_panns.py"
click node_yamnet_eval "https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_yamnet.py"
click node_window_index "https://github.com/bavithaxa/audio-context-layer/blob/main/data/processed/window_index.json"
click node_predictions "https://github.com/bavithaxa/audio-context-layer/blob/main/results/qa_predictions.json"
click node_metrics "https://github.com/bavithaxa/audio-context-layer/tree/main/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_composer,node_audio,node_manifest,node_qa_pairs,node_qa_generator toneBlue
class node_ast_detector,node_yamnet_context,node_context,node_window_index toneAmber
class node_answerer toneMint
class node_qa_evaluator,node_qa_scorer,node_error_analysis,node_perception_eval,node_panns_eval,node_yamnet_eval,node_predictions,node_metrics toneRose
class node_user,node_groq toneIndigo
```

## Components

### Scene dataset

- [Compose scenes](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/compose_scenes.py): `scripts/compose_scenes.py`
- [Scene audio](https://github.com/bavithaxa/audio-context-layer/tree/main/data): `data`
- [Event manifests](https://github.com/bavithaxa/audio-context-layer/tree/main/data/manifests): `data/manifests`
- [QA pairs](https://github.com/bavithaxa/audio-context-layer/tree/main/data/qa): `data/qa`
- [Generate questions](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/generate_qa.py): `scripts/generate_qa.py`

### Audio perception

- [AST event detector](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/detect_events.py): `scripts/detect_events.py`
- [YAMNet context builder](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/build_context.py): `scripts/build_context.py`
- [Timestamped context](https://github.com/bavithaxa/audio-context-layer/tree/main/data/context): `data/context`
- [Window index](https://github.com/bavithaxa/audio-context-layer/blob/main/data/processed/window_index.json): `data/processed/window_index.json`

### Question answering

- [Answer question](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/answer_question.py): `scripts/answer_question.py`

### Evaluation

- [QA evaluation](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_qa.py): `scripts/evaluate_qa.py`
- [Score answers](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/score_qa.py): `scripts/score_qa.py`
- [Error attribution](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/error_attribution.py): `scripts/error_attribution.py`
- [Perception evaluation](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_ast.py): `scripts/evaluate_ast.py`
- [PANNs evaluation](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_panns.py): `scripts/evaluate_panns.py`
- [YAMNet evaluation](https://github.com/bavithaxa/audio-context-layer/blob/main/scripts/evaluate_yamnet.py): `scripts/evaluate_yamnet.py`
- [QA predictions](https://github.com/bavithaxa/audio-context-layer/blob/main/results/qa_predictions.json): `results/qa_predictions.json`
- [Evaluation results](https://github.com/bavithaxa/audio-context-layer/tree/main/results): `results`

### Other components

- **Questioner**
- **Groq language model**

## Connections

- Compose scenes → Scene audio: writes scenes
- Compose scenes → Event manifests: writes events
- Event manifests → Generate questions: supplies events
- Generate questions → QA pairs: writes questions
- Scene audio → AST event detector: processes audio
- AST event detector → Timestamped context: writes detections
- Scene audio → YAMNet context builder: processes audio
- YAMNet context builder → Timestamped context: writes context
- Questioner → Answer question: asks question
- Timestamped context → Answer question: provides evidence
- Answer question → Groq language model: requests answer
- Answer question → Questioner: returns answer
- QA pairs → QA evaluation: supplies questions
- Timestamped context → QA evaluation: supplies evidence
- QA evaluation → Groq language model: requests predictions
- QA evaluation → QA predictions: writes predictions
- QA predictions → Score answers: supplies answers
- Score answers → Evaluation results: writes scores
- Evaluation results → Error attribution: supplies scores
- Event manifests → Error attribution: supplies ground truth
- Timestamped context → Error attribution: supplies detections
- Window index → Perception evaluation: supplies windows
- Window index → PANNs evaluation: supplies windows
- Window index → YAMNet evaluation: supplies windows
- Perception evaluation → Evaluation results: writes results
- PANNs evaluation → Evaluation results: writes results
- YAMNet evaluation → Evaluation results: writes results