# jack-zc8/m3av-dataset architecture

> Architecture diagram of the GitHub repository jack-zc8/m3av-dataset, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jack-zc8/m3av-dataset
- Repository: https://github.com/jack-zc8/m3av-dataset

## Overview

M³AV is an academic-lecture dataset release with sample data and benchmark implementations. Its core workflow is to provide synchronized speech, slide/OCR and (for some computer-science videos) paper data for research; the demo script additionally aligns slide OCR text with timestamped spoken words. Benchmark branches cover ASR, contextual ASR, summarization and spontaneous TTS. The README establishes the dataset modalities and benchmark scope; only the demo alignment script was sampled, so other benchmark internals and their wiring are represented conservatively.

Main entry points: [`demo/get_seg_speech_ocr.py`](https://github.com/jack-zc8/m3av-dataset/blob/main/demo/get_seg_speech_ocr.py) (OCR-speech aligner), [`demo/speech`](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/speech) (Timestamped speech), [`demo/ocr`](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/ocr) (Slide OCR), [`download`](https://github.com/jack-zc8/m3av-dataset/tree/main/download) (Download release)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dataset["Dataset release"]
  node_download_release["Download release"]
  node_speech_data["Timestamped speech"]
  node_slide_data["Slide OCR"]
  node_paper_data["Paper sentences"]
  node_video_sample["Video reference"]
end

subgraph group_demo["Demo alignment"]
  node_aligner["OCR-speech aligner"]
  node_aligned_output["Segmented OCR speech"]
end

subgraph group_benchmarks["Research benchmarks"]
  node_asr["ASR benchmark"]
  node_casr["Contextual ASR"]
  node_ssg_data["SSG dataset tools"]
  node_ssg_models["SSG generation"]
  node_ssg_scoring["SSG scoring"]
  node_tts["Spontaneous TTS"]
end

node_researcher(("Researcher"))
node_lecture_sources(("Lecture videos"))

node_researcher -->|"downloads"| node_download_release
node_lecture_sources -.->|"references"| node_video_sample
node_researcher -->|"uses"| node_speech_data
node_researcher -->|"uses"| node_slide_data
node_researcher -->|"uses"| node_paper_data
node_slide_data -->|"reads OCR"| node_aligner
node_speech_data -->|"reads timestamps"| node_aligner
node_aligner -->|"writes segments"| node_aligned_output
node_researcher -->|"runs benchmark"| node_asr
node_researcher -->|"runs benchmark"| node_casr
node_researcher -->|"builds data"| node_ssg_data
node_researcher -->|"runs generation"| node_ssg_models
node_researcher -->|"evaluates output"| node_ssg_scoring
node_researcher -->|"runs benchmark"| node_tts

click node_download_release "https://github.com/jack-zc8/m3av-dataset/tree/main/download"
click node_speech_data "https://github.com/jack-zc8/m3av-dataset/tree/main/demo/speech"
click node_slide_data "https://github.com/jack-zc8/m3av-dataset/tree/main/demo/ocr"
click node_paper_data "https://github.com/jack-zc8/m3av-dataset/tree/main/demo/paper"
click node_video_sample "https://github.com/jack-zc8/m3av-dataset/tree/main/demo/raw"
click node_aligner "https://github.com/jack-zc8/m3av-dataset/blob/main/demo/get_seg_speech_ocr.py"
click node_aligned_output "https://github.com/jack-zc8/m3av-dataset/tree/main/demo/ocr"
click node_asr "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/ASR"
click node_casr "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/CASR"
click node_ssg_data "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG/get_ssg_dataset"
click node_ssg_models "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG"
click node_ssg_scoring "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG/score"
click node_tts "https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/Spontaneous_TTS"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_download_release,node_speech_data,node_slide_data,node_paper_data,node_video_sample toneBlue
class node_aligner,node_aligned_output toneAmber
class node_asr,node_casr,node_ssg_data,node_ssg_models,node_ssg_scoring,node_tts toneMint
class node_researcher,node_lecture_sources toneIndigo
```

## Components

### Dataset release

- [Download release](https://github.com/jack-zc8/m3av-dataset/tree/main/download): `download`
- [Timestamped speech](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/speech): `demo/speech`
- [Slide OCR](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/ocr): `demo/ocr`
- [Paper sentences](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/paper): `demo/paper`
- [Video reference](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/raw): `demo/raw`

### Demo alignment

- [OCR-speech aligner](https://github.com/jack-zc8/m3av-dataset/blob/main/demo/get_seg_speech_ocr.py): `demo/get_seg_speech_ocr.py`
- [Segmented OCR speech](https://github.com/jack-zc8/m3av-dataset/tree/main/demo/ocr): `demo/ocr`

### Research benchmarks

- [ASR benchmark](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/ASR): `benchmarks/ASR`
- [Contextual ASR](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/CASR): `benchmarks/CASR`
- [SSG dataset tools](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG/get_ssg_dataset): `benchmarks/SSG/get_ssg_dataset`
- [SSG generation](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG): `benchmarks/SSG`
- [SSG scoring](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/SSG/score): `benchmarks/SSG/score`
- [Spontaneous TTS](https://github.com/jack-zc8/m3av-dataset/tree/main/benchmarks/Spontaneous_TTS): `benchmarks/Spontaneous_TTS`

### Other components

- **Researcher**
- **Lecture videos**

## Connections

- Researcher → Download release: downloads
- Lecture videos → Video reference: references
- Researcher → Timestamped speech: uses
- Researcher → Slide OCR: uses
- Researcher → Paper sentences: uses
- Slide OCR → OCR-speech aligner: reads OCR
- Timestamped speech → OCR-speech aligner: reads timestamps
- OCR-speech aligner → Segmented OCR speech: writes segments
- Researcher → ASR benchmark: runs benchmark
- Researcher → Contextual ASR: runs benchmark
- Researcher → SSG dataset tools: builds data
- Researcher → SSG generation: runs generation
- Researcher → SSG scoring: evaluates output
- Researcher → Spontaneous TTS: runs benchmark