# multysquid/kitsune-transcribe architecture

> Architecture diagram of the GitHub repository multysquid/kitsune-transcribe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/multysquid/kitsune-transcribe
- Repository: https://github.com/multysquid/kitsune-transcribe

## Overview

Kitsune-Transcribe builds and evaluates compact Japanese speech-recognition students by preparing audio data, generating teacher labels, selecting examples, pruning a teacher model, and distilling and evaluating the result. Its documented study workflow adds Parakeet-based CTC students and pre-registered comparisons. The graph follows the README’s stated artifact flow and documented script responsibilities; sampled source excerpts are partial, so internal call wiring not established by them is omitted.

Main entry points: [`kitsune/store.py`](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/store.py) (Label stores), [`scripts/01_prepare_data.py`](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/01_prepare_data.py) (Prepare shards), [`kitsune/student.py`](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/student.py) (Student checkpoint), [`scripts/02_teacher_pass.py`](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02_teacher_pass.py) (Teacher labeling)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data_labels["Data and labels"]
  node_prepare["Prepare shards<br/>[01_prepare_data.py]"]
  node_teacher_pass["Teacher labeling<br/>[02_teacher_pass.py]"]
  node_parakeet_pass["Parakeet labeling"]
  node_second_opinion["Second opinion"]
  node_select["Select examples<br/>[make_selection.py]"]
  node_store[("Label stores<br/>[store.py]")]
end

subgraph group_students["Student models"]
  node_student_builder["Prune teacher"]
  node_ctc_builder["Build CTC student"]
  node_student_model["Student checkpoint<br/>[student.py]"]
  node_ctc_student["CTC student<br/>[ctc_student.py]"]
end

subgraph group_training_eval["Training and evaluation"]
  node_trainset["Training data<br/>[trainset.py]"]
  node_distill["Distill student<br/>[04_distill.py]"]
  node_features["Audio features<br/>[features.py]"]
  node_generation["Speech generation<br/>[generation.py]"]
  node_evaluate["Evaluate checkpoint<br/>[05_evaluate.py]"]
  node_eval_logic["Evaluation metrics<br/>[evaluate.py]"]
end

subgraph group_study_ops["Study and operations"]
  node_prereg["Study rules<br/>[prereg.py]"]
  node_study_stats["Study statistics<br/>[study_stats.py]"]
  node_study_report["Study report<br/>[study_report.py]"]
  node_label_box["Label box control<br/>[label.py]"]
end

node_user(("Researcher"))
node_datasets[("Speech datasets")]
node_teacher_model["Cohere teacher"]
node_parakeet_model["Parakeet teacher"]

node_user -->|"starts data prep"| node_prepare
node_datasets -->|"provides audio"| node_prepare
node_prepare -->|"writes shards"| node_teacher_pass
node_teacher_pass -->|"runs inference"| node_teacher_model
node_teacher_pass -->|"writes teacher labels"| node_store
node_prepare -->|"provides shards"| node_parakeet_pass
node_parakeet_pass -->|"runs inference"| node_parakeet_model
node_parakeet_pass -->|"writes Parakeet labels"| node_store
node_prepare -->|"provides utterances"| node_second_opinion
node_second_opinion -->|"writes hypotheses"| node_store
node_store -->|"supplies labels"| node_select
node_select -->|"provides selection"| node_student_builder
node_teacher_model -->|"initializes student"| node_student_builder
node_student_builder -->|"writes checkpoint"| node_student_model
node_parakeet_model -->|"provides architecture"| node_ctc_builder
node_ctc_builder -->|"writes checkpoint"| node_ctc_student
node_store -->|"provides targets"| node_trainset
node_student_model -->|"provides model metadata"| node_trainset
node_trainset -->|"provides batches"| node_distill
node_student_model -->|"loads checkpoint"| node_distill
node_distill -->|"saves checkpoints"| node_student_model
node_store -->|"provides audio"| node_features
node_features -->|"provides features"| node_generation
node_student_model -->|"loads checkpoint"| node_evaluate
node_store -->|"provides eval data"| node_evaluate
node_evaluate -->|"uses scoring"| node_eval_logic
node_evaluate -->|"writes eval tables"| node_study_report
node_prereg -->|"defines comparisons"| node_study_report
node_study_report -->|"uses statistics"| node_study_stats
node_label_box -->|"runs labeling"| node_teacher_pass
node_label_box -->|"runs labeling"| node_parakeet_pass
node_label_box -->|"runs labeling"| node_second_opinion

click node_prepare "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/01_prepare_data.py"
click node_teacher_pass "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02_teacher_pass.py"
click node_parakeet_pass "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02p_parakeet_pass.py"
click node_second_opinion "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02b_second_opinion.py"
click node_select "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/make_selection.py"
click node_store "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/store.py"
click node_student_builder "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/03_build_student.py"
click node_ctc_builder "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/03c_build_ctc_student.py"
click node_student_model "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/student.py"
click node_ctc_student "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/ctc_student.py"
click node_trainset "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/trainset.py"
click node_distill "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/04_distill.py"
click node_features "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/features.py"
click node_generation "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/generation.py"
click node_evaluate "https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/05_evaluate.py"
click node_eval_logic "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/evaluate.py"
click node_prereg "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/prereg.py"
click node_study_stats "https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/study_stats.py"
click node_study_report "https://github.com/multysquid/kitsune-transcribe/blob/main/tools/study_report.py"
click node_label_box "https://github.com/multysquid/kitsune-transcribe/blob/main/vast/label.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare,node_teacher_pass,node_parakeet_pass,node_second_opinion,node_select,node_store toneBlue
class node_student_builder,node_ctc_builder,node_student_model,node_ctc_student,node_datasets toneAmber
class node_trainset,node_distill,node_features,node_generation,node_evaluate,node_eval_logic toneMint
class node_prereg,node_study_stats,node_study_report,node_label_box toneRose
class node_user,node_teacher_model,node_parakeet_model toneIndigo
```

## Components

### Data and labels

- [Prepare shards](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/01_prepare_data.py): `scripts/01_prepare_data.py`
- [Teacher labeling](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02_teacher_pass.py): `scripts/02_teacher_pass.py`
- [Parakeet labeling](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02p_parakeet_pass.py): `scripts/02p_parakeet_pass.py`
- [Second opinion](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/02b_second_opinion.py): `scripts/02b_second_opinion.py`
- [Select examples](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/make_selection.py): `scripts/make_selection.py`
- [Label stores](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/store.py): `kitsune/store.py`

### Student models

- [Prune teacher](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/03_build_student.py): `scripts/03_build_student.py`
- [Build CTC student](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/03c_build_ctc_student.py): `scripts/03c_build_ctc_student.py`
- [Student checkpoint](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/student.py): `kitsune/student.py`
- [CTC student](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/ctc_student.py): `kitsune/ctc_student.py`

### Training and evaluation

- [Training data](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/trainset.py): `kitsune/trainset.py`
- [Distill student](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/04_distill.py): `scripts/04_distill.py`
- [Audio features](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/features.py): `kitsune/features.py`
- [Speech generation](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/generation.py): `kitsune/generation.py`
- [Evaluate checkpoint](https://github.com/multysquid/kitsune-transcribe/blob/main/scripts/05_evaluate.py): `scripts/05_evaluate.py`
- [Evaluation metrics](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/evaluate.py): `kitsune/evaluate.py`

### Study and operations

- [Study rules](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/prereg.py): `kitsune/prereg.py`
- [Study statistics](https://github.com/multysquid/kitsune-transcribe/blob/main/kitsune/study_stats.py): `kitsune/study_stats.py`
- [Study report](https://github.com/multysquid/kitsune-transcribe/blob/main/tools/study_report.py): `tools/study_report.py`
- [Label box control](https://github.com/multysquid/kitsune-transcribe/blob/main/vast/label.py): `vast/label.py`

### Other components

- **Researcher**
- **Speech datasets**
- **Cohere teacher**
- **Parakeet teacher**

## Connections

- Researcher → Prepare shards: starts data prep
- Speech datasets → Prepare shards: provides audio
- Prepare shards → Teacher labeling: writes shards
- Teacher labeling → Cohere teacher: runs inference
- Teacher labeling → Label stores: writes teacher labels
- Prepare shards → Parakeet labeling: provides shards
- Parakeet labeling → Parakeet teacher: runs inference
- Parakeet labeling → Label stores: writes Parakeet labels
- Prepare shards → Second opinion: provides utterances
- Second opinion → Label stores: writes hypotheses
- Label stores → Select examples: supplies labels
- Select examples → Prune teacher: provides selection
- Cohere teacher → Prune teacher: initializes student
- Prune teacher → Student checkpoint: writes checkpoint
- Parakeet teacher → Build CTC student: provides architecture
- Build CTC student → CTC student: writes checkpoint
- Label stores → Training data: provides targets
- Student checkpoint → Training data: provides model metadata
- Training data → Distill student: provides batches
- Student checkpoint → Distill student: loads checkpoint
- Distill student → Student checkpoint: saves checkpoints
- Label stores → Audio features: provides audio
- Audio features → Speech generation: provides features
- Student checkpoint → Evaluate checkpoint: loads checkpoint
- Label stores → Evaluate checkpoint: provides eval data
- Evaluate checkpoint → Evaluation metrics: uses scoring
- Evaluate checkpoint → Study report: writes eval tables
- Study rules → Study report: defines comparisons
- Study report → Study statistics: uses statistics
- Label box control → Teacher labeling: runs labeling
- Label box control → Parakeet labeling: runs labeling
- Label box control → Second opinion: runs labeling