# zmills1212/rap-transcriber architecture

> Architecture diagram of the GitHub repository zmills1212/rap-transcriber, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zmills1212/rap-transcriber
- Repository: https://github.com/zmills1212/rap-transcriber

## Overview

Rap Transcriber turns audio into rap-aware lyrics, optionally using reference lyrics to improve alignment and confidence. Its primary workflow is coordinated by the CLI: Whisper transcription, rap-specific corrections, then optional lyrics matching. The repository also contains REST/WebSocket serving, a custom model inference stack, training-data preparation and fine-tuning, and evaluation tools. Edges reflect the README pipeline or relationships visible in excerpts and the SOURCE INDEX; unsampled modules are included where documented, but uncertain internal wiring is omitted.

Main entry points: [`rap_transcribe.py`](https://github.com/zmills1212/rap-transcriber/blob/main/rap_transcribe.py) (Transcription CLI), [`src/api/server.py`](https://github.com/zmills1212/rap-transcriber/blob/main/src/api/server.py) (REST API), [`src/inference/engine.py`](https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/engine.py) (Inference engine), [`src/models/rap_transcriber.py`](https://github.com/zmills1212/rap-transcriber/blob/main/src/models/rap_transcriber.py) (Rap model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_transcription["Transcription workflow"]
  node_cli["Transcription CLI<br/>[rap_transcribe.py]"]
  node_whisper["Whisper ASR<br/>[whisper_engine.py]"]
  node_rapfix["Rap corrections"]
  node_matcher["Lyrics matcher<br/>[lyrics_matcher.py]"]
  node_genius["Genius lyrics<br/>[genius.py]"]
  node_transcript["Transcription result"]
end

subgraph group_serving["API serving"]
  node_api_client["API client<br/>[client.py]"]
  node_api_server["REST API<br/>[server.py]"]
  node_rap_server["Rap API server<br/>[rap_server.py]"]
  node_websocket["WebSocket API<br/>[websocket.py]"]
end

subgraph group_model["Model inference"]
  node_engine["Inference engine<br/>[engine.py]"]
  node_audio["Audio processing<br/>[audio_processor.py]"]
  node_features["Feature extraction"]
  node_transcriber["Rap model<br/>[rap_transcriber.py]"]
  node_encoder["Conformer encoder<br/>[encoder.py]"]
  node_phoneme["Phoneme head<br/>[phoneme_head.py]"]
  node_texthead["Text head<br/>[text_head.py]"]
  node_textclean["Text postprocessor<br/>[postprocessor.py]"]
end

subgraph group_training["Training and evaluation"]
  node_collector["Data collection<br/>[data_collector.py]"]
  node_augment["Audio augmentation<br/>[augment_audio.py]"]
  node_dataset["Dataset preparation<br/>[prepare_dataset.py]"]
  node_finetune["Whisper fine-tuning<br/>[fine_tune.py]"]
  node_manifest[("Sample manifest<br/>[manifest.py]")]
  node_evaluate["WER evaluation<br/>[evaluate.py]"]
  node_pipeline_eval["Pipeline evaluation<br/>[pipeline_eval.py]"]
end

node_user(("Audio user"))
node_api_user(("API caller"))

node_user -->|"provides audio"| node_cli
node_cli -->|"transcribes with"| node_whisper
node_cli -->|"corrects with"| node_rapfix
node_cli -.->|"optionally matches"| node_matcher
node_cli -.->|"optionally fetches lyrics"| node_genius
node_cli -->|"returns result"| node_transcript
node_api_user -->|"requests transcription"| node_api_client
node_api_client -.->|"calls API"| node_api_server
node_api_server -->|"transcribes through"| node_engine
node_api_server -->|"cleans through"| node_textclean
node_api_server -->|"routes streaming"| node_websocket
node_rap_server -->|"transcribes with"| node_whisper
node_rap_server -->|"corrects with"| node_rapfix
node_engine -->|"processes audio"| node_audio
node_engine -->|"extracts features"| node_features
node_engine -->|"runs model"| node_transcriber
node_transcriber -->|"encodes features"| node_encoder
node_transcriber -->|"predicts phonemes"| node_phoneme
node_transcriber -->|"predicts text"| node_texthead
node_collector -.->|"prepares for augmentation"| node_augment
node_augment -.->|"creates variants for"| node_dataset
node_dataset -.->|"prepares training data for"| node_finetune
node_collector -.->|"records samples"| node_manifest
node_evaluate -.->|"evaluates samples from"| node_manifest
node_pipeline_eval -->|"applies corrections"| node_rapfix
node_pipeline_eval -->|"compares with matcher"| node_matcher

click node_cli "https://github.com/zmills1212/rap-transcriber/blob/main/rap_transcribe.py"
click node_whisper "https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/whisper_engine.py"
click node_rapfix "https://github.com/zmills1212/rap-transcriber/blob/main/postprocessor/rap_postprocessor.py"
click node_matcher "https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/lyrics_matcher.py"
click node_genius "https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/genius.py"
click node_api_client "https://github.com/zmills1212/rap-transcriber/blob/main/src/api/client.py"
click node_api_server "https://github.com/zmills1212/rap-transcriber/blob/main/src/api/server.py"
click node_rap_server "https://github.com/zmills1212/rap-transcriber/blob/main/src/api/rap_server.py"
click node_websocket "https://github.com/zmills1212/rap-transcriber/blob/main/src/api/websocket.py"
click node_engine "https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/engine.py"
click node_audio "https://github.com/zmills1212/rap-transcriber/blob/main/src/data/audio_processor.py"
click node_features "https://github.com/zmills1212/rap-transcriber/blob/main/src/data/feature_extractor.py"
click node_transcriber "https://github.com/zmills1212/rap-transcriber/blob/main/src/models/rap_transcriber.py"
click node_encoder "https://github.com/zmills1212/rap-transcriber/blob/main/src/models/encoder.py"
click node_phoneme "https://github.com/zmills1212/rap-transcriber/blob/main/src/models/phoneme_head.py"
click node_texthead "https://github.com/zmills1212/rap-transcriber/blob/main/src/models/text_head.py"
click node_textclean "https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/postprocessor.py"
click node_collector "https://github.com/zmills1212/rap-transcriber/blob/main/training/data_collector.py"
click node_augment "https://github.com/zmills1212/rap-transcriber/blob/main/training/augment_audio.py"
click node_dataset "https://github.com/zmills1212/rap-transcriber/blob/main/training/prepare_dataset.py"
click node_finetune "https://github.com/zmills1212/rap-transcriber/blob/main/training/fine_tune.py"
click node_manifest "https://github.com/zmills1212/rap-transcriber/blob/main/src/data/manifest.py"
click node_evaluate "https://github.com/zmills1212/rap-transcriber/blob/main/evaluation/evaluate.py"
click node_pipeline_eval "https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/pipeline_eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_whisper,node_rapfix,node_matcher,node_genius,node_transcript,node_user toneBlue
class node_api_client,node_api_server,node_rap_server,node_websocket toneAmber
class node_engine,node_audio,node_features,node_transcriber,node_encoder,node_phoneme,node_texthead,node_textclean,node_api_user toneMint
class node_collector,node_augment,node_dataset,node_finetune,node_manifest,node_evaluate,node_pipeline_eval toneRose
```

## Components

### Transcription workflow

- [Transcription CLI](https://github.com/zmills1212/rap-transcriber/blob/main/rap_transcribe.py): `rap_transcribe.py`
- [Whisper ASR](https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/whisper_engine.py): `src/inference/whisper_engine.py`
- [Rap corrections](https://github.com/zmills1212/rap-transcriber/blob/main/postprocessor/rap_postprocessor.py): `postprocessor/rap_postprocessor.py`
- [Lyrics matcher](https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/lyrics_matcher.py): `lyrics_matcher/lyrics_matcher.py`
- [Genius lyrics](https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/genius.py): `lyrics_matcher/genius.py`
- **Transcription result**

### API serving

- [API client](https://github.com/zmills1212/rap-transcriber/blob/main/src/api/client.py): `src/api/client.py`
- [REST API](https://github.com/zmills1212/rap-transcriber/blob/main/src/api/server.py): `src/api/server.py`
- [Rap API server](https://github.com/zmills1212/rap-transcriber/blob/main/src/api/rap_server.py): `src/api/rap_server.py`
- [WebSocket API](https://github.com/zmills1212/rap-transcriber/blob/main/src/api/websocket.py): `src/api/websocket.py`

### Model inference

- [Inference engine](https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/engine.py): `src/inference/engine.py`
- [Audio processing](https://github.com/zmills1212/rap-transcriber/blob/main/src/data/audio_processor.py): `src/data/audio_processor.py`
- [Feature extraction](https://github.com/zmills1212/rap-transcriber/blob/main/src/data/feature_extractor.py): `src/data/feature_extractor.py`
- [Rap model](https://github.com/zmills1212/rap-transcriber/blob/main/src/models/rap_transcriber.py): `src/models/rap_transcriber.py`
- [Conformer encoder](https://github.com/zmills1212/rap-transcriber/blob/main/src/models/encoder.py): `src/models/encoder.py`
- [Phoneme head](https://github.com/zmills1212/rap-transcriber/blob/main/src/models/phoneme_head.py): `src/models/phoneme_head.py`
- [Text head](https://github.com/zmills1212/rap-transcriber/blob/main/src/models/text_head.py): `src/models/text_head.py`
- [Text postprocessor](https://github.com/zmills1212/rap-transcriber/blob/main/src/inference/postprocessor.py): `src/inference/postprocessor.py`

### Training and evaluation

- [Data collection](https://github.com/zmills1212/rap-transcriber/blob/main/training/data_collector.py): `training/data_collector.py`
- [Audio augmentation](https://github.com/zmills1212/rap-transcriber/blob/main/training/augment_audio.py): `training/augment_audio.py`
- [Dataset preparation](https://github.com/zmills1212/rap-transcriber/blob/main/training/prepare_dataset.py): `training/prepare_dataset.py`
- [Whisper fine-tuning](https://github.com/zmills1212/rap-transcriber/blob/main/training/fine_tune.py): `training/fine_tune.py`
- [Sample manifest](https://github.com/zmills1212/rap-transcriber/blob/main/src/data/manifest.py): `src/data/manifest.py`
- [WER evaluation](https://github.com/zmills1212/rap-transcriber/blob/main/evaluation/evaluate.py): `evaluation/evaluate.py`
- [Pipeline evaluation](https://github.com/zmills1212/rap-transcriber/blob/main/lyrics_matcher/pipeline_eval.py): `lyrics_matcher/pipeline_eval.py`

### Other components

- **Audio user**
- **API caller**

## Connections

- Audio user → Transcription CLI: provides audio (evidence: `rap_transcribe.py`)
- Transcription CLI → Whisper ASR: transcribes with (evidence: `rap_transcribe.py`)
- Transcription CLI → Rap corrections: corrects with (evidence: `rap_transcribe.py`)
- Transcription CLI → Lyrics matcher: optionally matches (evidence: `rap_transcribe.py`)
- Transcription CLI → Genius lyrics: optionally fetches lyrics (evidence: `rap_transcribe.py`)
- Transcription CLI → Transcription result: returns result (evidence: `rap_transcribe.py`)
- API caller → API client: requests transcription (evidence: `src/api/client.py`)
- API client → REST API: calls API
- REST API → Inference engine: transcribes through (evidence: `src/api/server.py`)
- REST API → Text postprocessor: cleans through (evidence: `src/api/server.py`)
- REST API → WebSocket API: routes streaming (evidence: `src/api/server.py`)
- Rap API server → Whisper ASR: transcribes with (evidence: `src/api/rap_server.py`)
- Rap API server → Rap corrections: corrects with (evidence: `src/api/rap_server.py`)
- Inference engine → Audio processing: processes audio (evidence: `src/inference/engine.py`)
- Inference engine → Feature extraction: extracts features (evidence: `src/inference/engine.py`)
- Inference engine → Rap model: runs model (evidence: `src/inference/engine.py`)
- Rap model → Conformer encoder: encodes features (evidence: `src/models/rap_transcriber.py`)
- Rap model → Phoneme head: predicts phonemes (evidence: `src/models/rap_transcriber.py`)
- Rap model → Text head: predicts text (evidence: `src/models/rap_transcriber.py`)
- Data collection → Audio augmentation: prepares for augmentation
- Audio augmentation → Dataset preparation: creates variants for
- Dataset preparation → Whisper fine-tuning: prepares training data for
- Data collection → Sample manifest: records samples
- WER evaluation → Sample manifest: evaluates samples from
- Pipeline evaluation → Rap corrections: applies corrections (evidence: `lyrics_matcher/pipeline_eval.py`)
- Pipeline evaluation → Lyrics matcher: compares with matcher (evidence: `lyrics_matcher/pipeline_eval.py`)
