How zmills1212/rap-transcriber works
Rap Transcriber turns audio into rap-aware lyrics, optionally using reference lyrics to improve alignment and confidence. Its primary workflow is coordinated by the CLI: Whisper transcription, rap-specific corrections, then optional lyrics matching. The repository also contains REST/WebSocket serving, a custom model inference stack, training-data preparation and fine-tuning, and evaluation tools. Edges reflect the README pipeline or relationships visible in excerpts and the SOURCE INDEX; unsampled modules are included where documented, but uncertain internal wiring is omitted.
Main entry points: rap_transcribe.py Transcription CLI · server.py REST API · engine.py Inference engine · rap_transcriber.py Rap model
Components 27
Transcription workflow
- Transcription CLI
rap_transcribe.py - Whisper ASR
src/inference/whisper_engine.py - Rap corrections
postprocessor/rap_postprocessor.py - Lyrics matcher
lyrics_matcher/lyrics_matcher.py - Genius lyrics
lyrics_matcher/genius.py - Transcription result
API serving
- API client
src/api/client.py - REST API
src/api/server.py - Rap API server
src/api/rap_server.py - WebSocket API
src/api/websocket.py
Model inference
- Inference engine
src/inference/engine.py - Audio processing
src/data/audio_processor.py - Feature extraction
src/data/feature_extractor.py - Rap model
src/models/rap_transcriber.py - Conformer encoder
src/models/encoder.py - Phoneme head
src/models/phoneme_head.py - Text head
src/models/text_head.py - Text postprocessor
src/inference/postprocessor.py
Training and evaluation
- Data collection
training/data_collector.py - Audio augmentation
training/augment_audio.py - Dataset preparation
training/prepare_dataset.py - Whisper fine-tuning
training/fine_tune.py - Sample manifest
src/data/manifest.py - WER evaluation
evaluation/evaluate.py - Pipeline evaluation
lyrics_matcher/pipeline_eval.py
Other components
- Audio user
- API caller
Connections 26
- Audio user → Transcription CLI: provides audio
rap_transcribe.py - Transcription CLI → Whisper ASR: transcribes with
rap_transcribe.py - Transcription CLI → Rap corrections: corrects with
rap_transcribe.py - Transcription CLI → Lyrics matcher: optionally matches
rap_transcribe.py - Transcription CLI → Genius lyrics: optionally fetches lyrics
rap_transcribe.py - Transcription CLI → Transcription result: returns result
rap_transcribe.py - API caller → API client: requests transcription
src/api/client.py - API client → REST API: calls API
- REST API → Inference engine: transcribes through
src/api/server.py - REST API → Text postprocessor: cleans through
src/api/server.py - REST API → WebSocket API: routes streaming
src/api/server.py - Rap API server → Whisper ASR: transcribes with
src/api/rap_server.py - Rap API server → Rap corrections: corrects with
src/api/rap_server.py - Inference engine → Audio processing: processes audio
src/inference/engine.py - Inference engine → Feature extraction: extracts features
src/inference/engine.py - Inference engine → Rap model: runs model
src/inference/engine.py - Rap model → Conformer encoder: encodes features
src/models/rap_transcriber.py - Rap model → Phoneme head: predicts phonemes
src/models/rap_transcriber.py - Rap model → Text head: predicts text
src/models/rap_transcriber.py - Data collection → Audio augmentation: prepares for augmentation
- Audio augmentation → Dataset preparation: creates variants for
- Dataset preparation → Whisper fine-tuning: prepares training data for
- Data collection → Sample manifest: records samples
- WER evaluation → Sample manifest: evaluates samples from
- Pipeline evaluation → Rap corrections: applies corrections
lyrics_matcher/pipeline_eval.py - Pipeline evaluation → Lyrics matcher: compares with matcher
lyrics_matcher/pipeline_eval.py