How stibbbb/thesis-slm works
This repository develops a small-vocabulary sign-language communication SLM, with a bidirectional runtime pipeline and a separate model-training/evaluation workflow. The principal user flow accepts speech or sign-related text, processes it through normalization, reduction and tokenization, then produces a sentence or gloss output. The tree and excerpts also show PyTorch inference, a later ONNX/Raspberry Pi deployment path, dataset generation, and architecture comparisons. Some connections rely on the SOURCE INDEX’s import references; unsampled behavior is not inferred beyond those references.
Main entry points: slm_pipeline.py Bidirectional pipeline · slm_inference.py PyTorch inference · compare.py Model comparison · slm_api.py Deployment API
Components 21
User interaction
- Speech input
slm_training_v2/speech_input.py
Translation pipeline
- Bidirectional pipeline
slm_training_v2/slm_pipeline.py - Word tokenizer
slm_training_v2/tokenizer.py - Vocabulary pool
slm_training_v2/vocab_pool.py - Gloss and validation
slm_training_v2/onnx_config.py
Model runtime
- Word reduction
slm_training_v2/word_picker.py - Gloss restructuring
slm_training_v2/encoder_distilbert.py - Sentence generation
slm_training_v2/decoder_tinyllama.py - PyTorch inference
slm_training_v2/archive/slm_inference.py
Training and evaluation
- Dataset generation
slm_training_v2/generate_dataset_v4.py - Word-label creation
slm_training_v2/make_labels.py - Model comparison
slm_training_v2/training_comparison/compare.py - MobileBERT encoder
slm_training_v2/training_comparison/encoder_mobilebert.py - TinyBERT encoder
slm_training_v2/training_comparison/encoder_tinybert.py - BabyLlama decoder
slm_training_v2/training_comparison/decoder_babyllama.py - MobiLlama decoder
slm_training_v2/training_comparison/decoder_mobillama.py
Edge deployment
- Deployment API
slm_training_v2/pi_deployment_later/slm_api.py - ONNX inference
slm_training_v2/pi_deployment_later/onnx_inference.py - Conversation session
slm_training_v2/pi_deployment_later/conversation_session.py
Other components
- Hearing user
- Sign user
Connections 27
- Hearing user → Bidirectional pipeline: provides speech or text
slm_training_v2/slm_pipeline.py - Sign user → Bidirectional pipeline: provides signs
slm_training_v2/slm_pipeline.py - Bidirectional pipeline → Speech input: captures speech
slm_training_v2/slm_pipeline.py - Bidirectional pipeline → Word tokenizer: tokenizes words
slm_training_v2/slm_pipeline.py - Word tokenizer → Vocabulary pool: uses vocabulary
slm_training_v2/tokenizer.py - Bidirectional pipeline → Gloss and validation: applies gloss rules
slm_training_v2/slm_pipeline.py - Bidirectional pipeline → Word reduction: reduces words
slm_training_v2/slm_pipeline.py - Bidirectional pipeline → Sentence generation: generates sentences
slm_training_v2/slm_pipeline.py - Bidirectional pipeline → Gloss restructuring: restructures glosses
slm_training_v2/slm_pipeline.py - PyTorch inference → Word tokenizer: tokenizes input
slm_training_v2/archive/slm_inference.py - PyTorch inference → Vocabulary pool: loads vocabulary
slm_training_v2/archive/slm_inference.py - PyTorch inference → Gloss and validation: validates glosses
slm_training_v2/archive/slm_inference.py - PyTorch inference → Gloss restructuring: loads encoder
slm_training_v2/archive/slm_inference.py - PyTorch inference → Sentence generation: loads decoder
slm_training_v2/archive/slm_inference.py - PyTorch inference → Word reduction: loads picker
slm_training_v2/archive/slm_inference.py - Word reduction → Gloss restructuring: reuses transformer components
slm_training_v2/word_picker.py - Dataset generation → Vocabulary pool: uses vocabulary
slm_training_v2/generate_dataset_v4.py - Model comparison → MobileBERT encoder: compares architecture
slm_training_v2/training_comparison/compare.py - Model comparison → TinyBERT encoder: compares architecture
slm_training_v2/training_comparison/compare.py - Model comparison → BabyLlama decoder: compares architecture
slm_training_v2/training_comparison/compare.py - Model comparison → MobiLlama decoder: compares architecture
slm_training_v2/training_comparison/compare.py - Deployment API → ONNX inference: dispatches inference
slm_training_v2/pi_deployment_later/slm_api.py - Deployment API → Word tokenizer: uses tokenizer
slm_training_v2/pi_deployment_later/slm_api.py - Deployment API → Gloss and validation: uses validation
slm_training_v2/pi_deployment_later/slm_api.py - Conversation session → Deployment API: calls API
slm_training_v2/pi_deployment_later/conversation_session.py - Conversation session → Vocabulary pool: uses vocabulary
slm_training_v2/pi_deployment_later/conversation_session.py - Conversation session → Gloss and validation: uses configuration
slm_training_v2/pi_deployment_later/conversation_session.py