# stibbbb/thesis-slm architecture

> Architecture diagram of the GitHub repository stibbbb/thesis-slm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/stibbbb/thesis-slm
- Repository: https://github.com/stibbbb/thesis-slm

## Overview

This repository develops a small-vocabulary sign-language communication SLM, with a bidirectional runtime pipeline and a separate model-training/evaluation workflow. The principal user flow accepts speech or sign-related text, processes it through normalization, reduction and tokenization, then produces a sentence or gloss output. The tree and excerpts also show PyTorch inference, a later ONNX/Raspberry Pi deployment path, dataset generation, and architecture comparisons. Some connections rely on the SOURCE INDEX’s import references; unsampled behavior is not inferred beyond those references.

Main entry points: [`slm_training_v2/slm_pipeline.py`](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/slm_pipeline.py) (Bidirectional pipeline), [`slm_training_v2/archive/slm_inference.py`](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/archive/slm_inference.py) (PyTorch inference), [`slm_training_v2/training_comparison/compare.py`](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/compare.py) (Model comparison), [`slm_training_v2/pi_deployment_later/slm_api.py`](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/slm_api.py) (Deployment API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["User interaction"]
  node_speech_input["Speech input<br/>[speech_input.py]"]
end

subgraph group_pipeline["Translation pipeline"]
  node_pipeline_runtime["Bidirectional pipeline<br/>[slm_pipeline.py]"]
  node_tokenizer["Word tokenizer<br/>[tokenizer.py]"]
  node_vocabulary["Vocabulary pool<br/>[vocab_pool.py]"]
  node_gloss_rules["Gloss and validation<br/>[onnx_config.py]"]
end

subgraph group_models["Model runtime"]
  node_word_picker["Word reduction<br/>[word_picker.py]"]
  node_encoder["Gloss restructuring"]
  node_decoder["Sentence generation"]
  node_torch_inference["PyTorch inference<br/>[slm_inference.py]"]
end

subgraph group_research["Training and evaluation"]
  node_dataset_generation["Dataset generation"]
  node_label_generation["Word-label creation<br/>[make_labels.py]"]
  node_model_comparison["Model comparison<br/>[compare.py]"]
  node_encoder_mobilebert["MobileBERT encoder"]
  node_encoder_tinybert["TinyBERT encoder"]
  node_decoder_babyllama["BabyLlama decoder"]
  node_decoder_mobillama["MobiLlama decoder"]
end

subgraph group_deployment["Edge deployment"]
  node_deployment_api["Deployment API<br/>[slm_api.py]"]
  node_onnx_inference["ONNX inference<br/>[onnx_inference.py]"]
  node_conversation_session["Conversation session"]
end

node_hearing_user(("Hearing user"))
node_sign_user(("Sign user"))

node_hearing_user -->|"provides speech or text"| node_pipeline_runtime
node_sign_user -->|"provides signs"| node_pipeline_runtime
node_pipeline_runtime -->|"captures speech"| node_speech_input
node_pipeline_runtime -->|"tokenizes words"| node_tokenizer
node_tokenizer -->|"uses vocabulary"| node_vocabulary
node_pipeline_runtime -->|"applies gloss rules"| node_gloss_rules
node_pipeline_runtime -->|"reduces words"| node_word_picker
node_pipeline_runtime -->|"generates sentences"| node_decoder
node_pipeline_runtime -->|"restructures glosses"| node_encoder
node_torch_inference -->|"tokenizes input"| node_tokenizer
node_torch_inference -->|"loads vocabulary"| node_vocabulary
node_torch_inference -->|"validates glosses"| node_gloss_rules
node_torch_inference -->|"loads encoder"| node_encoder
node_torch_inference -->|"loads decoder"| node_decoder
node_torch_inference -->|"loads picker"| node_word_picker
node_word_picker -->|"reuses transformer components"| node_encoder
node_dataset_generation -->|"uses vocabulary"| node_vocabulary
node_model_comparison -->|"compares architecture"| node_encoder_mobilebert
node_model_comparison -->|"compares architecture"| node_encoder_tinybert
node_model_comparison -->|"compares architecture"| node_decoder_babyllama
node_model_comparison -->|"compares architecture"| node_decoder_mobillama
node_deployment_api -->|"dispatches inference"| node_onnx_inference
node_deployment_api -->|"uses tokenizer"| node_tokenizer
node_deployment_api -->|"uses validation"| node_gloss_rules
node_conversation_session -->|"calls API"| node_deployment_api
node_conversation_session -->|"uses vocabulary"| node_vocabulary
node_conversation_session -->|"uses configuration"| node_gloss_rules

click node_pipeline_runtime "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/slm_pipeline.py"
click node_speech_input "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/speech_input.py"
click node_tokenizer "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/tokenizer.py"
click node_vocabulary "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/vocab_pool.py"
click node_gloss_rules "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/onnx_config.py"
click node_word_picker "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/word_picker.py"
click node_encoder "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/encoder_distilbert.py"
click node_decoder "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/decoder_tinyllama.py"
click node_torch_inference "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/archive/slm_inference.py"
click node_dataset_generation "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/generate_dataset_v4.py"
click node_label_generation "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/make_labels.py"
click node_model_comparison "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/compare.py"
click node_encoder_mobilebert "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/encoder_mobilebert.py"
click node_encoder_tinybert "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/encoder_tinybert.py"
click node_decoder_babyllama "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/decoder_babyllama.py"
click node_decoder_mobillama "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/decoder_mobillama.py"
click node_deployment_api "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/slm_api.py"
click node_onnx_inference "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/onnx_inference.py"
click node_conversation_session "https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/conversation_session.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_speech_input,node_hearing_user,node_sign_user toneBlue
class node_pipeline_runtime,node_tokenizer,node_vocabulary,node_gloss_rules toneAmber
class node_word_picker,node_encoder,node_decoder,node_torch_inference toneMint
class node_dataset_generation,node_label_generation,node_model_comparison,node_encoder_mobilebert,node_encoder_tinybert,node_decoder_babyllama,node_decoder_mobillama toneRose
class node_deployment_api,node_onnx_inference,node_conversation_session toneIndigo
```

## Components

### User interaction

- [Speech input](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/speech_input.py): `slm_training_v2/speech_input.py`

### Translation pipeline

- [Bidirectional pipeline](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/slm_pipeline.py): `slm_training_v2/slm_pipeline.py`
- [Word tokenizer](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/tokenizer.py): `slm_training_v2/tokenizer.py`
- [Vocabulary pool](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/vocab_pool.py): `slm_training_v2/vocab_pool.py`
- [Gloss and validation](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/onnx_config.py): `slm_training_v2/onnx_config.py`

### Model runtime

- [Word reduction](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/word_picker.py): `slm_training_v2/word_picker.py`
- [Gloss restructuring](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/encoder_distilbert.py): `slm_training_v2/encoder_distilbert.py`
- [Sentence generation](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/decoder_tinyllama.py): `slm_training_v2/decoder_tinyllama.py`
- [PyTorch inference](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/archive/slm_inference.py): `slm_training_v2/archive/slm_inference.py`

### Training and evaluation

- [Dataset generation](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/generate_dataset_v4.py): `slm_training_v2/generate_dataset_v4.py`
- [Word-label creation](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/make_labels.py): `slm_training_v2/make_labels.py`
- [Model comparison](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/compare.py): `slm_training_v2/training_comparison/compare.py`
- [MobileBERT encoder](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/encoder_mobilebert.py): `slm_training_v2/training_comparison/encoder_mobilebert.py`
- [TinyBERT encoder](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/encoder_tinybert.py): `slm_training_v2/training_comparison/encoder_tinybert.py`
- [BabyLlama decoder](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/decoder_babyllama.py): `slm_training_v2/training_comparison/decoder_babyllama.py`
- [MobiLlama decoder](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/training_comparison/decoder_mobillama.py): `slm_training_v2/training_comparison/decoder_mobillama.py`

### Edge deployment

- [Deployment API](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/slm_api.py): `slm_training_v2/pi_deployment_later/slm_api.py`
- [ONNX inference](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/onnx_inference.py): `slm_training_v2/pi_deployment_later/onnx_inference.py`
- [Conversation session](https://github.com/stibbbb/thesis-slm/blob/main/slm_training_v2/pi_deployment_later/conversation_session.py): `slm_training_v2/pi_deployment_later/conversation_session.py`

### Other components

- **Hearing user**
- **Sign user**

## Connections

- Hearing user → Bidirectional pipeline: provides speech or text (evidence: `slm_training_v2/slm_pipeline.py`)
- Sign user → Bidirectional pipeline: provides signs (evidence: `slm_training_v2/slm_pipeline.py`)
- Bidirectional pipeline → Speech input: captures speech (evidence: `slm_training_v2/slm_pipeline.py`)
- Bidirectional pipeline → Word tokenizer: tokenizes words (evidence: `slm_training_v2/slm_pipeline.py`)
- Word tokenizer → Vocabulary pool: uses vocabulary (evidence: `slm_training_v2/tokenizer.py`)
- Bidirectional pipeline → Gloss and validation: applies gloss rules (evidence: `slm_training_v2/slm_pipeline.py`)
- Bidirectional pipeline → Word reduction: reduces words (evidence: `slm_training_v2/slm_pipeline.py`)
- Bidirectional pipeline → Sentence generation: generates sentences (evidence: `slm_training_v2/slm_pipeline.py`)
- Bidirectional pipeline → Gloss restructuring: restructures glosses (evidence: `slm_training_v2/slm_pipeline.py`)
- PyTorch inference → Word tokenizer: tokenizes input (evidence: `slm_training_v2/archive/slm_inference.py`)
- PyTorch inference → Vocabulary pool: loads vocabulary (evidence: `slm_training_v2/archive/slm_inference.py`)
- PyTorch inference → Gloss and validation: validates glosses (evidence: `slm_training_v2/archive/slm_inference.py`)
- PyTorch inference → Gloss restructuring: loads encoder (evidence: `slm_training_v2/archive/slm_inference.py`)
- PyTorch inference → Sentence generation: loads decoder (evidence: `slm_training_v2/archive/slm_inference.py`)
- PyTorch inference → Word reduction: loads picker (evidence: `slm_training_v2/archive/slm_inference.py`)
- Word reduction → Gloss restructuring: reuses transformer components (evidence: `slm_training_v2/word_picker.py`)
- Dataset generation → Vocabulary pool: uses vocabulary (evidence: `slm_training_v2/generate_dataset_v4.py`)
- Model comparison → MobileBERT encoder: compares architecture (evidence: `slm_training_v2/training_comparison/compare.py`)
- Model comparison → TinyBERT encoder: compares architecture (evidence: `slm_training_v2/training_comparison/compare.py`)
- Model comparison → BabyLlama decoder: compares architecture (evidence: `slm_training_v2/training_comparison/compare.py`)
- Model comparison → MobiLlama decoder: compares architecture (evidence: `slm_training_v2/training_comparison/compare.py`)
- Deployment API → ONNX inference: dispatches inference (evidence: `slm_training_v2/pi_deployment_later/slm_api.py`)
- Deployment API → Word tokenizer: uses tokenizer (evidence: `slm_training_v2/pi_deployment_later/slm_api.py`)
- Deployment API → Gloss and validation: uses validation (evidence: `slm_training_v2/pi_deployment_later/slm_api.py`)
- Conversation session → Deployment API: calls API (evidence: `slm_training_v2/pi_deployment_later/conversation_session.py`)
- Conversation session → Vocabulary pool: uses vocabulary (evidence: `slm_training_v2/pi_deployment_later/conversation_session.py`)
- Conversation session → Gloss and validation: uses configuration (evidence: `slm_training_v2/pi_deployment_later/conversation_session.py`)
