# neccam/slt architecture

> Architecture diagram of the GitHub repository neccam/slt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neccam/slt
- Repository: https://github.com/neccam/slt

## Overview

SignJoey trains and evaluates a joint sign-language recognition and translation model. The user workflow starts with the module CLI, loads sign features, glosses and text, batches examples, then trains or predicts with the model; evaluation scores recognition and translation outputs. The model combines sign embeddings, an encoder, gloss recognition and text decoding, with search for generation. The SOURCE INDEX supports the listed module references; details of unsampled implementation paths and configuration wiring are not inferred.

Main entry points: [`signjoey/model.py`](https://github.com/neccam/slt/blob/master/signjoey/model.py) (Joint SLT model), [`signjoey/prediction.py`](https://github.com/neccam/slt/blob/master/signjoey/prediction.py) (Prediction and testing), [`signjoey/training.py`](https://github.com/neccam/slt/blob/master/signjoey/training.py) (Training loop), [`signjoey/data.py`](https://github.com/neccam/slt/blob/master/signjoey/data.py) (Data loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_datasets[("Sign datasets<br/>[dataset.py]")]
  node_loader["Data loader<br/>[data.py]"]
  node_vocab["Gloss and text vocabularies<br/>[vocabulary.py]"]
  node_batch["Batch preparation<br/>[batch.py]"]
end

subgraph group_model["Joint model"]
  node_builder["Model builder<br/>[builders.py]"]
  node_modelcore["Joint SLT model<br/>[model.py]"]
  node_embedding["Sign and token embeddings<br/>[embeddings.py]"]
  node_encoder["Sequence encoders<br/>[encoders.py]"]
  node_layers["Transformer layers"]
  node_recognition["Gloss recognition<br/>[model.py]"]
  node_decoder["Text decoders and attention<br/>[decoders.py]"]
  node_search["Recognition and translation search<br/>[search.py]"]
end

subgraph group_run["Training and inference"]
  node_cli["Module CLI<br/>[__main__.py]"]
  node_trainer["Training loop<br/>[training.py]"]
  node_predict["Prediction and testing<br/>[prediction.py]"]
  node_loss["Joint training loss<br/>[loss.py]"]
end

subgraph group_eval["Evaluation"]
  node_metrics["Task metrics<br/>[metrics.py]"]
  node_bleu["BLEU and related scores<br/>[sacrebleu.py]"]
  node_rouge["ROUGE scores<br/>[mscoco_rouge.py]"]
  node_cleanup["Phoenix output cleanup<br/>[phoenix_cleanup.py]"]
end

node_user(("Researcher"))

node_user -->|"runs module"| node_cli
node_cli -->|"dispatches training"| node_trainer
node_cli -->|"dispatches prediction"| node_predict
node_trainer -->|"loads data"| node_loader
node_predict -->|"loads data"| node_loader
node_loader -->|"constructs datasets"| node_datasets
node_loader -->|"builds vocabularies"| node_vocab
node_trainer -->|"builds model"| node_builder
node_predict -->|"builds model"| node_builder
node_trainer -->|"batches examples"| node_batch
node_predict -->|"batches examples"| node_batch
node_builder -->|"constructs"| node_modelcore
node_modelcore -->|"embeds inputs"| node_embedding
node_modelcore -->|"encodes sign sequence"| node_encoder
node_encoder -->|"uses layers"| node_layers
node_modelcore -->|"predicts glosses"| node_recognition
node_modelcore -->|"decodes text"| node_decoder
node_decoder -->|"uses layers"| node_layers
node_modelcore -->|"decodes outputs"| node_search
node_search -->|"invokes decoder"| node_decoder
node_trainer -->|"trains model"| node_modelcore
node_trainer -->|"computes loss"| node_loss
node_predict -->|"runs batches"| node_modelcore
node_predict -->|"scores predictions"| node_metrics
node_metrics -->|"computes BLEU"| node_bleu
node_metrics -->|"computes ROUGE"| node_rouge
node_predict -->|"cleans outputs"| node_cleanup

click node_cli "https://github.com/neccam/slt/blob/master/signjoey/__main__.py"
click node_datasets "https://github.com/neccam/slt/blob/master/signjoey/dataset.py"
click node_loader "https://github.com/neccam/slt/blob/master/signjoey/data.py"
click node_vocab "https://github.com/neccam/slt/blob/master/signjoey/vocabulary.py"
click node_batch "https://github.com/neccam/slt/blob/master/signjoey/batch.py"
click node_trainer "https://github.com/neccam/slt/blob/master/signjoey/training.py"
click node_predict "https://github.com/neccam/slt/blob/master/signjoey/prediction.py"
click node_builder "https://github.com/neccam/slt/blob/master/signjoey/builders.py"
click node_modelcore "https://github.com/neccam/slt/blob/master/signjoey/model.py"
click node_embedding "https://github.com/neccam/slt/blob/master/signjoey/embeddings.py"
click node_encoder "https://github.com/neccam/slt/blob/master/signjoey/encoders.py"
click node_layers "https://github.com/neccam/slt/blob/master/signjoey/transformer_layers.py"
click node_recognition "https://github.com/neccam/slt/blob/master/signjoey/model.py"
click node_decoder "https://github.com/neccam/slt/blob/master/signjoey/decoders.py"
click node_search "https://github.com/neccam/slt/blob/master/signjoey/search.py"
click node_loss "https://github.com/neccam/slt/blob/master/signjoey/loss.py"
click node_metrics "https://github.com/neccam/slt/blob/master/signjoey/metrics.py"
click node_bleu "https://github.com/neccam/slt/blob/master/signjoey/external_metrics/sacrebleu.py"
click node_rouge "https://github.com/neccam/slt/blob/master/signjoey/external_metrics/mscoco_rouge.py"
click node_cleanup "https://github.com/neccam/slt/blob/master/signjoey/phoenix_utils/phoenix_cleanup.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datasets,node_loader,node_vocab,node_batch toneBlue
class node_builder,node_modelcore,node_embedding,node_encoder,node_layers,node_recognition,node_decoder,node_search toneAmber
class node_cli,node_trainer,node_predict,node_loss toneMint
class node_metrics,node_bleu,node_rouge,node_cleanup toneRose
class node_user toneIndigo
```

## Components

### Data preparation

- [Sign datasets](https://github.com/neccam/slt/blob/master/signjoey/dataset.py): `signjoey/dataset.py`
- [Data loader](https://github.com/neccam/slt/blob/master/signjoey/data.py): `signjoey/data.py`
- [Gloss and text vocabularies](https://github.com/neccam/slt/blob/master/signjoey/vocabulary.py): `signjoey/vocabulary.py`
- [Batch preparation](https://github.com/neccam/slt/blob/master/signjoey/batch.py): `signjoey/batch.py`

### Joint model

- [Model builder](https://github.com/neccam/slt/blob/master/signjoey/builders.py): `signjoey/builders.py`
- [Joint SLT model](https://github.com/neccam/slt/blob/master/signjoey/model.py): `signjoey/model.py`
- [Sign and token embeddings](https://github.com/neccam/slt/blob/master/signjoey/embeddings.py): `signjoey/embeddings.py`
- [Sequence encoders](https://github.com/neccam/slt/blob/master/signjoey/encoders.py): `signjoey/encoders.py`
- [Transformer layers](https://github.com/neccam/slt/blob/master/signjoey/transformer_layers.py): `signjoey/transformer_layers.py`
- [Gloss recognition](https://github.com/neccam/slt/blob/master/signjoey/model.py): `signjoey/model.py`
- [Text decoders and attention](https://github.com/neccam/slt/blob/master/signjoey/decoders.py): `signjoey/decoders.py`
- [Recognition and translation search](https://github.com/neccam/slt/blob/master/signjoey/search.py): `signjoey/search.py`

### Training and inference

- [Module CLI](https://github.com/neccam/slt/blob/master/signjoey/__main__.py): `signjoey/__main__.py`
- [Training loop](https://github.com/neccam/slt/blob/master/signjoey/training.py): `signjoey/training.py`
- [Prediction and testing](https://github.com/neccam/slt/blob/master/signjoey/prediction.py): `signjoey/prediction.py`
- [Joint training loss](https://github.com/neccam/slt/blob/master/signjoey/loss.py): `signjoey/loss.py`

### Evaluation

- [Task metrics](https://github.com/neccam/slt/blob/master/signjoey/metrics.py): `signjoey/metrics.py`
- [BLEU and related scores](https://github.com/neccam/slt/blob/master/signjoey/external_metrics/sacrebleu.py): `signjoey/external_metrics/sacrebleu.py`
- [ROUGE scores](https://github.com/neccam/slt/blob/master/signjoey/external_metrics/mscoco_rouge.py): `signjoey/external_metrics/mscoco_rouge.py`
- [Phoenix output cleanup](https://github.com/neccam/slt/blob/master/signjoey/phoenix_utils/phoenix_cleanup.py): `signjoey/phoenix_utils/phoenix_cleanup.py`

### Other components

- **Researcher**

## Connections

- Researcher → Module CLI: runs module (evidence: `README.md`)
- Module CLI → Training loop: dispatches training (evidence: `signjoey/__main__.py`)
- Module CLI → Prediction and testing: dispatches prediction (evidence: `signjoey/__main__.py`)
- Training loop → Data loader: loads data (evidence: `signjoey/training.py`)
- Prediction and testing → Data loader: loads data (evidence: `signjoey/prediction.py`)
- Data loader → Sign datasets: constructs datasets (evidence: `signjoey/data.py`)
- Data loader → Gloss and text vocabularies: builds vocabularies (evidence: `signjoey/data.py`)
- Training loop → Model builder: builds model (evidence: `signjoey/training.py`)
- Prediction and testing → Model builder: builds model (evidence: `signjoey/prediction.py`)
- Training loop → Batch preparation: batches examples (evidence: `signjoey/training.py`)
- Prediction and testing → Batch preparation: batches examples (evidence: `signjoey/prediction.py`)
- Model builder → Joint SLT model: constructs (evidence: `signjoey/model.py`)
- Joint SLT model → Sign and token embeddings: embeds inputs (evidence: `signjoey/model.py`)
- Joint SLT model → Sequence encoders: encodes sign sequence (evidence: `signjoey/model.py`)
- Sequence encoders → Transformer layers: uses layers (evidence: `signjoey/encoders.py`)
- Joint SLT model → Gloss recognition: predicts glosses (evidence: `signjoey/model.py`)
- Joint SLT model → Text decoders and attention: decodes text (evidence: `signjoey/model.py`)
- Text decoders and attention → Transformer layers: uses layers (evidence: `signjoey/decoders.py`)
- Joint SLT model → Recognition and translation search: decodes outputs (evidence: `signjoey/model.py`)
- Recognition and translation search → Text decoders and attention: invokes decoder (evidence: `signjoey/search.py`)
- Training loop → Joint SLT model: trains model (evidence: `signjoey/training.py`)
- Training loop → Joint training loss: computes loss (evidence: `signjoey/training.py`)
- Prediction and testing → Joint SLT model: runs batches (evidence: `signjoey/prediction.py`)
- Prediction and testing → Task metrics: scores predictions (evidence: `signjoey/prediction.py`)
- Task metrics → BLEU and related scores: computes BLEU (evidence: `signjoey/metrics.py`)
- Task metrics → ROUGE scores: computes ROUGE (evidence: `signjoey/metrics.py`)
- Prediction and testing → Phoenix output cleanup: cleans outputs (evidence: `signjoey/prediction.py`)
