# senthilthangaiah/transformer_translate architecture

> Architecture diagram of the GitHub repository senthilthangaiah/transformer_translate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/senthilthangaiah/transformer_translate
- Repository: https://github.com/senthilthangaiah/transformer_translate

## Overview

This repository trains a German-to-English Transformer and exposes sentence translation through a Streamlit interface. The training script prepares tokenized data, trains and evaluates the model, and saves its best weights; the app loads weights and translates user input. The README also describes vocabulary artifacts, but the sampled code rebuilds minimal vocabularies in the app and does not save training vocabularies, so that mismatch is shown cautiously. The dataset-to-training link follows the README; source excerpts do not establish how the listed data files are consumed.

Main entry points: [`train.py`](https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py) (Train and evaluate), [`translate_app.py`](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py) (Sentence translation), [`data`](https://github.com/senthilthangaiah/transformer_translate/tree/master/data) (Translation datasets)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Model training"]
  node_train_flow["Train and evaluate<br/>[train.py]"]
  node_train_tokens["Training tokenizers<br/>[train.py]"]
  node_train_model["Transformer model<br/>[train.py]"]
end

subgraph group_inference["Translation app"]
  node_app_ui["Streamlit interface<br/>[translate_app.py]"]
  node_translate["Sentence translation<br/>[translate_app.py]"]
  node_app_tokens["Inference tokenizers<br/>[translate_app.py]"]
  node_app_model["Inference Transformer<br/>[translate_app.py]"]
  node_app_vocab["Inference vocabularies<br/>[translate_app.py]"]
end

subgraph group_artifacts["Data and artifacts"]
  node_train_data["Translation datasets"]
  node_model_weights["Model weights"]
end

node_user(("German speaker"))

node_train_flow -->|"tokenizes data"| node_train_tokens
node_train_flow -->|"trains model"| node_train_model
node_train_flow -.->|"loads splits"| node_train_data
node_train_flow -->|"saves best weights"| node_model_weights
node_user -->|"enters sentence"| node_app_ui
node_app_ui -->|"invokes translation"| node_translate
node_translate -->|"tokenizes input"| node_app_tokens
node_translate -->|"looks up tokens"| node_app_vocab
node_translate -->|"decodes sentence"| node_app_model
node_app_model -->|"loads weights"| node_model_weights
node_app_vocab -->|"sets token fields"| node_app_tokens
node_app_ui -->|"displays translation"| node_user

click node_train_flow "https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py"
click node_train_tokens "https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py"
click node_train_model "https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py"
click node_train_data "https://github.com/senthilthangaiah/transformer_translate/tree/master/data"
click node_app_ui "https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py"
click node_translate "https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py"
click node_app_tokens "https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py"
click node_app_model "https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py"
click node_app_vocab "https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_flow,node_train_tokens,node_train_model toneBlue
class node_app_ui,node_translate,node_app_tokens,node_app_model,node_app_vocab toneAmber
class node_train_data,node_model_weights toneMint
class node_user toneIndigo
```

## Components

### Model training

- [Train and evaluate](https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py): `train.py`
- [Training tokenizers](https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py): `train.py`
- [Transformer model](https://github.com/senthilthangaiah/transformer_translate/blob/master/train.py): `train.py`

### Translation app

- [Streamlit interface](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py): `translate_app.py`
- [Sentence translation](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py): `translate_app.py`
- [Inference tokenizers](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py): `translate_app.py`
- [Inference Transformer](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py): `translate_app.py`
- [Inference vocabularies](https://github.com/senthilthangaiah/transformer_translate/blob/master/translate_app.py): `translate_app.py`

### Data and artifacts

- [Translation datasets](https://github.com/senthilthangaiah/transformer_translate/tree/master/data): `data`
- **Model weights**

### Other components

- **German speaker**

## Connections

- Train and evaluate → Training tokenizers: tokenizes data (evidence: `train.py`)
- Train and evaluate → Transformer model: trains model (evidence: `train.py`)
- Train and evaluate → Translation datasets: loads splits (evidence: `Readme.md`)
- Train and evaluate → Model weights: saves best weights (evidence: `train.py`)
- German speaker → Streamlit interface: enters sentence (evidence: `Readme.md`)
- Streamlit interface → Sentence translation: invokes translation (evidence: `translate_app.py`)
- Sentence translation → Inference tokenizers: tokenizes input (evidence: `translate_app.py`)
- Sentence translation → Inference vocabularies: looks up tokens (evidence: `translate_app.py`)
- Sentence translation → Inference Transformer: decodes sentence (evidence: `translate_app.py`)
- Inference Transformer → Model weights: loads weights (evidence: `translate_app.py`)
- Inference vocabularies → Inference tokenizers: sets token fields (evidence: `translate_app.py`)
- Streamlit interface → German speaker: displays translation (evidence: `translate_app.py`)
