How senthilthangaiah/transformer_translate works
This repository trains a German-to-English Transformer and exposes sentence translation through a Streamlit interface. The training script prepares tokenized data, trains and evaluates the model, and saves its best weights; the app loads weights and translates user input. The README also describes vocabulary artifacts, but the sampled code rebuilds minimal vocabularies in the app and does not save training vocabularies, so that mismatch is shown cautiously. The dataset-to-training link follows the README; source excerpts do not establish how the listed data files are consumed.
Main entry points: train.py Train and evaluate · translate_app.py Sentence translation · data Translation datasets
Components 11
Model training
- Train and evaluate
train.py - Training tokenizers
train.py - Transformer model
train.py
Translation app
- Streamlit interface
translate_app.py - Sentence translation
translate_app.py - Inference tokenizers
translate_app.py - Inference Transformer
translate_app.py - Inference vocabularies
translate_app.py
Data and artifacts
- Translation datasets
data - Model weights
Other components
- German speaker
Connections 12
- Train and evaluate → Training tokenizers: tokenizes data
train.py - Train and evaluate → Transformer model: trains model
train.py - Train and evaluate → Translation datasets: loads splits
Readme.md - Train and evaluate → Model weights: saves best weights
train.py - German speaker → Streamlit interface: enters sentence
Readme.md - Streamlit interface → Sentence translation: invokes translation
translate_app.py - Sentence translation → Inference tokenizers: tokenizes input
translate_app.py - Sentence translation → Inference vocabularies: looks up tokens
translate_app.py - Sentence translation → Inference Transformer: decodes sentence
translate_app.py - Inference Transformer → Model weights: loads weights
translate_app.py - Inference vocabularies → Inference tokenizers: sets token fields
translate_app.py - Streamlit interface → German speaker: displays translation
translate_app.py