How copticoder/mt-via-round-trip-rl works
1. Project purpose
This repo is a research training pipeline for low-resource machine translation, centered on a round-trip reinforcement-learning fine-tuning method for NLLB models. The core experiment is: translate English to a low-resource language, translate back to English, and reward the model based on reconstruction quality.
2. Top-level architecture
The repository splits into two main areas:
- the main training/evaluation pipeline in main.py, dl.py, and utils.py
- baseline implementations under baselines/ for backtranslation and UMNMT comparisons
This separation makes the main method and the comparison methods independently runnable.
3. Orchestrator layer
main.py is the central entry point. It coordinates:
- configuration loading
- dataset/model setup
- training loop
- validation/test evaluation
- experiment logging
It is the primary place where data flow and control flow converge.
4. Data ingestion boundary
dl.py isolates dataset loading for the NLLB-MD benchmark. This keeps dataset-specific logic out of the training loop and makes it easier to swap task/data settings through configuration rather than code changes.
5. Training logic and reward/loss computation
utils.py contains shared helpers plus the GRPO loss function used for the reinforcement-learning-style fine-tuning. Architecturally, this file sits between model outputs and optimization, converting translation quality signals into a training objective.
6. Configuration subsystem
configs/ is organized into layered configuration groups:
- configs/train.yaml for overall training defaults
- configs/task/ for dataset/task selection
- configs/device/ for hardware-specific settings
- configs/hydra/default.yaml for Hydra behavior
This indicates a config-driven architecture where environment and experiment variants are selected without changing code.
7. Device and runtime portability
The device configs include cpu, gpu, mps, a100, and a100l, which suggests the pipeline is intended to run across local machines and multiple accelerator types. Hardware-specific behavior is handled through configuration rather than separate code paths.
8. Model and experiment scope
The core experiment targets NLLB models, specifically the 600M and 1.3B variants mentioned in the README. The architecture is therefore centered on fine-tuning pretrained translation models rather than training MT models from scratch.
9. Evaluation and logging flow
Evaluation on validation and test sets happens during and after training, so the system is not just a training script but a full experiment loop. The README also references Weights & Biases, so experiment metrics and plots are logged externally for analysis.
10. Baseline subsystem
baselines/ contains separate implementations for:
- backtranslation
- roundtrip-bt
- UMNMT
Each baseline has its own Python entry point and shell launcher, which means the comparison methods are intentionally decoupled from the main RL pipeline.
11. Execution model
The repo is run as a Python experiment project, typically via python main.py, with shell scripts used for baseline runs. This implies a workflow where the primary unit of execution is a configured training job rather than a long-running service or API.
12. How to draw the architecture
A good architecture diagram would show:
- configuration inputs feeding the main training job
- dataset loading via dl.py
- model/training/reward computation in main.py + utils.py
- logging/evaluation as side outputs
- separate baseline pipelines under baselines/
The main boundary is between the configurable RL training system and the independent baseline experiment runners.
Main entry points: main.py Main job · backtranslation.py Backtranslation · roundtrip-bt.py Roundtrip BT · umnmt.py UMNMT
Components 17
Main pipeline
Config-driven RL fine-tuning and evaluation flow
- Main job · Central training and evaluation entry point
main.py - Data loader · Loads the NLLB-MD benchmark and task data
dl.py - RL utils · Shared helpers and GRPO-style loss computation
utils.py - NLLB model · Fine-tuned translation backbone used by the main experiment
- Eval loop · Validation and test evaluation during training
- W&B logs · External metric and run logging
Configs
Hydra-based experiment and runtime selection
- Train config · Top-level training defaults
configs/train.yaml - Task config · Task and dataset selection for NLLB-MD
configs/task/nllb_md.yaml - Device presets · CPU, GPU, MPS, A100, and A100L hardware profiles
- Hydra config · Hydra defaults and config behavior
configs/hydra/default.yaml
Baselines
Independent comparison pipelines
- Backtranslation · Standalone backtranslation baseline implementation
baselines/backtranslation/backtranslation.py - Roundtrip BT · Standalone round-trip backtranslation baseline implementation
baselines/roundtrip-bt/roundtrip-bt.py - UMNMT · Standalone UMNMT baseline implementation
baselines/UMNMT/umnmt.py - BT script · Shell launcher for the backtranslation baseline
baselines/backtranslation/run_backtranslation.sh - RTBT script · Shell launcher for the roundtrip baseline
baselines/roundtrip-bt/run_roundtrip_bt.sh - UMNMT script · Shell launcher for the UMNMT baseline
baselines/UMNMT/run_training.sh - Baseline utils · Shared baseline support code
baselines/utils.py
Connections 18
- Train config → Main job: defaults · Provides core training configuration
- Task config → Main job: task · Selects the dataset/task variant
- Device presets → Main job: runtime · Chooses the target hardware profile
- Hydra config → Main job: compose · Controls Hydra composition and overrides
- Main job → Data loader: load data · Delegates benchmark data ingestion
- Main job → RL utils: optimize · Uses shared helpers and GRPO loss
- Main job → NLLB model: fine-tune · Runs round-trip RL fine-tuning on the pretrained model
- Main job → Eval loop: validate · Performs validation and test evaluation
- Main job → W&B logs: log · Sends metrics and experiment outputs to tracking
- BT script → Backtranslation: run · Launches the backtranslation baseline
- RTBT script → Roundtrip BT: run · Launches the roundtrip backtranslation baseline
- UMNMT script → UMNMT: run · Launches the UMNMT baseline
- Backtranslation → Baseline utils: shared code · Reuses common baseline utilities
- Roundtrip BT → Baseline utils: shared code · Reuses common baseline utilities
- UMNMT → Baseline utils: shared code · Reuses common baseline utilities
- Backtranslation → Eval loop: compare · Produces baseline experiment outputs for comparison
- Roundtrip BT → Eval loop: compare · Produces baseline experiment outputs for comparison
- UMNMT → Eval loop: compare · Produces baseline experiment outputs for comparison