# stif83/translator architecture

> Architecture diagram of the GitHub repository stif83/translator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/stif83/translator
- Repository: https://github.com/stif83/translator

## Overview

##### Project Type and Purpose
- This is an open-source machine-learning translation tool (French ⇄ English) built in Python using NLTK.
- It provides two primary workflows: training a translation model and running inference (testing) against parallel text data.

##### Key Directories and Files
- data/raw/en-fr small.tmx: parallel corpus in TMX format (raw bilingual training data)
- src/base_model.py: abstract or shared logic for model definition, training, saving, loading, and inference
- main_train.py (invoked from README): orchestrates data loading, preprocessing, model instantiation, training loop
- main_test.py (invoked from README): loads a trained model, runs translation on test inputs, evaluates or outputs translations
- requirements.txt: Python dependencies (NLTK, numpy, etc.)
- README.md: setup instructions

##### Architectural Components
a. Data Ingestion & Storage
- Raw corpus loader (reads TMX)
- Preprocessor (tokenization, sentence alignment)
b. Model Layer
- BaseModel class (in src/base_model.py)
- Model Trainer (main_train.py workflow)
- Model Serializer (save/load checkpoints)
c. Inference Layer
- Model Loader (loads serialized model)
- Translator / Inference Engine (main_test.py workflow)
d. Environment & Dependencies
- Python Virtual Environment (venv)
- requirements.txt (dependency management)

##### Component Interactions
- Training Data → Model Trainer → Trained Model
- Loaded Model → Inference Engine → Translated Output

##### Architectural Patterns & Principles
- Modular separation: data prep, model logic, training, and inference are distinct modules/files
- Single Responsibility: base_model.py defines core ML functions; main_*.py scripts orchestrate flows
- Reusability: BaseModel can be extended for alternative translation strategies
- Batch pipelines: clear flow of data from raw corpus through to final translation

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Environment & Dependencies"
        Env["Python venv"]:::env
        Req["requirements.txt"]:::env
    end

    subgraph "Data Store" 
        RawCorpus["Raw Corpus\n(TMXX Data)"]:::data
    end

    subgraph "Preprocessing" 
        Preprocessor["Preprocessor\n(tokenization, alignment)"]:::data
    end

    subgraph "Training Engine" 
        Trainer["main_train.py\n(Orchestrator)"]:::code
        BaseModel["BaseModel\n(src/base_model.py)"]:::code
    end

    subgraph "Model Store"
        ModelChkpt["Trained Model\n(checkpoint)"]:::data
    end

    subgraph "Inference Engine"
        Loader["Model Loader\n(BaseModel.load)"]:::code
        Translator["main_test.py\n(Translator)"]:::code
    end

    %% Data Flow
    RawCorpus -->|"read TMX"| Preprocessor
    Preprocessor -->|"produces training data"| Trainer
    Trainer -->|"uses BaseModel"| BaseModel
    Trainer -->|"outputs checkpoint"| ModelChkpt

    %% Serialization
    BaseModel -->|"save()/load()"| ModelChkpt
    ModelChkpt -->|"load()"| Loader

    %% Inference Flow
    Loader -->|"provides model"| Translator
    Translator -->|"outputs translations"| Output["Translated Output"]:::data

    %% Environment linkage
    Env -.-> Trainer
    Env -.-> Translator
    Env -.-> Preprocessor

    %% Click Events
    click RawCorpus "https://github.com/stif83/translator/blob/main/data/raw/en-fr small.tmx"
    click BaseModel "https://github.com/stif83/translator/blob/main/src/base_model.py"
    click Req "https://github.com/stif83/translator/blob/main/requirements.txt"
    click README "https://github.com/stif83/translator/blob/main/README.md"

    %% Styles
    classDef data fill:#ccffcc,stroke:#06b,stroke-width:1px;
    classDef code fill:#cce5ff,stroke:#036,stroke-width:1px;
    classDef env  fill:#ffe0b3,stroke:#d28c00,stroke-width:1px;
```
