# strifee/arabic2english architecture

> Architecture diagram of the GitHub repository strifee/arabic2english, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/strifee/arabic2english
- Repository: https://github.com/strifee/arabic2english

## Overview

##### Project type and purpose
- This is a neural machine‐translation pipeline implemented in PyTorch using the Transformer architecture.
- It consists of data ingestion, preprocessing, model definition, training, and inference (translation).

##### File structure
- data/ → raw bilingual sentence pairs (arabic_english.txt)
- data_preparation.ipynb & data_processing.py → ETL and dataset creation
- transformer.py → model architecture (Encoder/Decoder layers, multi-head attention)
- train.py → training loop (loads data, optimizer, loss, checkpoint saving)
- translate.py → inference script (loads saved model, runs translation)
- device.py → GPU/CPU selection helper
- hyper_parameters.py → central place for all hyperparameter values
- notebooks (translation.ipynb) & docs → exploratory analysis and background
- requirements.txt → environment dependencies

##### Extract insights from the README
- Confirms use of “Attention Is All You Need” transformer
- Lists hyperparameters, training commands, sample results
- No external web/API layers—pure ML pipeline

##### Main components & their interactions
a. Data Storage
– Raw Data (data/arabic_english.txt)
– Processed Dataset (after data_processing.py)
b. Data Preparation Module
– Tokenization, cleaning, batching
– Outputs PyTorch Dataset/DataLoader objects
c. Model Definition
– transformer.py provides Encoder/Decoder classes
– hyper_parameters.py injects model/config values
d. Device Configuration
– device.py selects GPU or CPU for training/inference
e. Training Pipeline
– train.py ties together DataLoader, model, optimizer, loss function
– Logs metrics, saves checkpoints
f. Inference Pipeline
– translate.py loads checkpoint, takes user input, runs model forward pass, post-processes output
g. (Optional) Notebooks & Documentation
– translation.ipynb for visualizations and prototyping
– docs/Machine_translation.pdf for theoretical background

##### Architectural patterns and design principles
- Modular separation:
– ETL separated from model code
– Hyperparameters centralized
– Device management isolated
- Pipeline pattern: clear flow from raw data → preprocessing → model training → inference
- Single responsibility: each script/module does one thing

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Arabic2English ML Pipeline"
        direction TB
        DataStorage["Raw Data\n(data/arabic_english.txt)"]:::data
        DataProc["Data Processing Module\n(data_processing.py)"]:::preproc
        DataNB["Data Preparation Notebook\n(Notebook)"]:::preproc
        Transformer["Transformer Encoder/Decoder\n(transformer.py)"]:::model
        HyperParams["Hyperparameters\n(hyper_parameters.py)"]:::model
        Device["Device Config (GPU/CPU)\n(device.py)"]:::exec
        Train["Training Engine\n(train.py)"]:::exec
        Checkpoints["Model Checkpoints"]:::artifacts
        Inference["Inference Engine\n(translate.py)"]:::exec
        User["User CLI\n(stdin/stdout"]:::external
    end

    subgraph "Analysis & Documentation"
        direction TB
        TranslationNB["Exploration Notebook\n(translation.ipynb)"]:::docs
        Docs["Documentation\n(Machine_translation.pdf)"]:::docs
    end

    Req["Environment Dependencies\n(requirements.txt)"]:::infra

    DataStorage -->|"raw text"| DataProc
    DataStorage --> DataNB
    DataProc -->|"processed dataset"| Transformer
    DataNB -.-> Transformer
    HyperParams --> Transformer
    Transformer -->|"model architecture"| Train
    DataProc --> Train
    Device --> Train
    HyperParams --> Train
    Train -->|"model checkpoints"| Checkpoints
    Checkpoints --> Inference
    Device --> Inference
    Inference -->|"translated text"| User
    Req -.-> DataProc
    Req -.-> Transformer
    Req -.-> Train
    Req -.-> Inference

    click DataStorage "https://github.com/strifee/arabic2english/blob/main/data/arabic_english.txt"
    click DataProc "https://github.com/strifee/arabic2english/blob/main/data_processing.py"
    click DataNB "https://github.com/strifee/arabic2english/blob/main/data_preparation.ipynb"
    click Transformer "https://github.com/strifee/arabic2english/blob/main/transformer.py"
    click HyperParams "https://github.com/strifee/arabic2english/blob/main/hyper_parameters.py"
    click Device "https://github.com/strifee/arabic2english/blob/main/device.py"
    click Train "https://github.com/strifee/arabic2english/blob/main/train.py"
    click Inference "https://github.com/strifee/arabic2english/blob/main/translate.py"
    click TranslationNB "https://github.com/strifee/arabic2english/blob/main/translation.ipynb"
    click Docs "https://github.com/strifee/arabic2english/blob/main/docs/Machine_translation.pdf"
    click Req "https://github.com/strifee/arabic2english/blob/main/requirements.txt"

    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef preproc fill:#d4edda,stroke:#333,stroke-width:1px;
    classDef model fill:#ffe5b4,stroke:#333,stroke-width:1px;
    classDef exec fill:#e2e3e5,stroke:#333,stroke-width:1px;
    classDef artifacts fill:#e0e0e0,stroke:#333,stroke-width:1px;
    classDef external fill:#f8d7da,stroke:#333,stroke-width:1px;
    classDef docs fill:#fff3cd,stroke:#333,stroke-width:1px;
    classDef infra fill:#f0f0f0,stroke:#333,stroke-width:1px;
```
