# vrclc-duk/st-ta-en architecture

> Architecture diagram of the GitHub repository vrclc-duk/st-ta-en, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vrclc-duk/st-ta-en
- Repository: https://github.com/vrclc-duk/st-ta-en

## Overview

##### Project Type and Purpose
- This is an end-to-end speech translation system using the Whisper model. Its purpose is twofold—training (fine-tuning the Whisper model on Tamil speech data) and inference (translating Tamil speech into English text with associated quality metrics).

##### Analyzing the File Structure
- The project has a lean file structure with key scripts: one for training (“ST_training.py” as per the file tree, mentioned as ST-train.py in the README) and one for inference (“ST-inference.py”), plus a README and a requirements file.
- This structure indicates that the project is designed around two primary pipelines which share common dependencies (found in “requirements.txt”).

##### Main Components
a. Training Pipeline:
- Input Data: Tamil speech data (audio waveforms) provided for training.
- Preprocessing/Validation: Implicit component where audio and text data may be prepared and validated.
- Whisper Model Fine-Tuning: Core component in “ST_training.py” that adjusts the pre-trained Whisper model on Tamil data.
- Checkpoint & Log Management: Saving model checkpoints and tracking training metrics.
- Evaluation: Compute BLEU and METEOR scores to gauge training performance.

b. Inference Pipeline:
- Model Loading: Retrieval of the trained or fine-tuned Whisper model from either the HF Hub or local checkpoints.
- Audio Processing: Module for accepting and processing incoming Tamil audio (batch processing support as described).
- Translation Generation: Translating the processed audio into English text.
- Quality Metrics Computation: Using SacreBLEU and METEOR to measure translation quality.
- Output: Writing results to a CSV containing original text (if available), ground truth, model-generated translation, and metric scores.

c. Shared Components:
- Dependencies and Libraries: Essential packages (PyTorch, Transformers, Datasets, SacreBLEU, METEOR, NumPy, pandas) that support both training and inference.
- Configuration: Environment settings and dependency management specified in “requirements.txt”.

##### Relationships & Interactions
- The training component takes in raw data and outputs a fine-tuned Whisper model (plus logs and checkpoints), which then acts as the internal “product” of the training process.
- The inference component depends on loading the trained model, processing new audio input, running the translation, and then evaluating the result.
- From raw data → Training pipeline → Model checkpoints/outputs.
- From Model checkpoints → Inference pipeline.
- From inference → Processed audio → output CSV and metrics.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Clear division between training and inference, each handling specific aspects of the model lifecycle.
- Pipeline Structure: A sequential flow where training feeds “model artifacts” to inference.
- Modularity: Although the project is small-scale, think in terms of modular components (data handling, model management, evaluation) that can expand if needed.

##### Including Relevant Technologies
- Use of deep learning frameworks such as PyTorch for model operations.
- Significance of Hugging Face’s Transformers library which is central for implementing the Whisper model.
- Evaluation tools (SacreBLEU, METEOR) as critical for translation assessment.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Training Pipeline
    subgraph "Training Pipeline"
        DataInput["Raw Tamil Audio Data"]:::input
        Preprocessing["Data Preprocessing & Validation"]:::process
        FineTune["Whisper Model Fine-Tuning"]:::training
        Checkpoint["Checkpoint & Log Management"]:::storage
        Evaluation["Evaluation (BLEU,METEOR)"]:::evaluation
        ModelArtifact["Trained Model Artifact"]:::artifact
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        ModelLoading["Model Loading"]:::inference
        AudioProc["Audio Processing"]:::process
        Translation["Translation Generation"]:::inference
        Metrics["Quality Metrics Computation"]:::evaluation
        CSVOutput["CSV Output with Translations & Metrics"]:::output
    end

    %% Shared Components
    subgraph "Shared Components"
        Dependencies["Dependencies & Libraries"]:::shared
        Config["Configuration"]:::config
        Documentation["Project Documentation"]:::doc
    end

    %% Data Flow - Training Pipeline
    DataInput --> Preprocessing
    Preprocessing --> FineTune
    FineTune --> Checkpoint
    Checkpoint --> Evaluation
    Evaluation --> ModelArtifact

    %% Data Flow - Inference Pipeline
    ModelArtifact --> ModelLoading
    ModelLoading --> AudioProc
    AudioProc --> Translation
    Translation --> Metrics
    Metrics --> CSVOutput

    %% Shared Dependencies Flow
    Config --- FineTune
    Config --- ModelLoading
    Dependencies --- FineTune
    Dependencies --- ModelLoading
    Documentation --- Config

    %% Click Events
    click FineTune "https://github.com/vrclc-duk/st-ta-en/blob/master/ST_training.py"
    click ModelLoading "https://github.com/vrclc-duk/st-ta-en/blob/master/ST-inference.py"
    click Config "https://github.com/vrclc-duk/st-ta-en/blob/master/requirements.txt"
    click Documentation "https://github.com/vrclc-duk/st-ta-en/blob/master/README.md"

    %% Style Classes
    classDef input fill:#f9c,stroke:#333,stroke-width:2px;
    classDef process fill:#bbf,stroke:#333,stroke-width:2px;
    classDef training fill:#cfc,stroke:#333,stroke-width:2px;
    classDef storage fill:#fcf,stroke:#333,stroke-width:2px;
    classDef evaluation fill:#ff9,stroke:#333,stroke-width:2px;
    classDef artifact fill:#9ff,stroke:#333,stroke-width:2px;
    classDef inference fill:#cff,stroke:#333,stroke-width:2px;
    classDef output fill:#fbf,stroke:#333,stroke-width:2px;
    classDef shared fill:#ddd,stroke:#333,stroke-width:2px;
    classDef config fill:#ffd,stroke:#333,stroke-width:2px;
    classDef doc fill:#dfd,stroke:#333,stroke-width:2px;
```
