# yuangongnd/gopt architecture

> Architecture diagram of the GitHub repository yuangongnd/gopt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yuangongnd/gopt
- Repository: https://github.com/yuangongnd/gopt

## Overview

##### Project Type and Purpose
- Type: Open-source ML research tool / pipeline for multi-aspect, multi-granularity pronunciation assessment.
- Purpose: Given raw speech (or precomputed GOP features), preprocess into fixed-length sequences, feed into a Transformer-based model (GOPT), and evaluate pronunciation quality at phoneme, word, and utterance levels. Supports both local scripts and Google Colab execution.

##### File Structure
- Top-level dirs:
· colab: notebook orchestrating the pipeline on Google Colab
· data: stored GOP features and sequence datasets
· exp: experiment outputs (trained models, logs, predictions)
· figure: static assets for README visualization
· pretrained_models: saved GOPT checkpoints and conversion scripts
· src: core code, subdivided into
– extract_kaldi_gop: scripts for GOP feature extraction (requires Kaldi)
– prep_data: scripts to convert GOP features into sequence tensors
– models: baseline.py and gopt.py defining PyTorch models
– traintest.py and run.sh: entry point for training and evaluation
- Config & tooling:
· requirements.txt for Python deps
· steps_of_inference.md for end-user inference guide

##### README for Insights
- Data flows: raw Speechocean762 → (optional) Kaldi recipe → GOP features → seq_data generators → model training & evaluation
- Supports two execution modes:
a. Local (requires Kaldi for raw GOP extraction, or skip with downloaded data)
b. Colab (uses precomputed GOP, zero-setup)
- Outputs: experiment artifacts saved under exp_dir, pretrained checkpoints under pretrained_models.

##### Main Components
a. Data Ingestion
– Speechocean762 dataset (raw WAVs, transcripts)
– Precomputed GOP features (downloadable ZIP or Dropbox)
b. Feature Extraction
– extract_gop_feats.py / extract_gop_feats_word.py
c. Data Preparation
– gen_seq_data_{phn,word,utt}.py in src/prep_data
– Pads and masks sequences to fixed length
– Outputs: data/seq_data_*
d. Model Training & Evaluation
– shell script run.sh → traintest.py
– Loads sequence data → instantiates GOPT (models/gopt.py) or baseline
– Computes losses, metrics; logs to exp/<experiment_name>
e. Pretrained Models & Inference
– load_model.py for converting old logs
– steps_of_inference.md / Colab notebook invokes data prep and traintest stages in cloud
f. Experiment Artifacts
– exp: trained checkpoints, result.csv, logs
– pretrained_models: packaged best_audio_model.pth, result.csv

##### Relationships and Data Flow
1. Raw data → (optional) Kaldi GOP → GOP features
2. GOP features → seq_data generators → sequence datasets
3. seq_data → run.sh → traintest.py → model
4. Model outputs → metrics & checkpoints → exp
5. Inference path: raw audio → data/seq_data (via Colab) → load pretrained model → predictions

##### Architectural Patterns & Principles
- Pipeline pattern: distinct stages with clear hand-off artifacts in file system
- Modularity: each script or module has a single responsibility (extract, prep, train, evaluate)
- Extensibility: drop-in new model in src/models, update __init__.py and traintest.py
- Separation of concerns: data, preprocessing, modeling, evaluation, reporting

##### Technologies and External Dependencies
- Python 3, PyTorch
- Kaldi (for raw GOP extraction)
- Google Colab (optional execution environment)
- Shell scripting (run.sh), SLURM support for HPC
- JSON (scores.json), CSV (result.csv)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion
    subgraph "Data Ingestion"
        Speechocean762["Speechocean762 (raw WAVs & transcripts)"]:::data
        PrecomputedGOP["Precomputed GOP features (ZIP)"]:::data
        DataDir["data/"]:::data
    end

    %% Feature Extraction
    subgraph "Feature Extraction"
        KaldiGOPPipe["Kaldi GOP pipeline (optional)"]:::external
        ExtractGOP["extract_gop_feats.py"]:::preproc
        ExtractGOP_Word["extract_gop_feats_word.py"]:::preproc
    end

    %% Sequence Data Preparation
    subgraph "Sequence Data Preparation"
        SeqPhn["gen_seq_data_phn.py"]:::preproc
        SeqWord["gen_seq_data_word.py"]:::preproc
        SeqUtt["gen_seq_data_utt.py"]:::preproc
        SeqStore["Sequence Data Stores"]:::data
    end

    %% Model Training & Evaluation
    subgraph "Model Training & Evaluation"
        RunSh["run.sh"]:::script
        TrainTest["traintest.py"]:::script
        Baseline["baseline.py"]:::model
        GOPT["gopt.py"]:::model
    end

    %% Pretrained Models & Inference
    subgraph "Pretrained Models & Inference"
        ConvertOld["convert_old_log_to_new.py"]:::script
        LoadModel["load_model.py"]:::script
        ColabNB["GOPT_GPU.ipynb"]:::notebook
        StepsInfer["steps_of_inference.md"]:::doc
    end

    %% Experiment Artifacts
    subgraph "Experiment Artifacts"
        ExpDir["exp/"]:::store
        PretrainedDir["pretrained_models/"]:::store
    end

    %% Connections
    Speechocean762 -->|"raw WAVs & transcripts"| KaldiGOPPipe
    KaldiGOPPipe -.-> ExtractGOP
    ExtractGOP -->|"ark→npy GOP feats"| SeqPhn
    ExtractGOP -->|"ark→npy GOP feats"| SeqWord
    ExtractGOP_Word -->|"ark→npy GOP feats"| SeqWord
    PrecomputedGOP --> SeqPhn
    PrecomputedGOP --> SeqWord
    PrecomputedGOP --> SeqUtt
    SeqPhn --> SeqStore
    SeqWord --> SeqStore
    SeqUtt --> SeqStore
    SeqStore --> RunSh
    RunSh --> TrainTest
    TrainTest --> GOPT
    TrainTest --> Baseline
    GOPT --> ExpDir
    Baseline --> ExpDir
    TrainTest --> ExpDir
    ExpDir --> PretrainedDir
    ConvertOld --> PretrainedDir
    ColabNB --> SeqStore
    ColabNB --> LoadModel
    StepsInfer --> ColabNB

    %% Click Events
    click DataDir "https://github.com/yuangongnd/gopt/tree/master/data/"
    click ExtractGOP "https://github.com/yuangongnd/gopt/blob/master/src/extract_kaldi_gop/extract_gop_feats.py"
    click ExtractGOP_Word "https://github.com/yuangongnd/gopt/blob/master/src/extract_kaldi_gop/extract_gop_feats_word.py"
    click SeqPhn "https://github.com/yuangongnd/gopt/blob/master/src/prep_data/gen_seq_data_phn.py"
    click SeqWord "https://github.com/yuangongnd/gopt/blob/master/src/prep_data/gen_seq_data_word.py"
    click SeqUtt "https://github.com/yuangongnd/gopt/blob/master/src/prep_data/gen_seq_data_utt.py"
    click Baseline "https://github.com/yuangongnd/gopt/blob/master/src/models/baseline.py"
    click GOPT "https://github.com/yuangongnd/gopt/blob/master/src/models/gopt.py"
    click RunSh "https://github.com/yuangongnd/gopt/blob/master/src/run.sh"
    click TrainTest "https://github.com/yuangongnd/gopt/blob/master/src/traintest.py"
    click ExpDir "https://github.com/yuangongnd/gopt/tree/master/exp/"
    click PretrainedDir "https://github.com/yuangongnd/gopt/tree/master/pretrained_models/"
    click ConvertOld "https://github.com/yuangongnd/gopt/blob/master/pretrained_models/convert_old_log_to_new.py"
    click LoadModel "https://github.com/yuangongnd/gopt/blob/master/pretrained_models/load_model.py"
    click ColabNB "https://github.com/yuangongnd/gopt/blob/master/colab/GOPT_GPU.ipynb"
    click StepsInfer "https://github.com/yuangongnd/gopt/blob/master/steps_of_inference.md"

    %% Styles
    classDef data fill:#E8F1FA,stroke:#333,stroke-width:1px
    classDef preproc fill:#FCF3CF,stroke:#333,stroke-width:1px
    classDef model fill:#D5F5E3,stroke:#333,stroke-width:1px
    classDef script fill:#FADBD8,stroke:#333,stroke-width:1px
    classDef notebook fill:#EBDEF0,stroke:#333,stroke-width:1px
    classDef doc fill:#F9E79F,stroke:#333,stroke-width:1px
    classDef store fill:#E8DAEF,stroke:#333,stroke-width:1px
    classDef external fill:#D5DBDB,stroke-dasharray: 5 5,stroke:#333,stroke-width:1px
```
