# bicheng1225/conpco architecture

> Architecture diagram of the GitHub repository bicheng1225/conpco, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bicheng1225/conpco
- Repository: https://github.com/bicheng1225/conpco

## Overview

##### Project type and purpose
- This is not a web app but an end-to-end machine-learning research codebase for Automatic Pronunciation Assessment (APA).
- It implements “ConPCO,” a contrastive ordinal regularizer, built on top of prior APA models (GOPT, HierCB) to produce phoneme-discriminative features.
- Key indicators: a `data/` folder, model implementations under `src/models/`, training scripts (`train_hierCB.sh`, `traintest_…py`) and references in the README to feature extraction (GOP, SSL), model training, and evaluation.

##### File structure
- Top-level:
- `data/` – raw + preprocessed dataset assets (Speechocean762).
- `figure/` – static poster/image.
- `requirements.txt` – Python dependencies (PyTorch, PyKaldi, espnet modules, etc.).
- `src/` – core implementation.
- Under `src/`:
- `models/` – Python modules for each model variant and ConPCO loss (`conPCO_norm.py`, `gopt_ssl_3m_bfr_cat_utt_clap.py`).
- `train_hierCB.sh` – shell script orchestrating training of HierCB variants.
- `traintest_eng_dur_ssl_3m_HierBFR_conPCO_norm.py` – Python entry point for training/testing with ConPCO.

##### README for additional insights
- Confirms the data sources (Speechocean762), features used (GOP, energy, duration, SSL, word IDs).
- Describes the contrastive + ordinal regularization pipeline: align phoneme features with textual embeddings, regulate inter/intra-phoneme distances, maintain ordinal relationships of regression targets.
- Lists dependencies and upstream libraries (GOPT, PyKaldi, ESPnet, OrdinalEntropy).

##### Main components
a. Data Layer
- Raw audio & transcripts (Speechocean762)
- Preprocessed features stored on disk (`data/…`)
b. Feature Extraction Module
- GOP feature extractor (PyKaldi wrapper)
- Energy & duration calculators
- SSL feature extractor (Wav2vec2, HuBERT, WavLM via espnet)
- Word ID embedding lookup
c. Model Architecture Module
- Base encoder (GOPT Transformer backbone)
- ConPCO contrastive-ordinal regularizer
- Hierarchical CB head (HierCB) for regression
d. Training & Orchestration
- Shell scripts & Python entry points
- Hyperparameter/config management (via `requirements.txt`, CLI args)
e. Evaluation & Logging
- Train/test loops reporting phoneme-level accuracy
- Model checkpointing and metric logging

##### Relationships and interactions
- Data Layer → Feature Extraction → Model Architecture → Training Loop → Evaluation → Model Artifacts
- External APIs/libraries (PyKaldi, espnet, OrdinalEntropy) feed into feature extraction and loss computation.
- The ConPCO module sits inside the loss function in the training loop, interacting with phoneme embeddings and model outputs.

##### Architectural patterns & design principles
- Layered pipeline: clear separation of data ingestion, feature extraction, modeling, training, evaluation.
- Modular code: each model variant is encapsulated in its own file under `src/models/`.
- Orchestration via simple shell+Python orchestration scripts.

##### Elements
- Rectangles for modules: Data Ingestion, Feature Extractor, Encoder, ConPCO Loss, Regression Head, Evaluation
Raw Audio → Feature Extractor → Encoder → Loss → Backprop → Checkpoint

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        DataLayer[(Data Layer)]:::data
        DataReadme["Data README"]:::data
    end
    DataLayer --> DataReadme

    subgraph "Feature Extraction" 
        GOP["GOP Extractor"]:::feature
        Energy["Energy & Duration"]:::feature
        SSL["SSL Feature Extractor"]:::feature
        WordID["Word ID Embedding"]:::feature
    end
    DataLayer --> GOP
    DataLayer --> Energy
    DataLayer --> SSL
    DataLayer --> WordID

    subgraph "Model Architecture"
        Encoder["GOPT Backbone"]:::model
        ConPCO["ConPCO Loss Module"]:::model
        HierCB["Hierarchical CB Head"]:::model
    end
    GOP --> Encoder
    Energy --> Encoder
    SSL --> Encoder
    WordID --> Encoder
    Encoder --> ConPCO
    Encoder --> HierCB
    OrdEntropy["OrdinalEntropy"]:::external
    OrdEntropy --> ConPCO

    subgraph "Training & Orchestration"
        Shell["train_hierCB.sh"]:::orchestration
        Entry["traintest_eng_dur_ssl_3m_HierBFR_conPCO_norm.py"]:::orchestration
        Requirements["requirements.txt"]:::orchestration
        Readme["README.md"]:::orchestration
    end
    Shell --> Entry
    Entry --> Encoder
    Entry --> ConPCO
    Entry --> HierCB

    subgraph "Evaluation & Logging"
        Eval["Evaluation & Metrics"]:::orchestration
        Checkpoints[(Model Checkpoints)]:::data
    end
    Entry --> Eval
    Eval --> Checkpoints

    PyKaldi["PyKaldi"]:::external
    ESPnet["espnet"]:::external
    PyKaldi --> GOP
    ESPnet --> SSL

    click DataLayer "https://github.com/bicheng1225/conpco/tree/main/data/"
    click DataReadme "https://github.com/bicheng1225/conpco/blob/main/data/README.md"
    click ModelsPkg "https://github.com/bicheng1225/conpco/blob/main/src/models/__init__.py"
    click GOP "https://github.com/bicheng1225/conpco/blob/main/src/models/gopt_ssl_3m_bfr_cat_utt_clap.py"
    click ConPCO "https://github.com/bicheng1225/conpco/blob/main/src/models/conPCO_norm.py"
    click Shell "https://github.com/bicheng1225/conpco/blob/main/src/train_hierCB.sh"
    click Entry "https://github.com/bicheng1225/conpco/blob/main/src/traintest_eng_dur_ssl_3m_HierBFR_conPCO_norm.py"
    click Requirements "https://github.com/bicheng1225/conpco/blob/main/requirements.txt"
    click Readme "https://github.com/bicheng1225/conpco/blob/main/README.md"

    classDef data fill:#AED3F7,stroke:#333,stroke-width:1px;
    classDef feature fill:#A6EBC9,stroke:#333,stroke-width:1px;
    classDef model fill:#FFC978,stroke:#333,stroke-width:1px;
    classDef orchestration fill:#D1C4E9,stroke:#333,stroke-width:1px;
    classDef external fill:#E0E0E0,stroke:#333,stroke-width:1px;
```
