How bicheng1225/conpco works
Project type and purpose
• This is not a web app but an end-to-end machine-learning research codebase for Automatic Pronunciation Assessment (APA).
• It implements “ConPCO,” a contrastive ordinal regularizer, built on top of prior APA models (GOPT, HierCB) to produce phoneme-discriminative features.
• Key indicators: a data/ folder, model implementations under src/models/, training scripts (train_hierCB.sh, traintest_…py) and references in the README to feature extraction (GOP, SSL), model training, and evaluation.
File structure
• Top-level:
• data/ – raw + preprocessed dataset assets (Speechocean762).
• figure/ – static poster/image.
• requirements.txt – Python dependencies (PyTorch, PyKaldi, espnet modules, etc.).
• src/ – core implementation.
• Under src/:
• models/ – Python modules for each model variant and ConPCO loss (conPCO_norm.py, gopt_ssl_3m_bfr_cat_utt_clap.py).
• train_hierCB.sh – shell script orchestrating training of HierCB variants.
• traintest_eng_dur_ssl_3m_HierBFR_conPCO_norm.py – Python entry point for training/testing with ConPCO.
README for additional insights
• Confirms the data sources (Speechocean762), features used (GOP, energy, duration, SSL, word IDs).
• Describes the contrastive + ordinal regularization pipeline: align phoneme features with textual embeddings, regulate inter/intra-phoneme distances, maintain ordinal relationships of regression targets.
• Lists dependencies and upstream libraries (GOPT, PyKaldi, ESPnet, OrdinalEntropy).
Main components
a. Data Layer
• Raw audio & transcripts (Speechocean762)
• Preprocessed features stored on disk (data/…)
b. Feature Extraction Module
• GOP feature extractor (PyKaldi wrapper)
• Energy & duration calculators
• SSL feature extractor (Wav2vec2, HuBERT, WavLM via espnet)
• Word ID embedding lookup
c. Model Architecture Module
• Base encoder (GOPT Transformer backbone)
• ConPCO contrastive-ordinal regularizer
• Hierarchical CB head (HierCB) for regression
d. Training & Orchestration
• Shell scripts & Python entry points
• Hyperparameter/config management (via requirements.txt, CLI args)
e. Evaluation & Logging
• Train/test loops reporting phoneme-level accuracy
• Model checkpointing and metric logging
Relationships and interactions
• Data Layer → Feature Extraction → Model Architecture → Training Loop → Evaluation → Model Artifacts
• External APIs/libraries (PyKaldi, espnet, OrdinalEntropy) feed into feature extraction and loss computation.
• The ConPCO module sits inside the loss function in the training loop, interacting with phoneme embeddings and model outputs.
Architectural patterns & design principles
• Layered pipeline: clear separation of data ingestion, feature extraction, modeling, training, evaluation.
• Modular code: each model variant is encapsulated in its own file under src/models/.
• Orchestration via simple shell+Python orchestration scripts.
Elements
• Rectangles for modules: Data Ingestion, Feature Extractor, Encoder, ConPCO Loss, Regression Head, Evaluation
Raw Audio → Feature Extractor → Encoder → Loss → Backprop → Checkpoint