# tom99763/birdclef-2026-codebase architecture

> Architecture diagram of the GitHub repository tom99763/birdclef-2026-codebase, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tom99763/birdclef-2026-codebase
- Repository: https://github.com/tom99763/birdclef-2026-codebase

## Overview

1. **Repository purpose**
   - This is a BirdCLEF 2026 competition codebase for multi-label audio classification on 5-second soundscape windows.
   - The repo is organized around an iterative semi-supervised training loop, not a single model: train models, generate pseudo labels, retrain, ensemble, and post-process for Kaggle submission.

2. **Top-level architecture**
   - The system has four main layers:
     - **Training**: `train.py`, `train_sed_ns.py`, `train_tucker_sed.py`
     - **Pseudo-label generation**: scripts that turn model outputs into new training labels
     - **Inference / ensemble / post-processing**: ONNX export, blending, smoothing, and calibration notebooks/scripts
     - **Research and orchestration**: `reports/`, `wiki/`, shell pipelines, and config YAMLs
   - The repo is built to run many experiments with round-specific configs rather than one monolithic training job.

3. **Core data boundary**
   - The main data boundary is between:
     - labeled `train_audio/`
     - unlabeled `train_soundscapes/`
     - sparse labeled soundscapes
     - generated pseudo labels in `pseudo_labels/`
     - model checkpoints and inference artifacts in `outputs/`
   - Each training round consumes a mix of ground-truth and pseudo-labeled audio, then emits new predictions for the next round.

4. **Modeling subsystem: SED training**
   - `train_sed_ns.py` is the main sound event detection training entrypoint for the Noisy Student / Noisy Classmate family.
   - The model stack in `src/model/` includes backbones, heads, loss functions, PCEN, Perch adapters, and prototype/SSM variants.
   - `src/data/` handles audio datasets and augmentation; `src/audio/` contains filtering logic for removing human-related sounds.
   - `src/metrics/` and `src/utils/metrics.py` implement competition-oriented evaluation helpers.

5. **Noisy Student training chain**
   - The Noisy Student flow is a multi-round self-training loop:
     - Perch teacher generates initial pseudo labels
     - A student SED model trains on labeled audio plus pseudo-labeled soundscapes
     - The student becomes the next teacher
   - Configs in `configs/sed_ns_b0_20s_r*.yaml` and `configs/sed_ns_pvt_20s_r*.yaml` encode round-specific training settings.
   - Shell scripts like `scripts/auto_sed_ns_20s_full.sh` and `scripts/auto_sed_ns_pvt_20s_r*.sh` orchestrate fold training, inference, residual correction, and pseudo-label regeneration.

6. **Noisy Classmate subsystem**
   - The Noisy Classmate phase extends self-training into peer-teaching across heterogeneous students, mainly B0 and PVT.
   - Pseudo labels are generated by blending multiple model streams in `scripts/gen_noisy_classmate_pseudo.py`, with optional confidence weighting, disagreement mining, and soft-label distillation.
   - The training loop also supports bidirectional co-evolution, where models teach each other rather than only self-distill.

7. **Residual correction and temporal post-processing**
   - `scripts/train_sed_residual_corrector.py` trains a correction model on top of raw SED probabilities.
   - The correction layer is small and separate from the main classifier, so it acts as a post-inference refinement boundary rather than changing the base model.
   - The `event_smooth/` and `sed_temporal_smooth/` directories contain notebooks and scripts for smoothing and post-processing ensembles, showing that final score gains come from inference-time refinement as much as from training.

8. **Alternative student family: Tucker SED / Tucker NS**
   - Tucker SED is a separate distilled model family with its own training and inference logic.
   - `train_tucker_sed.py`, `scripts/cache_unlabeled_ss.py`, and `scripts/infer_tucker_ns_ss.py` support a pipeline that caches unlabeled soundscapes and trains on mixed streams.
   - This branch is architecturally important because it provides a second inference pipeline that can be ensembled with the main VLOM/SED system.

9. **Inference, export, and submission boundary**
   - `scripts/export_sed_to_onnx.py` converts trained PyTorch checkpoints into ONNX, with quantized export for deployment.
   - Submission notebooks in `submissions/` and `submissions_v3/` are the final execution boundary: they load exported models, run sliding-window inference, ensemble predictions, and produce Kaggle-ready outputs.
   - The repo’s final competitive system depends on model diversity across rounds, folds, and backbone families rather than a single best checkpoint.

10. **Ensembling and blending subsystem**
   - `src/utils/model_soup.py` and `scripts/optimize_sed_weights.py` support model combination and weight search.
   - The repo uses multiple blend strategies:
     - logit-space blending via VLOM
     - per-class weighting
     - rank-based blending between pipelines
     - fold/round diversity selection
   - This is a distinct architectural layer: it consumes model outputs, not training data.

11. **Experiment management and automation**
   - The `scripts/` directory contains the operational pipeline drivers, monitors, watchdogs, and batch exporters.
   - The many YAML configs in `configs/` make each experiment round explicit and reproducible.
   - Shell scripts like `round2_pipeline.sh`, `run_pseudo_pipeline.sh`, `monitor_and_advance.sh`, and `next_stage.sh` show that the system is designed as a long-running experiment chain with checkpointed handoff between stages.

12. **Knowledge and reporting layer**
   - `reports/` contains generated analysis artifacts, papers, tables, and figures; `wiki/` contains the living architecture notes and experiment findings.
   - These are not runtime dependencies, but they are part of the repo’s architecture because they document design decisions, ablations, and why certain model combinations are used.
   - For diagramming, they can be shown as an analysis/documentation sidecar around the core training/inference pipeline.

13. **Important boundary summary for a diagram**
   - The cleanest architecture view is:
     - **Audio data sources** → preprocessing/augmentation → **trainable SED/Tucker models**
     - model outputs → **pseudo-label generation**
     - pseudo labels + labeled data → next training round
     - checkpoints → **ONNX export / ensemble / post-processing**
     - final predictions → **Kaggle submission**
   - The repo’s novelty is in the repeated loop across these boundaries, with multiple model families feeding each other and a separate post-processing layer improving inference after training.

Main entry points: [`train_sed_ns.py`](https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_sed_ns.py) (SED NS), [`train_tucker_sed.py`](https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_tucker_sed.py) (Tucker SED), [`submissions_v3`](https://github.com/tom99763/birdclef-2026-codebase/tree/main/submissions_v3) (Submission v3), [`scripts`](https://github.com/tom99763/birdclef-2026-codebase/tree/main/scripts) (Pipelines)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training"]
  node_train_py["train.py<br/>training entrypoint<br/>[train.py]"]
  node_train_sed_ns_py["SED NS<br/>noisy-student SED trainer<br/>[train_sed_ns.py]"]
  node_train_tucker_sed_py["Tucker SED<br/>distilled trainer"]
  node_src_data[("Data")]
  node_src_audio["Audio filter<br/>human-sound filter"]
  node_src_model["Model stack<br/>SED models"]
  node_src_metrics["Metrics<br/>eval helpers"]
end

subgraph group_pseudo["Pseudo-labeling"]
  node_gen_pseudo_ns["Pseudo labels<br/>teacher labeling<br/>[gen_pseudo_ns.py]"]
  node_gen_nc_pseudo["NC pseudo<br/>blend labeling"]
  node_extract_perch["Perch teacher<br/>teacher export"]
end

subgraph group_inference["Inference"]
  node_export_onnx["ONNX export<br/>model export"]
  node_model_soup["Model soup<br/>blend utilities<br/>[model_soup.py]"]
  node_optimize_weights["Weight search<br/>ensemble tuning"]
  node_residual_corrector["Residual corrector<br/>postproc trainer"]
  node_event_smooth["Event smooth<br/>temporal postproc"]
  node_sed_temporal_smooth["Temporal smooth<br/>temporal postproc"]
  node_submission_v3["Submission v3<br/>kaggle inference"]
end

subgraph group_ops["Orchestration"]
  node_configs["Configs"]
  node_scripts["Pipelines<br/>automation scripts"]
  node_pipeline_sh["Shell chains<br/>experiment orchestration<br/>[round2_pipeline.sh]"]
end

subgraph group_knowledge["Docs"]
  node_reports["Reports<br/>analysis artifacts"]
  node_wiki["Wiki<br/>architecture notes"]
end

node_src_data -->|"loads"| node_train_sed_ns_py
node_src_audio -->|"filters"| node_train_sed_ns_py
node_src_model -->|"implements"| node_train_sed_ns_py
node_src_metrics -->|"scores"| node_train_sed_ns_py
node_configs -->|"configures"| node_train_sed_ns_py
node_configs -->|"configures"| node_train_tucker_sed_py
node_src_data -->|"loads"| node_train_tucker_sed_py
node_src_model -->|"implements"| node_train_tucker_sed_py
node_scripts -->|"orchestrates"| node_train_sed_ns_py
node_scripts -->|"orchestrates"| node_train_tucker_sed_py
node_extract_perch -->|"feeds"| node_gen_pseudo_ns
node_train_sed_ns_py -->|"produces"| node_gen_pseudo_ns
node_train_sed_ns_py -->|"produces"| node_gen_nc_pseudo
node_gen_pseudo_ns -->|"relabels"| node_train_sed_ns_py
node_gen_nc_pseudo -->|"relabels"| node_train_sed_ns_py
node_train_sed_ns_py -->|"exports"| node_export_onnx
node_train_tucker_sed_py -->|"exports"| node_export_onnx
node_export_onnx -->|"serves"| node_submission_v3
node_model_soup -->|"tunes"| node_optimize_weights
node_optimize_weights -->|"blends"| node_submission_v3
node_residual_corrector -->|"refines"| node_event_smooth
node_event_smooth -->|"postprocesses"| node_submission_v3
node_sed_temporal_smooth -->|"postprocesses"| node_submission_v3
node_pipeline_sh -->|"drives"| node_scripts
node_pipeline_sh -->|"selects"| node_configs
node_reports -.->|"analyzes"| node_train_sed_ns_py
node_wiki -.->|"documents"| node_scripts
node_wiki -.->|"documents"| node_train_sed_ns_py
node_wiki -.->|"documents"| node_submission_v3

click node_train_py "https://github.com/tom99763/birdclef-2026-codebase/blob/main/train.py"
click node_train_sed_ns_py "https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_sed_ns.py"
click node_train_tucker_sed_py "https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_tucker_sed.py"
click node_src_data "https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/data"
click node_src_audio "https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/audio"
click node_src_model "https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/model"
click node_src_metrics "https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/metrics"
click node_gen_pseudo_ns "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/gen_pseudo_ns.py"
click node_gen_nc_pseudo "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/gen_noisy_classmate_pseudo.py"
click node_extract_perch "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/extract_perch_teacher_all_ss.py"
click node_export_onnx "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/export_sed_to_onnx.py"
click node_model_soup "https://github.com/tom99763/birdclef-2026-codebase/blob/main/src/utils/model_soup.py"
click node_optimize_weights "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/optimize_sed_weights.py"
click node_residual_corrector "https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/train_sed_residual_corrector.py"
click node_event_smooth "https://github.com/tom99763/birdclef-2026-codebase/tree/main/event_smooth"
click node_sed_temporal_smooth "https://github.com/tom99763/birdclef-2026-codebase/tree/main/sed_temporal_smooth"
click node_submission_v3 "https://github.com/tom99763/birdclef-2026-codebase/tree/main/submissions_v3"
click node_configs "https://github.com/tom99763/birdclef-2026-codebase/tree/main/configs"
click node_scripts "https://github.com/tom99763/birdclef-2026-codebase/tree/main/scripts"
click node_pipeline_sh "https://github.com/tom99763/birdclef-2026-codebase/blob/main/round2_pipeline.sh"
click node_reports "https://github.com/tom99763/birdclef-2026-codebase/tree/main/reports"
click node_wiki "https://github.com/tom99763/birdclef-2026-codebase/tree/main/wiki"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_py,node_train_sed_ns_py,node_train_tucker_sed_py,node_src_data,node_src_audio,node_src_model,node_src_metrics toneBlue
class node_gen_pseudo_ns,node_gen_nc_pseudo,node_extract_perch toneAmber
class node_export_onnx,node_model_soup,node_optimize_weights,node_residual_corrector,node_event_smooth,node_sed_temporal_smooth,node_submission_v3 toneMint
class node_configs,node_scripts,node_pipeline_sh toneRose
class node_reports,node_wiki toneIndigo
```

## Components

### Training

Model training loops for SED, Noisy Student, Noisy Classmate, and Tucker variants.

- [train.py](https://github.com/tom99763/birdclef-2026-codebase/blob/main/train.py): training entrypoint · General training launcher for experiments. · `train.py`
- [SED NS](https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_sed_ns.py): noisy-student SED trainer · Main Noisy Student / Noisy Classmate training entrypoint for sound event detection. · `train_sed_ns.py`
- [Tucker SED](https://github.com/tom99763/birdclef-2026-codebase/blob/main/train_tucker_sed.py): distilled trainer · Alternate student family with its own training loop and mixed-stream setup. · `train_tucker_sed.py`
- [Data](https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/data): audio datasets · Dataset loading, mixing, and augmentation for labeled and unlabeled audio. · `src/data`
- [Audio filter](https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/audio): human-sound filter · Filtering logic for removing human-related sounds from training streams. · `src/audio`
- [Model stack](https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/model): SED models · Backbones, heads, losses, PCEN, Perch adapters, and prototype/SSM variants. · `src/model`
- [Metrics](https://github.com/tom99763/birdclef-2026-codebase/tree/main/src/metrics): eval helpers · Competition-oriented scoring and evaluation utilities. · `src/metrics`

### Pseudo-labeling

Generate and refresh labels from teacher model outputs for the next round.

- [Pseudo labels](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/gen_pseudo_ns.py): teacher labeling · Generates round-specific pseudo labels from teacher predictions. · `scripts/gen_pseudo_ns.py`
- [NC pseudo](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/gen_noisy_classmate_pseudo.py): blend labeling · Blends multiple model streams into Noisy Classmate pseudo labels. · `scripts/gen_noisy_classmate_pseudo.py`
- [Perch teacher](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/extract_perch_teacher_all_ss.py): teacher export · Extracts teacher outputs over soundscapes for pseudo-label bootstrapping. · `scripts/extract_perch_teacher_all_ss.py`

### Inference

Export, ensemble, smooth, and post-process predictions for submission.

- [ONNX export](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/export_sed_to_onnx.py): model export · Converts trained checkpoints to ONNX, including quantized export paths. · `scripts/export_sed_to_onnx.py`
- [Model soup](https://github.com/tom99763/birdclef-2026-codebase/blob/main/src/utils/model_soup.py): blend utilities · Model combination and weight-search utilities for checkpoint ensembling. · `src/utils/model_soup.py`
- [Weight search](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/optimize_sed_weights.py): ensemble tuning · Searches blend weights and per-class combination settings. · `scripts/optimize_sed_weights.py`
- [Residual corrector](https://github.com/tom99763/birdclef-2026-codebase/blob/main/scripts/train_sed_residual_corrector.py): postproc trainer · Trains a small correction model on raw SED probabilities. · `scripts/train_sed_residual_corrector.py`
- [Event smooth](https://github.com/tom99763/birdclef-2026-codebase/tree/main/event_smooth): temporal postproc · Event smoothing and ensemble post-processing notebooks/scripts. · `event_smooth`
- [Temporal smooth](https://github.com/tom99763/birdclef-2026-codebase/tree/main/sed_temporal_smooth): temporal postproc · Alternative temporal smoothing and post-processing pipeline. · `sed_temporal_smooth`
- [Submission v3](https://github.com/tom99763/birdclef-2026-codebase/tree/main/submissions_v3): kaggle inference · Final notebook-based inference and submission assembly. · `submissions_v3`

### Orchestration

Configs, shell pipelines, and experiment automation across rounds.

- [Configs](https://github.com/tom99763/birdclef-2026-codebase/tree/main/configs): round configs · Round-specific YAML experiment definitions and training settings. · `configs`
- [Pipelines](https://github.com/tom99763/birdclef-2026-codebase/tree/main/scripts): automation scripts · Shell and Python drivers for training, pseudo-labeling, export, and monitoring. · `scripts`
- [Shell chains](https://github.com/tom99763/birdclef-2026-codebase/blob/main/round2_pipeline.sh): experiment orchestration · Long-running pipeline scripts that advance rounds and hand off checkpoints. · `round2_pipeline.sh`

### Docs

Reports, wiki notes, and analysis artifacts that explain the system.

- [Reports](https://github.com/tom99763/birdclef-2026-codebase/tree/main/reports): analysis artifacts · Generated reports, figures, tables, and experiment summaries. · `reports`
- [Wiki](https://github.com/tom99763/birdclef-2026-codebase/tree/main/wiki): architecture notes · Living documentation for architecture, training, and experiment findings. · `wiki`

## Connections

- Data → SED NS: loads · Training consumes labeled and pseudo-labeled datasets.
- Audio filter → SED NS: filters · Human-sound filtering shapes the training stream.
- Model stack → SED NS: implements · Model stack supplies backbones, heads, and losses.
- Metrics → SED NS: scores · Competition metrics guide training and validation.
- Configs → SED NS: configures · Round-specific YAMLs select model family and schedule.
- Configs → Tucker SED: configures · Configs define Tucker-specific experiment rounds.
- Data → Tucker SED: loads · Tucker training uses cached unlabeled and labeled streams.
- Model stack → Tucker SED: implements · Alternate student family reuses core model components.
- Pipelines → SED NS: orchestrates · Automation scripts drive folds, rounds, and reruns.
- Pipelines → Tucker SED: orchestrates · Batch drivers run Tucker pipelines end to end.
- Perch teacher → Pseudo labels: feeds · Teacher outputs seed the pseudo-label generation loop.
- SED NS → Pseudo labels: produces · Student checkpoints generate new pseudo labels for later rounds.
- SED NS → NC pseudo: produces · Multiple student outputs are blended into peer-teaching labels.
- Pseudo labels → SED NS: relabels · Generated pseudo labels are consumed by the next training round.
- NC pseudo → SED NS: relabels · Noisy Classmate pseudo labels feed subsequent peer training.
- SED NS → ONNX export: exports · Trained checkpoints are converted for inference deployment.
- Tucker SED → ONNX export: exports · Tucker checkpoints also enter the deployment boundary.
- ONNX export → Submission v3: serves · Submission notebooks load exported models for inference.
- Model soup → Weight search: tunes · Weight search optimizes blend composition.
- Weight search → Submission v3: blends · Optimized ensemble weights are applied in final inference.
- Residual corrector → Event smooth: refines · Residual correction is layered into final post-processing.
- Event smooth → Submission v3: postprocesses · Smoothed event probabilities feed the submission pipeline.
- Temporal smooth → Submission v3: postprocesses · Temporal smoothing provides an alternate inference refinement path.
- Shell chains → Pipelines: drives · Shell chains coordinate scripted training and pseudo-label stages.
- Shell chains → Configs: selects · Pipeline scripts choose round-specific configurations.
- Reports → SED NS: analyzes · Reports summarize training behavior and ablation results.
- Wiki → Pipelines: documents · Architecture notes describe automation and pipeline flow.
- Wiki → SED NS: documents · Wiki pages explain the noisy-student architecture.
- Wiki → Submission v3: documents · Wiki pages describe final inference and submission assembly.
