How tom99763/birdclef-2026-codebase works
1. Repository purpose
- This is a BirdCLEF 2026 competition codebase for multi-label audio classification on 5-second soundscape windows.
- The repo is organized around an iterative semi-supervised training loop, not a single model: train models, generate pseudo labels, retrain, ensemble, and post-process for Kaggle submission.
2. Top-level architecture
- The system has four main layers:
- Training: train.py, train_sed_ns.py, train_tucker_sed.py
- Pseudo-label generation: scripts that turn model outputs into new training labels
- Inference / ensemble / post-processing: ONNX export, blending, smoothing, and calibration notebooks/scripts
- Research and orchestration: reports/, wiki/, shell pipelines, and config YAMLs
- The repo is built to run many experiments with round-specific configs rather than one monolithic training job.
3. Core data boundary
- The main data boundary is between:
- labeled train_audio/
- unlabeled train_soundscapes/
- sparse labeled soundscapes
- generated pseudo labels in pseudo_labels/
- model checkpoints and inference artifacts in outputs/
- Each training round consumes a mix of ground-truth and pseudo-labeled audio, then emits new predictions for the next round.
4. Modeling subsystem: SED training
- train_sed_ns.py is the main sound event detection training entrypoint for the Noisy Student / Noisy Classmate family.
- The model stack in src/model/ includes backbones, heads, loss functions, PCEN, Perch adapters, and prototype/SSM variants.
- src/data/ handles audio datasets and augmentation; src/audio/ contains filtering logic for removing human-related sounds.
- src/metrics/ and src/utils/metrics.py implement competition-oriented evaluation helpers.
5. Noisy Student training chain
- The Noisy Student flow is a multi-round self-training loop:
- Perch teacher generates initial pseudo labels
- A student SED model trains on labeled audio plus pseudo-labeled soundscapes
- The student becomes the next teacher
- Configs in configs/sed_ns_b0_20s_r*.yaml and configs/sed_ns_pvt_20s_r*.yaml encode round-specific training settings.
- Shell scripts like scripts/auto_sed_ns_20s_full.sh and scripts/auto_sed_ns_pvt_20s_r*.sh orchestrate fold training, inference, residual correction, and pseudo-label regeneration.
6. Noisy Classmate subsystem
- The Noisy Classmate phase extends self-training into peer-teaching across heterogeneous students, mainly B0 and PVT.
- Pseudo labels are generated by blending multiple model streams in scripts/gen_noisy_classmate_pseudo.py, with optional confidence weighting, disagreement mining, and soft-label distillation.
- The training loop also supports bidirectional co-evolution, where models teach each other rather than only self-distill.
7. Residual correction and temporal post-processing
- scripts/train_sed_residual_corrector.py trains a correction model on top of raw SED probabilities.
- The correction layer is small and separate from the main classifier, so it acts as a post-inference refinement boundary rather than changing the base model.
- The event_smooth/ and sed_temporal_smooth/ directories contain notebooks and scripts for smoothing and post-processing ensembles, showing that final score gains come from inference-time refinement as much as from training.
8. Alternative student family: Tucker SED / Tucker NS
- Tucker SED is a separate distilled model family with its own training and inference logic.
- train_tucker_sed.py, scripts/cache_unlabeled_ss.py, and scripts/infer_tucker_ns_ss.py support a pipeline that caches unlabeled soundscapes and trains on mixed streams.
- This branch is architecturally important because it provides a second inference pipeline that can be ensembled with the main VLOM/SED system.
9. Inference, export, and submission boundary
- scripts/export_sed_to_onnx.py converts trained PyTorch checkpoints into ONNX, with quantized export for deployment.
- Submission notebooks in submissions/ and submissions_v3/ are the final execution boundary: they load exported models, run sliding-window inference, ensemble predictions, and produce Kaggle-ready outputs.
- The repo’s final competitive system depends on model diversity across rounds, folds, and backbone families rather than a single best checkpoint.
10. Ensembling and blending subsystem
- src/utils/model_soup.py and scripts/optimize_sed_weights.py support model combination and weight search.
- The repo uses multiple blend strategies:
- logit-space blending via VLOM
- per-class weighting
- rank-based blending between pipelines
- fold/round diversity selection
- This is a distinct architectural layer: it consumes model outputs, not training data.
11. Experiment management and automation
- The scripts/ directory contains the operational pipeline drivers, monitors, watchdogs, and batch exporters.
- The many YAML configs in configs/ make each experiment round explicit and reproducible.
- Shell scripts like round2_pipeline.sh, run_pseudo_pipeline.sh, monitor_and_advance.sh, and next_stage.sh show that the system is designed as a long-running experiment chain with checkpointed handoff between stages.
12. Knowledge and reporting layer
- reports/ contains generated analysis artifacts, papers, tables, and figures; wiki/ contains the living architecture notes and experiment findings.
- These are not runtime dependencies, but they are part of the repo’s architecture because they document design decisions, ablations, and why certain model combinations are used.
- For diagramming, they can be shown as an analysis/documentation sidecar around the core training/inference pipeline.
13. Important boundary summary for a diagram
- The cleanest architecture view is:
- Audio data sources → preprocessing/augmentation → trainable SED/Tucker models
- model outputs → pseudo-label generation
- pseudo labels + labeled data → next training round
- checkpoints → ONNX export / ensemble / post-processing
- final predictions → Kaggle submission
- The repo’s novelty is in the repeated loop across these boundaries, with multiple model families feeding each other and a separate post-processing layer improving inference after training.
Main entry points: train_sed_ns.py SED NS · train_tucker_sed.py Tucker SED · submissions_v3 Submission v3 · scripts Pipelines
Components 22
Training
Model training loops for SED, Noisy Student, Noisy Classmate, and Tucker variants.
- train.py · General training launcher for experiments.
train.py - SED NS · Main Noisy Student / Noisy Classmate training entrypoint for sound event detection.
train_sed_ns.py - Tucker SED · Alternate student family with its own training loop and mixed-stream setup.
train_tucker_sed.py - Data · Dataset loading, mixing, and augmentation for labeled and unlabeled audio.
src/data - Audio filter · Filtering logic for removing human-related sounds from training streams.
src/audio - Model stack · Backbones, heads, losses, PCEN, Perch adapters, and prototype/SSM variants.
src/model - Metrics · Competition-oriented scoring and evaluation utilities.
src/metrics
Pseudo-labeling
Generate and refresh labels from teacher model outputs for the next round.
- Pseudo labels · Generates round-specific pseudo labels from teacher predictions.
scripts/gen_pseudo_ns.py - NC pseudo · Blends multiple model streams into Noisy Classmate pseudo labels.
scripts/gen_noisy_classmate_pseudo.py - Perch teacher · Extracts teacher outputs over soundscapes for pseudo-label bootstrapping.
scripts/extract_perch_teacher_all_ss.py
Inference
Export, ensemble, smooth, and post-process predictions for submission.
- ONNX export · Converts trained checkpoints to ONNX, including quantized export paths.
scripts/export_sed_to_onnx.py - Model soup · Model combination and weight-search utilities for checkpoint ensembling.
src/utils/model_soup.py - Weight search · Searches blend weights and per-class combination settings.
scripts/optimize_sed_weights.py - Residual corrector · Trains a small correction model on raw SED probabilities.
scripts/train_sed_residual_corrector.py - Event smooth · Event smoothing and ensemble post-processing notebooks/scripts.
event_smooth - Temporal smooth · Alternative temporal smoothing and post-processing pipeline.
sed_temporal_smooth - Submission v3 · Final notebook-based inference and submission assembly.
submissions_v3
Orchestration
Configs, shell pipelines, and experiment automation across rounds.
- Configs · Round-specific YAML experiment definitions and training settings.
configs - Pipelines · Shell and Python drivers for training, pseudo-labeling, export, and monitoring.
scripts - Shell chains · Long-running pipeline scripts that advance rounds and hand off checkpoints.
round2_pipeline.sh
Connections 29
- Data → SED NS: loads · Training consumes labeled and pseudo-labeled datasets.
- Audio filter → SED NS: filters · Human-sound filtering shapes the training stream.
- Model stack → SED NS: implements · Model stack supplies backbones, heads, and losses.
- Metrics → SED NS: scores · Competition metrics guide training and validation.
- Configs → SED NS: configures · Round-specific YAMLs select model family and schedule.
- Configs → Tucker SED: configures · Configs define Tucker-specific experiment rounds.
- Data → Tucker SED: loads · Tucker training uses cached unlabeled and labeled streams.
- Model stack → Tucker SED: implements · Alternate student family reuses core model components.
- Pipelines → SED NS: orchestrates · Automation scripts drive folds, rounds, and reruns.
- Pipelines → Tucker SED: orchestrates · Batch drivers run Tucker pipelines end to end.
- Perch teacher → Pseudo labels: feeds · Teacher outputs seed the pseudo-label generation loop.
- SED NS → Pseudo labels: produces · Student checkpoints generate new pseudo labels for later rounds.
- SED NS → NC pseudo: produces · Multiple student outputs are blended into peer-teaching labels.
- Pseudo labels → SED NS: relabels · Generated pseudo labels are consumed by the next training round.
- NC pseudo → SED NS: relabels · Noisy Classmate pseudo labels feed subsequent peer training.
- SED NS → ONNX export: exports · Trained checkpoints are converted for inference deployment.
- Tucker SED → ONNX export: exports · Tucker checkpoints also enter the deployment boundary.
- ONNX export → Submission v3: serves · Submission notebooks load exported models for inference.
- Model soup → Weight search: tunes · Weight search optimizes blend composition.
- Weight search → Submission v3: blends · Optimized ensemble weights are applied in final inference.
- Residual corrector → Event smooth: refines · Residual correction is layered into final post-processing.
- Event smooth → Submission v3: postprocesses · Smoothed event probabilities feed the submission pipeline.
- Temporal smooth → Submission v3: postprocesses · Temporal smoothing provides an alternate inference refinement path.
- Shell chains → Pipelines: drives · Shell chains coordinate scripted training and pseudo-label stages.
- Shell chains → Configs: selects · Pipeline scripts choose round-specific configurations.
- Reports → SED NS: analyzes · Reports summarize training behavior and ablation results.
- Wiki → Pipelines: documents · Architecture notes describe automation and pipeline flow.
- Wiki → SED NS: documents · Wiki pages explain the noisy-student architecture.
- Wiki → Submission v3: documents · Wiki pages describe final inference and submission assembly.