How cure-lab/deep-active-learning works
1. What this repo is
This is a PyTorch-based deep active learning experiment framework. It is not a single model; it is an orchestration layer that repeatedly trains a classifier, scores an unlabeled pool, acquires new samples, and retrains over multiple rounds.
2. Top-level control flow
main.py is the experiment entrypoint, and run.sh is the convenience wrapper. The main loop is likely:
- load dataset and create initial labeled/unlabeled splits
- instantiate a backbone model
- train on the labeled subset
- evaluate on a held-out test set
- run a query strategy to select samples from the unlabeled pool
- move queried samples into the labeled pool
- repeat for several acquisition rounds
Results are saved under ./save.
3. Dataset boundary
dataset.py is the data boundary for the whole system. It likely handles dataset loading, train/test partitioning, and active-learning pool management for common image benchmarks such as CIFAR-10, as shown in the README example. This is where the labeled set, unlabeled set, and test set are defined and tracked across rounds.
4. Model zoo / training backends
The models/ directory contains the supervised learners used inside the active learning loop:
- standard CNN backbones: resnet.py, resnet64.py, vgg.py, mobilenet.py, wideresnet.py, shufflenet.py
- lightweight or auxiliary heads: linear.py, lossnet.py
- graph/network variants: gcn.py
- method-specific wrappers: wa_model.py
These modules are the main “training target” consumed by strategies; many query methods depend on the model’s logits, embeddings, gradients, or uncertainty estimates.
5. Strategy subsystem
The query_strategies/ package is the core architectural differentiator. strategy.py appears to define the common strategy interface/base class, while util.py and cpu_dist.py provide shared helpers and distance computations. Each strategy module implements one acquisition policy over the unlabeled pool.
6. Uncertainty-based acquisition
Several strategies select points by model uncertainty, using outputs from the current trained classifier:
- least_confidence.py, margin_sampling.py, entropy_sampling.py
- dropout variants: *_dropout.py
- ensemble/Bayesian methods: bayesian_active_learning_disagreement_dropout.py, batch_BALD.py
- simple baseline/random: baseline_sampling.py, random_sampling.py
These methods typically require only predictions from the trained model, so they are the least invasive path through the system.
7. Diversity / embedding-based acquisition
Another major branch uses the learned feature space or sample geometry rather than just uncertainty:
- core_set.py, kcenter_greedy.py, kmeans_sampling.py
- badge_sampling.py
- batch_active_learning_at_scale.py
- selection_via_proxy.py
These strategies depend on embeddings/gradients extracted from the model and are used to encourage coverage of the unlabeled space, not just uncertainty.
8. Adversarial / generative strategies
The repository also implements strategies that rely on adversarial objectives or generators:
- adversarial_bim.py, adversarial_deepfool.py
- vaal.py, wasserstein_adversarial.py
- aug_uda.py, augmentations.py
- mcadl.py
These introduce extra components beyond the classifier, such as discriminator-like models or augmentation pipelines, so the architecture expands from “train classifier + score pool” into multi-network training for selection.
9. Learning-to-rank / learned acquisition
Some strategies explicitly learn a query function:
- active_learning_by_learning.py
- learning_loss_for_al.py with models/lossnet.py
- lal.py
- batch_active_learning_at_scale.py
These methods add a second-stage predictor that estimates which unlabeled points will be most useful, based on features from the current model and/or metadata about training progress.
10. Graph-based active learning
models/gcn.py and strategies like coreGCN.py, uncertainGCN.py show a graph-centric branch. These methods build a graph over samples/features and apply GCN-style reasoning to choose queries. Architecturally, this is a distinct path because selection depends on neighborhood structure, not just pointwise classification output.
11. Semi-supervised + active learning hybrids
The semi_* and ssl_* modules implement methods that mix query acquisition with unlabeled-loss training:
- semi_strategy.py, semi_fixmatch.py, semi_flexmatch.py, semi_pseudolabel.py
- ssl_consistency.py, ssl_least_confidence.py, ssl_rand.py, ssl_diff2augdirect.py, ssl_diff2augkmeans.py
These strategies consume both labeled and unlabeled data during training, so the unlabeled pool is not only for selection; it also contributes to representation learning between acquisition rounds.
12. Experiment outputs and reproducibility
The repo is set up as an offline research benchmark rather than a service. requirements.txt pins the Python/PyTorch/scikit-learn stack, and the README notes Python 3.7 and PyTorch 1.4.0. The main observable output is experiment results saved in ./save, likely including metrics per round, selected indices, and model checkpoints.
13. Important architectural boundary to diagram
The key boundary is between:
- shared experiment orchestration (main.py, dataset.py, utils.py)
- backbone models (models/)
- acquisition policies (query_strategies/)
Each strategy plugs into the same pool-based active-learning loop but may require different model outputs or auxiliary networks. That plug-in architecture is the central design of the repository.
Main entry points: strategy.py Strategy base · main.py Main loop · semi_strategy.py Semi-supervised base · learning_loss_for_al.py Learn loss
Components 23
Orchestration
Entry point, dataset boundary, and shared experiment helpers.
- Main loop · Runs active-learning rounds: load data, train, evaluate, query, and update pools.
main.py - Run script · Convenience shell wrapper for launching experiments.
run.sh - Dataset boundary · Loads benchmarks and maintains labeled, unlabeled, and test splits.
dataset.py - Shared utils · Common experiment utilities used across training and querying.
utils.py
Models
Backbone and auxiliary networks trained inside active-learning rounds.
- ResNet · Standard convolutional classifier backbone.
models/resnet.py - WideResNet · Higher-capacity convolutional classifier backbone.
models/wideresnet.py - GCN model · Graph-based learner used by graph-centric acquisition methods.
models/gcn.py - LossNet · Predicts learning loss for learned acquisition strategies.
models/lossnet.py - WA model · Method-specific model wrapper used by some strategies.
models/wa_model.py - Linear head · Lightweight prediction head used in simpler setups.
models/linear.py
Query Strategies
Acquisition policies and semi-supervised variants over the unlabeled pool.
- Strategy base · Common interface implemented by all query policies.
query_strategies/strategy.py - Strategy utils · Shared helpers for query strategies and scoring.
query_strategies/util.py - Random · Simple random baseline for pool selection.
query_strategies/random_sampling.py - Entropy · Selects points with high predictive entropy.
query_strategies/entropy_sampling.py - Core-set · Covers feature space by selecting representative points.
query_strategies/core_set.py - BADGE · Uses gradient embeddings to balance uncertainty and diversity.
query_strategies/badge_sampling.py - VAAL · Uses a generative/adversarial learner to query informative samples.
query_strategies/vaal.py - Semi-supervised base · Base class for methods that train on labeled and unlabeled data.
query_strategies/semi_strategy.py - SSL consistency · Combines consistency training with active selection.
query_strategies/ssl_consistency.py - FixMatch · Semi-supervised active-learning variant using FixMatch-style training.
query_strategies/semi_fixmatch.py - Learn loss · Learns a query signal using LossNet-style loss prediction.
query_strategies/learning_loss_for_al.py - ALBL · Active learning by learning a query policy from feedback.
query_strategies/active_learning_by_learning.py - CoreGCN · Graph-based selection using neighborhood structure and GCN reasoning.
query_strategies/coreGCN.py
Connections 23
- Run script → Main loop: launches · Shell wrapper invokes the experiment driver.
- Main loop → Dataset boundary: loads · Main loop creates labeled/unlabeled/test splits.
- Main loop → Shared utils: uses · Shared helpers support training, evaluation, and bookkeeping.
- Main loop → ResNet: trains · One of the main classifier backbones used in rounds.
- Main loop → WideResNet: trains · Alternative backbone for supervised rounds.
- Main loop → GCN model: trains · Used by graph-centric acquisition methods.
- Main loop → LossNet: attaches · Auxiliary head for learning-loss strategies.
- Main loop → WA model: wraps · Method-specific wrapper for some strategies.
- Main loop → Strategy base: selects via · Experiment loop delegates pool selection to a strategy implementation.
- Strategy base → Strategy utils: depends on · Strategies share helper code and scoring utilities.
- Strategy base → Random: implements · Baseline acquisition policy.
- Strategy base → Entropy: implements · Uncertainty-based acquisition policy.
- Strategy base → Core-set: implements · Diversity-based acquisition policy.
- Strategy base → BADGE: implements · Gradient-embedding acquisition policy.
- Strategy base → VAAL: implements · Adversarial acquisition policy.
- Strategy base → Semi-supervised base: extends · Base for semi-supervised active-learning methods.
- Semi-supervised base → SSL consistency: specializes · Adds consistency-based unlabeled training.
- Semi-supervised base → FixMatch: specializes · Adds FixMatch-style semi-supervised training.
- Strategy base → Learn loss: implements · Learned acquisition based on loss prediction.
- Strategy base → ALBL: implements · Learned acquisition by policy learning.
- Strategy base → CoreGCN: implements · Graph-based acquisition policy.
- Learn loss → LossNet: uses · Depends on auxiliary loss prediction network.
- CoreGCN → GCN model: uses · Relies on graph neural network reasoning.