How ensta-u2is-ai/torch-uncertainty works
Project Type and Purpose
• This is an open-source Python package (“torch_uncertainty”) providing a modular framework for uncertainty quantification in deep neural networks.
• It delivers building blocks (layers, models, losses, metrics), data handling (datasets, datamodules, transforms), training/evaluation routines, CLI scripts, tutorials, and an experiments harness.
File Structure Overview
• Top-level folders:
• torch_uncertainty/ ← core library modules
• auto_tutorial_source/← Jupyter-style tutorial source code
• experiments/ ← runnable experiment scripts + YAML configs
• docs/ ← Sphinx documentation source
• scripts/ ← auxiliary scripts (e.g., to_hub_format.py)
• tests/ ← pytest-based unit and integration tests
• CI/deployment:
• .github/workflows/ ← build-docs, test runner, PyPI publishing
• Dockerfile + docker/ ← containerization for contributors
• Config files: pyproject.toml, codecov.yml, pre-commit, .gitignore
Key Components (to represent as separate blocks)
A. Data Layer
• torch_uncertainty/datasets/ ← raw dataset classes & utilities
• torch_uncertainty/datamodules/ ← PyTorch Lightning–style DataModules
• torch_uncertainty/transforms/ ← data augmentations & corruptions
B. Model Layer
• torch_uncertainty/layers/ ← custom layers (Bayesian, ensembling)
• torch_uncertainty/models/ ← backbone architectures + wrappers
• torch_uncertainty/losses/ ← task-specific loss functions
C. Metric & Post-processing Layer
• torch_uncertainty/metrics/ ← performance & uncertainty metrics
• torch_uncertainty/post_processing/← calibration, conformal, Laplace, etc.
D. Orchestration & Utilities
• torch_uncertainty/routines/ ← high-level train/eval loops for tasks
• torch_uncertainty/utils/ ← CLI helpers, checkpointing, plotting, evaluation loops
• torch_uncertainty/callbacks/ ← custom PyTorch Lightning callbacks
• torch_uncertainty/optim/ ← specialized optimizers & recipes
E. User-Facing Interfaces
• CLI entrypoint (utils/cli.py → console_scripts)
• scripts/to_hub_format.py
• auto_tutorial_source/ ← self-contained tutorials exposing the API
F. Experiments Harness
• experiments/*/ ← domain-specific scripts + YAML configs (classification, regression, segmentation, depth)
Relationships & Data Flow
1. User invokes CLI or Python API → selects a routines function (e.g., classification.train).
2. Routines instantiate a DataModule (datasets + transforms) → PyTorch Dataloaders.
3. Routines instantiate a model:
• Base architecture (models/) wraps layers (layers/)
• Optionally wrapped by ensembling or Bayesian wrappers (models/wrappers/)
4. Routines define loss (losses/) and metrics (metrics/) and attach callbacks (callbacks/).
5. Routines / Lightning Trainer run training:
• Data → Model → Loss → backprop → checkpoint via callbacks
6. After training: post-processing modules (post_processing/) calibrate or conformal predict.
7. Results plotted via utils/plotting or exported via scripts.
8. Experiments harness reads YAML configs → calls same routines with config parameters.
Architectural Patterns & Principles
• Modular plugin architecture: each capability (dataset, layer, model, loss, metric, post-processor) lives in its own namespace.
• Separation of concerns: data handling ↔ model definition ↔ training orchestration ↔ post-processing.
• Extensibility points:
• New layer under layers/
• New model or wrapper under models/
• New metric under metrics/
• New post-processor under post_processing/
• CI/CD pipeline automates testing, docs build, PyPI publishing.
• Docker container for reproducible contributor environment.
CI/CD & Documentation
• Sphinx docs generation: docs/source → ReadTheDocs/GitHub Pages.
• Dockerfile → Docker Hub (optional).