How bethgelab/foolbox works
Project type and purpose
• This is an open-source Python library (“Foolbox”) for running adversarial attacks on machine-learning models (PyTorch, TensorFlow, JAX) via a unified API.
• Its goal is to provide state-of-the-art gradient-based and decision-based attacks, distance metrics, criteria, and utilities to benchmark model robustness.
High-level analysis of the file structure
• Top-level directories:
• .github/ (CI/CD workflows for tests, docs, PyPI, funding)
• foolbox/ (core library modules: attacks, criteria, distances, models, utils, zoo)
• examples/ (usage scripts demonstrating attacks on ResNet, MNIST, etc.)
• docs/ (Sphinx documentation)
• guide/ (VuePress user guide)
• tests/ (pytest suite)
• performance/ (benchmark notebooks and data)
• Packaging/config: setup.py, pyproject.toml, MANIFEST.in, setup.cfg, requirements.txt
• Key patterns: modular attack classes inheriting from a base; a wrapper layer for multiple backends; separation of core library vs examples vs docs vs tests vs CI.
Important insights from README
• Built on top of EagerPy to abstract over multiple frameworks.
• Attack API: user creates a backend-specific Model wrapper, picks an attack from foolbox.attacks, passes images/labels/epsilons.
• Documentation and guide are published via ReadTheDocs and GitHub Pages (VuePress).
• Continuous integration runs tests, builds docs, and publishes to PyPI.
Main components & their relationships
a. User layer
• User code / Examples / Notebook – calls
• Foolbox API (foolbox/__init__.py) → selects
b. Attack orchestration
• Attack classes (foolbox/attacks/*.py)
• Criteria (foolbox/criteria.py)
• Distances (foolbox/distances.py)
• Gradient estimators & devutils (foolbox/gradient_estimators.py, foolbox/devutils.py)
• These reference
c. Model abstraction
• Model wrappers for PyTorch, TensorFlow, JAX (foolbox/models/*.py)
• Underlying: EagerPy abstraction
• Which dispatches to the actual backend frameworks: PyTorch, TF, JAX
d. Auxiliary modules
• Zoo (foolbox/zoo/): model loader, weight fetcher, git cloner
• Plot & TensorBoard logging (foolbox/plot.py, foolbox/tensorboard.py)
• Utilities & types (foolbox/utils.py, foolbox/types.py)
e. Infrastructure & pipelines
• Tests (tests/) run via pytest in CI
• Docs (docs/ + guide/) built by Sphinx and VuePress, published to ReadTheDocs and GitHub Pages
• CI workflows (.github/workflows/): tests, docs, PyPI deploy
• Packaging (setup.py, pyproject.toml, MANIFEST.in)
Architectural patterns & design principles
• Modularity: each attack, distance, and model wrapper lives in its own module.
• Abstraction layer: EagerPy hides backend differences.
• Open/Closed: new attacks or criteria can be added via subclassing.
• Separation of concerns: core code vs examples vs documentation vs tests vs CI.
Essential elements
• Clear component names matching directory names (e.g., foolbox.attacks, foolbox.models.pytorch)
• CI/CD flow linking .github/workflows to PyPI and ReadTheDocs