How geometric-intelligence/topobench works
Main System Components
• Command-Line Interface & Entry Point: At the very top is the “python -m topobench” invocation that starts execution via topobench/__main__.py.
• Configuration Manager: The project leverages Hydra to manage a hierarchical configuration system (found in the configs/ directory with subdirectories for models, datasets, trainers, transforms, etc.). This component is crucial as it dynamically instantiates the chosen model, dataset, transforms, and experiment settings.
• Data Pipeline:
– Preprocessor & Transforms: The project supports multiple pre-processing steps through “data_manipulations” and “liftings” (both under configs/transforms and topobench/transforms). These components apply modifications such as feature conversion, connectivity inference, and domain transformations (e.g., graph to simplicial or graph to cell).
• Neural Network Components:
– Models & Backbones: The core of topological deep learning is implemented in topobench/nn and topobench/model. This includes the diverse set of TNN architectures (e.g., CWN, CCCN, GCN, GAT, etc.) as well as wrappers that manage training details.
• Training & Optimization:
– Trainer & Optimizer: The training loop (coordinated by topobench/run.py and related trainer configurations in configs/trainer) uses optimizer settings (found in topobench/optimizer and configs/optimizer), loss functions (in topobench/loss and configs/loss), and evaluation metrics (topobench/evaluator) to guide learning.
• Logging, Callbacks, and Evaluation:
– Logging: There are multiple logger configurations (configs/logger) which direct outputs to platforms like TensorBoard, WandB, MLflow, etc.
– Callbacks and Evaluators: The callbacks (topobench/callbacks) and evaluation modules (topobench/evaluator) monitor the training process, perform early stopping, model checkpointing, and log metrics.
• Experiment Reproducibility & Scripts:
– Scripts such as “env_setup.sh” and “scripts/reproduce.sh” handle environment setup and provide reproducible experiment pipelines.
• Testing & Documentation:
– Comprehensive unit tests (in test/) ensure integration of the various components.
– Detailed Sphinx documentation (docs/) and tutorials help users understand how to extend and use the framework.
Relationships and Interactions
• The CLI entry point parses command-line arguments (with overrides for model, dataset, and transforms) and passes them to Hydra.
• The Configuration Manager retrieves default and user-specified configurations for datasets, models, transforms, logging, etc.
• The Data Pipeline takes raw datasets and applies preprocessing and lifting transforms before feeding data into the model.
• The Model component (selected via configuration) processes the data and produces predictions.
• The Loss, Optimizer, and Trainer modules drive the training loop, while evaluators compute performance metrics.
• Logging and callbacks (e.g., model checkpoint, timer) operate throughout the training process for monitoring and debugging.
• The whole system is modular; each part is independently configurable and replaceable, which is reflected in the directory structure.
Architectural Patterns & Design Principles
• Modularity & Extensibility: A heavy emphasis on decoupling—separating data loading, model definition, training, evaluation, and transformation.
• Configuration-Driven: The use of Hydra ensures a clear separation between code and configuration, allowing experiments to be easily configured via YAML files.
• Pipeline/Flow Design: Data flows in a sequential pipeline—from input and preprocessing, to model training, to evaluation—while liftings and transforms are applied as intermediate processing layers.
• Testability: The extensive test suites (unit tests for each module in test/) indicate a design that encourages test-driven development and modular verification.
Technologies, Frameworks, and Libraries
• Python (3.10+ or 3.11) is the primary language.
• Hydra for configuration management.
• PyTorch (or another deep learning framework that underpins the neural network modules).
• Logging tools: TensorBoard, WandB, among others.
• Auxiliary tools for testing (pytest) and continuous integration (GitHub Actions with workflows for linting, testing, and documentation building).