How lus105/deepvisionxplain works
Project Type & Purpose
• This is a deep learning experimentation framework focused on comparing the explainability of CNNs (using Class Activation Maps) with Vision Transformers (using attention rollout).
• It leverages PyTorch (and PyTorch Lightning), Hydra (for configuration management), and various MLOps/logging tools (e.g., WandB, MLflow) to facilitate training, evaluation, and experimentation.
• The README indicates that this project builds upon the DeepTrainer framework, modifying ViT and CNN architectures for explainability studies.
Analyzing the File Structure
• The top-level directories and files provide clear insights:
• The “configs” directory contains extensive configuration files:
• Callbacks, data configs, experiment configs, debug settings, hyperparameter search, logger setups, model specifications, and trainer configurations.
• This signals a strong emphasis on flexibility and reproducibility using Hydra-based configuration.
• The “src” directory is where the main code lives:
• Under “src/data”, you have modules for data loading (datamodules, datasets, preprocessing pipelines).
• Under “src/models”, you have model definition and components for both CNN and ViT architectures, including explainability-specific modifications.
• “src/train.py” and “src/eval.py” are core to executing training and evaluation routines.
• Additional directories like “notebooks” (interactive experiments and demonstration notebooks), “scripts” (for evaluations and scheduling), “tests” (test cases for components) and “docs” (documentation and assets) complete the ecosystem.
• Docker and environment configuration files indicate containerization and dependency management.
Key Components to Illustrate
a. Configuration and Experiment Management:
• Config system (Hydra-based) with its hierarchy: callbacks, trainer configs (cpu, gpu, ddp, etc.), model definitions, and hyperparameter search.
b. Data Pipeline:
• Data ingestion & preprocessing workflows are encapsulated in “src/data”.
• Split out “Data Module” which includes custom dataset components, label parsing, and preprocessing strategies.
c. Model Modules:
• Divide into two submodules: one for CNN (with its modifications for CAM explainability) and one for ViT (with attention rollout).
• Inside each, include the model definition and any explainability components.
• That these modules are accessed during both training and evaluation.
d. Training / Evaluation Engine:
• Primary executables: “src/train.py” for training routines and “src/eval.py” for evaluation/inference.
• Integration with PyTorch Lightning if applicable.
• This engine to both the configuration system and the model modules.
e. Logging & MLOps:
• External logging/instrumentation tools (WandB, Comet, Tensorboard, etc.) which are configured under “configs/logger”.
• Elements as external services with two-way interactions (upload training logs, experiment metrics, model checkpoints).
f. External Tools & Deployment:
• Docker, Conda environments, and CI/CD pipeline (GitHub Actions under .github/workflows) as supporting infrastructure.
• “Notebooks” as separate interactive components that demonstrate how to download, infer, and convert models.
g. Testing:
Architectural Patterns & Data Flow
• Separation of concerns:
• The configuration layer (Hydra) decouples the experiment specifications from code logic.
• Data processing and model components are modular so that various pipelines can plug into different experiments.
• Training and evaluation modules are designed to work with multiple settings (trainer types, hyperparameter search, etc.).
• Configuration → Data Pipeline & Model Modules
• Data Pipeline & Model Modules → Training Engine
• Training Engine <→ Logging & Callbacks (feedback loop)
• Training Engine → Evaluation Module → External Outputs (e.g., notebooks, logs)
Final Advice
• Directional flow: from configuration to data to models to training/evaluation, and finally to external logging and inference.
• Don’t hesitate to break down modules (such as the Data Module and Model Module) further into sub-components if needed.