How cfifty/ignite works
Main Components
• Data Acquisition & Download: • Represent the “misc/download_data” directory as the initial component where the large 77 GB IGNITE dataset is downloaded (using scripts like download_ignite.sh). Also, note the subfolder “misc/fsmol_unique_ligands” (which includes zipped data and JSON files) that supports additional dataset functionality.
• Data Handling & Processing: • The “src/data” directory provides modules to read binding affinity tasks (binding_affinity_task.py, binding_data_multitask.py), manage dataset batching (fsmol_batcher.py), and sample tasks (fsmol_task_sampler.py). Plus, the “src/datasets” folder includes a JSON file (ignite_data.json) that likely lists the training tasks or meta-information about the dataset.
• Preprocessing & Featurisation: • The “src/preprocessing” directory (and its nested “featurisers” folder) handles molecular featurisation, using utilities (like rdkit_helpers.py, featurised_data.py, molgraph_utils.py) and specific scripts (such as glide_to_binding_affinities.py, process_binding_scores.py) to prepare data for the model.
• Models & Modules: • Under “src/models”, the repository defines model architectures (for example, abstract_torch_fsmol_model.py and gnn_multitask.py) – these encapsulate the core neural network architecture. Meanwhile, “src/modules” contains the building blocks, such as the graph neural network implementation (gnn.py), graph feature extractor (graph_feature_extractor.py), graph readout (graph_readout.py), and standard MLP (mlp.py).
• Training Pipeline: • The key entry point “src/multitask_train.py” orchestrates training by tying together the data handling, preprocessing, and model components. This script coordinates the overall workflow.
• Prototype Simulation Dataset: • The “src/prototype_simulation_dataset” directory (with its “train” and “valid” subdirectories) offers additional data resources possibly intended for simulation or benchmarking.
• Utilities: • “src/utils” provides common helper functions (for logging, CLI utilities, metrics, torch-related utilities) which support and tie together various parts of the system.
Relationships and Interactions
• Data Flow:
• Start from data acquisition (download_data scripts) → stored datasets.
• Data ingestion/loaders in “src/data” read raw data (and JSON files in “src/datasets”) as input.
• Preprocessing (featurisers under “src/preprocessing”) then transforms raw molecular data into featurised representations.
• These processed representations are fed into the models defined under “src/models” with supporting computation provided by modules in “src/modules”.
• Training:
• The “multitask_train.py” script ties these layers together, receiving the processed data, initializing model architectures, and managing training iterations (for instance, through command-line parameters).
• External Dependencies:
• Although not directly visible, the system relies on machine learning frameworks (Torch) and cheminformatics libraries (like RDKit) for featurisation.
Architectural Patterns/Design Principles
• Modularization: The project splits responsibilities clearly across data handling, preprocessing, model definition, and training orchestration.
• Reusability & Extensibility: Components such as abstract models and modular neural network layers (in “modules/”) allow for reuse and easy modification.
• Pipeline Flow: A sequential data pipeline structure where data is downloaded, preprocessed, fed into a model, and then trained iteratively.
• Separation of Concerns: By isolating featurisation in “preprocessing” and model logic in separate “models”/“modules” directories, the design adheres to maintainable software principles.
• Simulation dataset side-by-side with the main dataset components to indicate alternative or additional data flows used for experiments.
• What each directory encapsulates.
• The role of the external dependencies (datasets, libraries like PyTorch or RDKit).