# cfifty/ignite architecture

> Architecture diagram of the GitHub repository cfifty/ignite, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cfifty/ignite
- Repository: https://github.com/cfifty/ignite

## Overview

##### Main Components
- Data Acquisition & Download: • Represent the “misc/download_data” directory as the initial component where the large 77 GB IGNITE dataset is downloaded (using scripts like download_ignite.sh). Also, note the subfolder “misc/fsmol_unique_ligands” (which includes zipped data and JSON files) that supports additional dataset functionality.
- Data Handling & Processing: • The “src/data” directory provides modules to read binding affinity tasks (binding_affinity_task.py, binding_data_multitask.py), manage dataset batching (fsmol_batcher.py), and sample tasks (fsmol_task_sampler.py). Plus, the “src/datasets” folder includes a JSON file (ignite_data.json) that likely lists the training tasks or meta-information about the dataset.
- Preprocessing & Featurisation: • The “src/preprocessing” directory (and its nested “featurisers” folder) handles molecular featurisation, using utilities (like rdkit_helpers.py, featurised_data.py, molgraph_utils.py) and specific scripts (such as glide_to_binding_affinities.py, process_binding_scores.py) to prepare data for the model.
- Models & Modules: • Under “src/models”, the repository defines model architectures (for example, abstract_torch_fsmol_model.py and gnn_multitask.py) – these encapsulate the core neural network architecture. Meanwhile, “src/modules” contains the building blocks, such as the graph neural network implementation (gnn.py), graph feature extractor (graph_feature_extractor.py), graph readout (graph_readout.py), and standard MLP (mlp.py).
- Training Pipeline: • The key entry point “src/multitask_train.py” orchestrates training by tying together the data handling, preprocessing, and model components. This script coordinates the overall workflow.
- Prototype Simulation Dataset: • The “src/prototype_simulation_dataset” directory (with its “train” and “valid” subdirectories) offers additional data resources possibly intended for simulation or benchmarking.
- Utilities: • “src/utils” provides common helper functions (for logging, CLI utilities, metrics, torch-related utilities) which support and tie together various parts of the system.

##### Relationships and Interactions
- Data Flow:
- Start from data acquisition (download_data scripts) → stored datasets.
- Data ingestion/loaders in “src/data” read raw data (and JSON files in “src/datasets”) as input.
- Preprocessing (featurisers under “src/preprocessing”) then transforms raw molecular data into featurised representations.
- These processed representations are fed into the models defined under “src/models” with supporting computation provided by modules in “src/modules”.
- Training:
- The “multitask_train.py” script ties these layers together, receiving the processed data, initializing model architectures, and managing training iterations (for instance, through command-line parameters).
- External Dependencies:
- Although not directly visible, the system relies on machine learning frameworks (Torch) and cheminformatics libraries (like RDKit) for featurisation.

##### Architectural Patterns/Design Principles
- Modularization: The project splits responsibilities clearly across data handling, preprocessing, model definition, and training orchestration.
- Reusability & Extensibility: Components such as abstract models and modular neural network layers (in “modules/”) allow for reuse and easy modification.
- Pipeline Flow: A sequential data pipeline structure where data is downloaded, preprocessed, fed into a model, and then trained iteratively.
- Separation of Concerns: By isolating featurisation in “preprocessing” and model logic in separate “models”/“modules” directories, the design adheres to maintainable software principles.

- Simulation dataset side-by-side with the main dataset components to indicate alternative or additional data flows used for experiments.
- What each directory encapsulates.
- The role of the external dependencies (datasets, libraries like PyTorch or RDKit).

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Acquisition & Download"
        A1("Download Data Scripts"):::data
        A2("Unique Ligands Data"):::data
    end

    subgraph "Data Handling & Processing"
        B1("Data Handlers (src/data)"):::data
        B2("Dataset Metadata (src/datasets/ignite_data.json)"):::data
    end

    subgraph "Preprocessing & Featurisation"
        C1("Featurisation & Preprocessing (src/preprocessing)"):::preprocessing
        C2("Featurisers (src/preprocessing/featurisers)"):::preprocessing
        C3("GLIDE Binding Affinity (glide_to_binding_affinities.py)"):::preprocessing
        C4("Process Binding Scores (process_binding_scores.py)"):::preprocessing
    end

    subgraph "Models & Modules"
        D1("Models (src/models)"):::models
        D2("Modules (src/modules)"):::models
    end

    subgraph "Training Pipeline"
        E1("Training Pipeline (src/multitask_train.py)"):::training
    end

    subgraph "Prototype Simulation"
        F1("Prototype Simulation Dataset (src/prototype_simulation_dataset)"):::protosim
    end

    subgraph "Utilities"
        G1("Utilities (src/utils)"):::utils
    end

    %% Data Flow Connections
    A1 -->|"feeds"| B1
    A2 -->|"feeds"| B1

    B1 -->|"ingests"| C1
    B2 -->|"supplies"| C1

    C1 -->|"applies"| C2
    C1 -->|"applies"| C3
    C1 -->|"applies"| C4

    C1 -->|"transforms"| D1
    C1 -->|"transforms"| D2

    D1 -->|"trains via"| E1
    D2 -->|"trains via"| E1

    F1 -->|"alternative data"| E1

    %% Utility Support (dashed connections)
    G1 -.->|"supports"| B1
    G1 -.->|"supports"| C1
    G1 -.->|"supports"| D1
    G1 -.->|"supports"| E1

    %% Click Events
    click A1 "https://github.com/cfifty/ignite/tree/main/misc/download_data"
    click A2 "https://github.com/cfifty/ignite/tree/main/misc/fsmol_unique_ligands"
    click B1 "https://github.com/cfifty/ignite/tree/main/src/data"
    click B2 "https://github.com/cfifty/ignite/blob/main/src/datasets/ignite_data.json"
    click C1 "https://github.com/cfifty/ignite/tree/main/src/preprocessing"
    click C2 "https://github.com/cfifty/ignite/tree/main/src/preprocessing/featurisers"
    click C3 "https://github.com/cfifty/ignite/blob/main/src/preprocessing/glide_to_binding_affinities.py"
    click C4 "https://github.com/cfifty/ignite/blob/main/src/preprocessing/process_binding_scores.py"
    click D1 "https://github.com/cfifty/ignite/tree/main/src/models"
    click D2 "https://github.com/cfifty/ignite/tree/main/src/modules"
    click E1 "https://github.com/cfifty/ignite/blob/main/src/multitask_train.py"
    click F1 "https://github.com/cfifty/ignite/tree/main/src/prototype_simulation_dataset"
    click G1 "https://github.com/cfifty/ignite/tree/main/src/utils"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef preprocessing fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef models fill:#FFA500,stroke:#000,stroke-width:2px;
    classDef training fill:#FF6961,stroke:#000,stroke-width:2px;
    classDef protosim fill:#B0E0E6,stroke:#000,stroke-width:2px;
    classDef utils fill:#D3D3D3,stroke:#000,stroke-width:2px;
```
