# duyhominhnguyen/conan-fgw architecture

> Architecture diagram of the GitHub repository duyhominhnguyen/conan-fgw, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/duyhominhnguyen/conan-fgw
- Repository: https://github.com/duyhominhnguyen/conan-fgw

## Overview

##### Project Type and Purpose
- This is a research-focused machine-learning library implementing “Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks” (ConAN-FGW).
- It is not a web or microservice application but a data-driven ML pipeline built on PyTorch Lightning and PyTorch Geometric, orchestrated by Bash scripts and YAML configs.

##### High-Level Architectural Components
a. Configuration Layer
- YAML files under `conan_fgw/config/<model>/<task>/<dataset>/…`
- `environment.yml`, `pyproject.toml`, `setup.cfg`
b. Orchestration Layer (CLI & Scripts)
- `conan_fgw/script/run.sh` and variants
- `Makefile` (build/test targets)
c. Parser & Experiment Entry Point
- `conan_fgw/src/config_parser.py`
- `conan_fgw/src/train_val.py` (accepts CLI args & config path)
- `conan_fgw/src/experiments.py` (defines SOTAExperiment, SOTAExperimentBaryCenter)
d. Data Layer
- Raw data location: `/data` (user-provided)
- Data pipeline modules in `conan_fgw/src/data/`:
– `datasets.py` (dataset abstractions)
– `datamodules.py` (Lightning DataModule wrappers)
– `conformers/generators.py` & `features.py` (conformer generation & graph‐feature extraction)
– `splitters.py` (train/val/test splits)
e. Model Layer
- Core neural modules in `conan_fgw/src/model/`:
– Graph-embedding backbones:
· `schnet_no_sum.py` (SchNet extension)
· `visnet.py` (ViSNet extension)
– Aggregation & FGW algorithms:
· `model/fgw/` (Barycenter, Bregman, Sinkhorn, utils)
– Attention & baseline components: `attention_layer.py`, `baseline_models.py`
– Common utilities: `common.py`, `utils.py`, `features.py`
f. Training & Validation
- `conan_fgw/src/train_val.py` dispatches to PyTorch Lightning Trainer using Experiment classes
- Distributed DataParallel for stage “conan_fgw”
- Logging via local `outputs/` and optional Weights & Biases
g. Artifacts & Reports
- Model checkpoints: `models/`
- Logs (tensorboard/Lightning logs): `outputs/`
- Figures and reports: `figs/`, `reports/`
- Notebooks for demo: `notebooks/fgw.ipynb`

##### Interactions & Data Flow
1. User invokes `run.sh` → sets environment variables → calls `train_val.py`.
2. `train_val.py` loads YAML config via `config_parser.py`.
3. `config_parser` → instantiates a Lightning DataModule (`datamodules.py`) → raw `.pt` or dataset files under `/data`.
4. DataModule pipelines conformer generation → feature extraction → batching.
5. `train_val.py` also instantiates the Experiment class (SOTAExperiment or SOTAExperimentBaryCenter)
6. Experiment builds the model: selects backbone (SchNetNoSum or ViSNet), wraps FGW aggregator, attention, etc.
7. Lightning Trainer executes training loops: forward passes→loss computation→backprop.
8. Checkpoints written to `models/…`, logs to `outputs/…`, optional W&B calls.
9. Post-training artifacts produce figures under `reports/` and `figs/` for analysis.

##### Important Architectural Patterns
- Configuration-Driven: All hyperparams and experiment settings defined in YAML and loaded at runtime.
- Modular ML Pipeline: Clear separation between DataModule, Model, Training loop (Experiment + Trainer).
- Pluggable Backbones & Aggregators: The Experiment class swaps between SchNet/ViSNet and different FGW stages (pretrain vs. barycenter).
- PyTorch Lightning Pattern: Standard Trainer–LightningModule–DataModule triad.

##### Elements
- Clear Component Labels (e.g., “SchNetNoSum” vs. “FGW Barycenter Module”)
- Blue = Configuration & Orchestration
- Green = Data Layer
- Orange = Model Layer
- Purple = Training & Logging
- Gray = Artifacts & Reports

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Orchestration & Config
    subgraph "Orchestration & Config"
        direction TB
        YAML["YAML Configs"]:::config
        ENV["environment.yml"]:::config
        TOML["pyproject.toml"]:::config
        CFG["setup.cfg"]:::config

        RUN_ALL["run.sh"]:::orchestration
        RUN_PRE["run_conan_fgw_pre_only.sh"]:::orchestration
        RUN_BARY["run_conan_fgw_only.sh"]:::orchestration
        MAKE["Makefile"]:::orchestration
    end

    %% Layer 2: Data Ingestion & Pre-processing
    subgraph "Data Ingestion & Pre-processing"
        direction TB
        PARSER["config_parser.py"]:::parser
        DMODULE["datamodules.py"]:::data
        DATASETS["datasets.py"]:::data
        GENERATORS["generators.py"]:::data
        FEATURES_DP["features.py"]:::data
        SPLITTERS["splitters.py"]:::data
    end

    %% Layer 3: Model & Algorithm
    subgraph "Model & Algorithm"
        direction TB
        EXPERIMENTS["experiments.py"]:::parser

        subgraph "Graph Embedding Backbones"
            SCHNET["schnet_no_sum.py"]:::model
            VISNET["visnet.py"]:::model
            DIMENET["dimenet.py"]:::model
            ESAN["esan.py"]:::model
            GAT["gat.py"]:::model
            TGP_VISNET["torch_geometric_visnet.py"]:::model
        end

        subgraph "FGW Aggregation"
            BARY["barycenter.py"]:::model
            BREGMAN["bregman.py"]:::model
            SINKHORN["sinkhorn.py"]:::model
            FGW_UTILS["utils.py"]:::model
        end

        ATTENTION["attention_layer.py"]:::model
        BASELINE["baseline_models.py"]:::model
        COMMON["common.py"]:::model
        UTIL_MODEL["utils.py"]:::model
        FEATURES_M["features.py"]:::model
        SCHNET_BASED["schnet_based_models.py"]:::model
    end

    %% Layer 4: Training Engine & Logging
    subgraph "Training Engine & Logging"
        direction TB
        TRAIN_VAL["train_val.py"]:::train
        TRAINER["trainer.py"]:::train
        PL_LIGHTNING["PyTorch Lightning"]:::external
        PYG["PyTorch Geometric"]:::external
        WNB["Weights & Biases (opt)"]:::external
    end

    %% Layer 5: Artifacts & Reports
    subgraph "Artifacts & Reports"
        direction TB
        MODELS["models/"]:::artifacts
        OUTPUTS["outputs/"]:::artifacts
        FIGS["figs/"]:::artifacts
        REPORTS["reports/"]:::artifacts
        NOTEBOOK["fgw.ipynb"]:::artifacts
    end

    %% Connections
    RUN_ALL -->|invokes| TRAIN_VAL
    RUN_PRE -->|invokes pretrain stage| TRAIN_VAL
    RUN_BARY -->|invokes barycenter stage| TRAIN_VAL
    YAML -->|loaded by| PARSER
    ENV -->|sets env| RUN_ALL
    TOML -->|defines project| RUN_ALL
    CFG -->|defines project| RUN_ALL

    TRAIN_VAL -->|parses config| PARSER
    PARSER -->|instantiates| DMODULE
    PARSER -->|passes to| EXPERIMENTS

    DMODULE -->|loads raw data| DATASETS
    DMODULE -->|generates conformers| GENERATORS
    DMODULE -->|extracts features| FEATURES_DP
    DMODULE -->|splits data| SPLITTERS

    EXPERIMENTS -->|selects backbone| SCHNET
    EXPERIMENTS --> VISNET
    EXPERIMENTS -->|wraps| BARY
    EXPERIMENTS --> BREGMAN
    EXPERIMENTS --> SINKHORN
    EXPERIMENTS --> FGW_UTILS
    EXPERIMENTS --> ATTENTION
    EXPERIMENTS --> BASELINE

    TRAIN_VAL -->|dispatches to| TRAINER
    TRAINER --> PL_LIGHTNING
    TRAINER --> PYG
    TRAINER --> WNB

    TRAINER -->|writes checkpoints| MODELS
    TRAINER -->|writes logs| OUTPUTS
    TRAINER -->|writes figures| FIGS
    TRAINER -->|writes reports| REPORTS
    NOTEBOOK -->|reads artifacts| MODELS

    %% Click Events
    click YAML "https://github.com/duyhominhnguyen/conan-fgw/tree/main/conan_fgw/config/"
    click ENV "https://github.com/duyhominhnguyen/conan-fgw/blob/main/environment.yml"
    click TOML "https://github.com/duyhominhnguyen/conan-fgw/blob/main/pyproject.toml"
    click CFG "https://github.com/duyhominhnguyen/conan-fgw/blob/main/setup.cfg"
    click RUN_ALL "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/script/run.sh"
    click RUN_PRE "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/script/run_conan_fgw_pre_only.sh"
    click RUN_BARY "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/script/run_conan_fgw_only.sh"
    click MAKE "https://github.com/duyhominhnguyen/conan-fgw/tree/main/Makefile"
    click PARSER "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/config_parser.py"
    click TRAIN_VAL "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/train_val.py"
    click EXPERIMENTS "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/experiments.py"
    click DATASETS "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/data/datasets.py"
    click DMODULE "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/data/datamodules.py"
    click GENERATORS "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/data/conformers/generators.py"
    click FEATURES_DP "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/data/conformers/features.py"
    click SPLITTERS "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/data/splitters.py"
    click SCHNET "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/schnet_no_sum.py"
    click VISNET "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/visnet.py"
    click DIMENET "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/dimenet.py"
    click ESAN "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/esan.py"
    click GAT "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/gat.py"
    click TGP_VISNET "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/graph_embeddings/torch_geometric_visnet.py"
    click BARY "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/fgw/barycenter.py"
    click BREGMAN "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/fgw/bregman.py"
    click SINKHORN "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/fgw/sinkhorn.py"
    click FGW_UTILS "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/fgw/utils.py"
    click ATTENTION "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/attention_layer.py"
    click BASELINE "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/baseline_models.py"
    click COMMON "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/common.py"
    click UTIL_MODEL "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/utils.py"
    click FEATURES_M "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/features.py"
    click SCHNET_BASED "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/model/schnet_based_models.py"
    click TRAINER "https://github.com/duyhominhnguyen/conan-fgw/blob/main/conan_fgw/src/trainer.py"
    click MODELS "https://github.com/duyhominhnguyen/conan-fgw/tree/main/models/"
    click OUTPUTS "https://github.com/duyhominhnguyen/conan-fgw/tree/main/outputs/"
    click FIGS "https://github.com/duyhominhnguyen/conan-fgw/tree/main/figs/"
    click REPORTS "https://github.com/duyhominhnguyen/conan-fgw/tree/main/reports/"
    click NOTEBOOK "https://github.com/duyhominhnguyen/conan-fgw/blob/main/notebooks/fgw.ipynb"

    %% Styles
    classDef config fill:#BBDEFB,stroke:#1E88E5,color:#000;
    classDef orchestration fill:#90CAF9,stroke:#1E88E5,color:#000;
    classDef parser fill:#E3F2FD,stroke:#1976D2,color:#000;
    classDef data fill:#C8E6C9,stroke:#388E3C,color:#000;
    classDef model fill:#FFE0B2,stroke:#F57C00,color:#000;
    classDef train fill:#E1BEE7,stroke:#8E24AA,color:#000;
    classDef artifacts fill:#EEEEEE,stroke:#757575,color:#000;
    classDef external fill:#CFD8DC,stroke:#546E7A,color:#000;
```
