# tsa87/cgflow architecture

> Architecture diagram of the GitHub repository tsa87/cgflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tsa87/cgflow
- Repository: https://github.com/tsa87/cgflow

## Overview

##### Project Type and Purpose
- CGFlow is a research-oriented Python library and toolkit for compositional generative flows, combining 3D molecule conformation generation (pose prediction) with synthesis pathway design.
- It integrates and extends three core domains:
- cgflow: co-design of molecule pose + synthesis path
- gflownet: general GFlowNet algorithms
- rxnflow: retrosynthesis pathway sampling via GFlowNets
- synthflow: high-level API and pocket-conditional/specific training

##### High-Level Architecture Overview
- This is not a monolithic web or full-stack app but a modular ML framework with:
A. Core packages under src/: cgflow, gflownet, rxnflow, synthflow
B. Data preparation & experiment orchestration under experiments/ and scripts/
C. Configuration & metadata: pyproject.toml, yaml sweeps, README, LICENSE
D. Assets & figures for documentation and demo

1) Data Sources
- experiments/data/: raw datasets (PLINDER, LIT-PCBA, CrossDocked, Enamine catalogs)
- assets/: reference images
2) Data Processing & Loading
- experiments/data/scripts/ & scripts/: Python + shell scripts that download, clean, convert SDF → SMILES, build LMDB, create environments
- src/*/data modules (e.g. cgflow/data/datamodules.py, gflownet/data/data_source.py, rxnflow/utils/*)
3) Core Libraries
- cgflow/ buildutil, data, models, util
- gflownet/ algo, data, envs, models, trainer, tasks, utils
- rxnflow/ base, algo, communication, envs, models, policy, tasks, utils
- synthflow/ api, base, pocket_conditional, pocket_specific, tasks, utils
4) Training & Inference Engines
- Trainer classes (cgflow/buildutil/build_trainer.py, gflownet/trainer.py, rxnflow/base/trainer.py, synthflow/base/trainer.py)
- Online trainers & CLI entrypoints (src/gflownet/online_trainer.py, src/rxnflow/base/gflownet/online_trainer.py, synthflow/api/inference.py)
5) Experiment Orchestration & Hyperparameter Sweeps
- experiments/scripts/exp*.py and experiments/sweep/*.yaml drive training, evaluation, wandb sweeps
- Top-level shell wrappers in scripts/*.sh
6) Outputs & Artifacts
- weights/: model checkpoints
- logs/: wandb & local logs
- results: images, tables, saved LMDB datasets

- Data Sources → Data Processing scripts → DataModule (in core libs)
- DataModule → Trainer → Model
- Model ↔ GFlowNet Algorithms (flow_matching, trajectory_balance, A2C, etc.)
- Trainer → Logs & Checkpoints → Experiments & Evaluation scripts
- Evaluation scripts → Metrics & Visualizations (assets/)
- synthflow/api → inference endpoint (e.g. Streamlit demo)

##### Architectural Patterns & Principles
- Layered modularity: separation of data, model, training, evaluation
- Plugin-style algorithms: gflownet/algo and rxnflow/algo share a common config interface
- Config-driven: YAML sweep files, python config modules (config.py), pyproject.toml
- Extensibility: each flow (cgflow, gflownet, rxnflow, synthflow) can be used standalone or composed

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources
    subgraph "Data Layer"
        direction TB
        DS["Data Sources"]:::data
        Assets["Reference Assets"]:::data
    end

    %% Data Processing
    subgraph "Processing Layer"
        direction TB
        ScriptsData["Data Processing Scripts"]:::process
        ScriptsShell["Shell Wrappers"]:::process
    end

    %% Core Libraries
    subgraph "Core Libraries"
        direction TB
        subgraph "cgflow" 
            direction TB
            CG_DM["DataModules"]:::core
            CG_Core["Core Library"]:::core
        end
        subgraph "gflownet"
            direction TB
            GF_DS["DataSource"]:::core
            GF_Core["Core Library"]:::core
        end
        subgraph "rxnflow"
            direction TB
            RX_Utils["Utilities"]:::core
            RX_Core["Core Library"]:::core
        end
        subgraph "synthflow"
            direction TB
            SF_Core["Core Library"]:::core
        end
    end

    %% Training & Inference Engines
    subgraph "Training & Inference"
        direction TB
        CG_Trainer["cgflow Trainer"]:::train
        GF_Trainer["gflownet Trainer"]:::train
        RX_Trainer["rxnflow Trainer"]:::train
        SF_Trainer["synthflow Trainer"]:::train
        GF_Online["gflownet Online CLI"]:::train
        RX_Online["rxnflow Online CLI"]:::train
        SF_Infer["synthflow Inference API"]:::infer
    end

    %% Orchestration & Sweeps
    subgraph "Experiment Orchestration"
        direction TB
        Exp_Scripts["Experiment Scripts"]:::orchestr
        Sweeps["Hyperparameter Sweeps"]:::orchestr
    end

    %% Outputs & Artifacts
    Outputs["Outputs & Artifacts"]:::output

    %% Inference Demo
    Demo["3DSynthFlow Demo UI"]:::infer

    %% Connections
    DS -->|"download/clean"| ScriptsData
    Assets -->|"reference"| ScriptsData
    ScriptsData -->|"build_datamodule"| CG_DM
    ScriptsData -->|"build_datamodule"| GF_DS
    ScriptsData -->|"build_datamodule"| RX_Utils

    CG_DM -->|"train_flow"| CG_Trainer
    GF_DS -->|"train_flow"| GF_Trainer
    RX_Utils -->|"train_flow"| RX_Trainer

    CG_Trainer -->|"checkpoint/log"| Outputs
    GF_Trainer -->|"checkpoint/log"| Outputs
    RX_Trainer -->|"checkpoint/log"| Outputs
    SF_Trainer -->|"checkpoint/log"| Outputs

    GF_Online -->|"train_flow"| GF_Trainer
    RX_Online -->|"train_flow"| RX_Trainer
    SF_Infer -->|"infer_pose"| Demo

    Exp_Scripts -->|"orchestrate"| CG_Trainer
    Exp_Scripts -->|"orchestrate"| GF_Trainer
    Exp_Scripts -->|"orchestrate"| RX_Trainer
    Exp_Scripts -->|"orchestrate"| SF_Trainer
    Sweeps -->|"config"| Exp_Scripts
    ScriptsShell -->|"run"| Exp_Scripts

    Outputs -->|"evaluate_pose"| Exp_Scripts
    Demo -->|"visualize"| Outputs

    %% Click Events
    click DS "https://github.com/tsa87/cgflow/tree/main/experiments/data/"
    click Assets "https://github.com/tsa87/cgflow/tree/main/assets/"
    click ScriptsData "https://github.com/tsa87/cgflow/tree/main/experiments/data/scripts/"
    click ScriptsShell "https://github.com/tsa87/cgflow/tree/main/scripts/"
    click CG_DM "https://github.com/tsa87/cgflow/blob/main/src/cgflow/data/datamodules.py"
    click GF_DS "https://github.com/tsa87/cgflow/blob/main/src/gflownet/data/data_source.py"
    click RX_Utils "https://github.com/tsa87/cgflow/tree/main/src/rxnflow/utils/"
    click CG_Core "https://github.com/tsa87/cgflow/tree/main/src/cgflow/buildutil/"
    click GF_Core "https://github.com/tsa87/cgflow/tree/main/src/gflownet/algo/"
    click RX_Core "https://github.com/tsa87/cgflow/tree/main/src/rxnflow/algo/"
    click SF_Core "https://github.com/tsa87/cgflow/tree/main/src/synthflow/api/"
    click CG_Trainer "https://github.com/tsa87/cgflow/blob/main/src/cgflow/buildutil/build_trainer.py"
    click GF_Trainer "https://github.com/tsa87/cgflow/blob/main/src/gflownet/trainer.py"
    click RX_Trainer "https://github.com/tsa87/cgflow/blob/main/src/rxnflow/base/trainer.py"
    click SF_Trainer "https://github.com/tsa87/cgflow/blob/main/src/synthflow/base/trainer.py"
    click GF_Online "https://github.com/tsa87/cgflow/blob/main/src/gflownet/online_trainer.py"
    click RX_Online "https://github.com/tsa87/cgflow/blob/main/src/rxnflow/base/gflownet/online_trainer.py"
    click SF_Infer "https://github.com/tsa87/cgflow/blob/main/src/synthflow/api/inference.py"
    click Exp_Scripts "https://github.com/tsa87/cgflow/tree/main/experiments/scripts/"
    click Sweeps "https://github.com/tsa87/cgflow/tree/main/experiments/sweep/"

    %% Styles
    classDef data fill:#D6E9F8,stroke:#2B4F81,stroke-width:1px
    classDef process fill:#E2F0D9,stroke:#3A7D44,stroke-width:1px
    classDef core fill:#FDE9D9,stroke:#B35A1A,stroke-width:1px
    classDef train fill:#FBE7A1,stroke:#8A6804,stroke-width:1px
    classDef orchestr fill:#E8DAEF,stroke:#6E2C91,stroke-width:1px
    classDef output fill:#DFE7E8,stroke:#4A646C,stroke-width:1px
    classDef infer fill:#D5F5E3,stroke:#239B56,stroke-width:1px
```
