# gfnorg/chunk-gfn architecture

> Architecture diagram of the GitHub repository gfnorg/chunk-gfn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gfnorg/chunk-gfn
- Repository: https://github.com/gfnorg/chunk-gfn

## Overview

##### Project Type and Purpose
- This is a research-oriented Python package for reinforcement learning (RL) and GFlowNet experiments.
- It uses PyTorch Lightning for training loops, Hydra for configuration, SLURM (via sbatch scripts) for cluster scheduling, and optionally WandB or local file logging.

##### Main Components
A. Entry Point & Config
– main.py + run_all.sh + sbatch_scripts/
– Hydra config tree (configs/)
B. Experiment Orchestrator
– PyTorch Lightning Trainer
– Hydra instantiation (chunkgfn/utils/instantiators.py)
C. Core Modules (chunkgfn/)
1. environment/ : defines MDP or generative environment
2. models/      : neural nets (MLP, RNN, GNN, Transformer, encoders)
3. algo/        : RL & GFlowNet algorithms (A2C, SAC, OptionCritic, TB-GFN, etc.)
4. replay_buffer/: stores transitions for off-policy updates
5. schedulers/  : LR and temperature schedules
6. rewards/     : loss or reward functions (MSE, etc.)
7. utils/       : logging, caching, Hydra + PyLogger helpers
D. Data & Artifacts
– data/ : raw datasets, download scripts
– logs/ : checkpoint and TensorBoard/WandB outputs
E. External Services
– SLURM (job submission via sbatch scripts)
– WandB or local file logger
– Filesystem (datasets, logs, mode pickles)

##### Data Flows and Interactions
1. User/CI invokes `python main.py` or calls an sbatch wrapper
2. Hydra reads YAML configs → builds a config object
3. Instantiators create Environment, Model, Algorithm, ReplayBuffer, Scheduler
4. Lightning Trainer kicks off training loop:
a. At each step, Trainer → `algorithm.sample_action(state)`
b. Environment returns next_state, reward → stored in ReplayBuffer
c. Algorithm updates Model parameters (backprop via PyTorch Lightning)
5. Trainer logs metrics & checkpoints to logs/ and WandB
6. At the end: results and artifacts in logs/, optional downstream analysis via notebooks/

##### Architectural Patterns & Design Principles
- Modular plugin architecture (Hydra + dynamic instantiation)
- Separation of concerns:
– Environments (MDP definitions) vs. Algorithms (update rules) vs. Models (NN architectures)
- Experiment reproducibility: config files per experiment, SLURM scripts
- Extensibility: add new envs, algos, reward modules by dropping files in chunkgfn/\*

## Diagram (Mermaid)

```mermaid
graph TD
    %% Swimlane: User & CLI Layer
    subgraph "User & CLI Layer"
        direction TB
        CLI["User/CI CLI"]:::external
    end

    %% Configuration & Orchestration Layer
    subgraph "Configuration & Orchestration Layer"
        direction TB
        Main["main.py"]:::infra
        RunAll["run_all.sh"]:::infra
        SLURMScripts["sbatch_scripts/"]:::infra
        Configs["configs/"]:::infra
        Instantiator["Hydra Instantiator\n(chunkgfn/utils/instantiators.py)"]:::infra
        Trainer["PyTorch Lightning Trainer"]:::infra
    end

    %% Core Computation Layer
    subgraph "Core Computation Layer"
        direction TB
        Env["Environment\n(chunkgfn/environment/)"]:::compute
        Model["Model\n(chunkgfn/models/)"]:::compute
        Algo["Algorithm\n(chunkgfn/algo/)"]:::compute
        Replay["ReplayBuffer\n(chunkgfn/replay_buffer/)"]:::compute
        Sched["Scheduler\n(chunkgfn/schedulers/)"]:::compute
        Rewards["Rewards/Loss\n(chunkgfn/rewards/)"]:::compute
        Utils["Utilities\n(chunkgfn/utils/)"]:::compute
    end

    %% Persistence & Logging Layer
    subgraph "Persistence & Logging Layer"
        direction TB
        Data["data/"]:::store
        Logs["logs/"]:::store
    end

    %% External Services Layer
    subgraph "External Services Layer"
        direction TB
        SLURM["SLURM Scheduler"]:::external
        WandB["WandB"]:::external
        FS["Filesystem"]:::external
    end

    %% Connections
    CLI -->|"invoke"| Main
    CLI -->|"invoke"| RunAll
    RunAll -->|"wraps call"| Main
    CLI -->|"via sbatch"| SLURMScripts
    SLURMScripts -->|"submits job"| SLURM

    Main -->|"reads configs"| Configs
    Configs -->|"provide YAML"| Instantiator
    SLURMScripts -->|"env vars"| Instantiator

    Instantiator -->|"instantiate"| Env
    Instantiator -->|"instantiate"| Model
    Instantiator -->|"instantiate"| Algo
    Instantiator -->|"instantiate"| Replay
    Instantiator -->|"instantiate"| Sched
    Instantiator -->|"instantiate"| Rewards
    Instantiator -->|"helpers"| Utils

    Main -->|"initializes"| Trainer
    Trainer -->|"sample_action()"| Algo
    Algo -->|"forward()"| Model
    Algo -->|"push_transition()"| Replay
    Replay -->|"batch()"| Trainer
    Trainer -->|"step()"| Algo
    Trainer -->|"step()"| Model
    Trainer -->|"step()"| Replay
    Trainer -->|"update_lr()"| Sched
    Trainer -->|"compute_loss()"| Rewards
    Trainer -->|"use utils"| Utils

    Trainer -->|"log_metrics()"| Logs
    Trainer -->|"log_metrics()"| WandB
    Data -->|"read/write"| FS
    Logs -->|"store"| FS

    %% Click Events
    click Main "https://github.com/gfnorg/chunk-gfn/blob/main/project root/main.py"
    click RunAll "https://github.com/gfnorg/chunk-gfn/blob/main/project root/run_all.sh"
    click SLURMScripts "https://github.com/gfnorg/chunk-gfn/tree/main/project root/sbatch_scripts/"
    click Configs "https://github.com/gfnorg/chunk-gfn/tree/main/project root/configs/"
    click Instantiator "https://github.com/gfnorg/chunk-gfn/blob/main/project root/chunkgfn/utils/instantiators.py"
    click Env "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/environment/"
    click Model "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/models/"
    click Algo "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/algo/"
    click Replay "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/replay_buffer/"
    click Sched "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/schedulers/"
    click Rewards "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/rewards/"
    click Utils "https://github.com/gfnorg/chunk-gfn/tree/main/project root/chunkgfn/utils/"
    click Data "https://github.com/gfnorg/chunk-gfn/tree/main/project root/data/"
    click Logs "https://github.com/gfnorg/chunk-gfn/tree/main/project root/logs/"

    %% Styles
    classDef infra fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef compute fill:#90EE90,stroke:#333,stroke-width:1px
    classDef store fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#FF6347,stroke:#333,stroke-width:1px
```
