# 9rum/flatflow architecture

> Architecture diagram of the GitHub repository 9rum/flatflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/9rum/flatflow
- Repository: https://github.com/9rum/flatflow

## Overview

##### Project type and purpose
- From the file tree we see:
- “flatflow” Python/C++ package (with sub-packages: ops, rpc, scheduler, sys, torch, nemo, megatron)
- Build files: CMakeLists.txt, setup.py, pyproject.toml; tests in C++/Python
- Examples and experiments for NLP language-modeling (Megatron GPT, NeMo, Dolly, Alpaca)
- Purpose: A hybrid C++/Python framework (“Flatflow”) that provides a core execution engine (operators, scheduler, RPC) and Python bindings (via pybind.cc), plus prebuilt collections for NVIDIA NeMo/Megatron NLP workflows. It orchestrates distributed training pipelines with custom operators and scheduling.

##### File structure
- Top-level:
- Build/config: CMakeLists.txt, Makefile, Dockerfile, setup.py/pyproject.toml, requirements.txt
- Source: flatflow/ (with subfolders: ops, rpc, scheduler, sys, torch, nemo, megatron)
- Examples: examples/nemo/nlp/language_modeling/tuning
- Experiments: shell scripts + Dockerfile for multi-node training, data prep, eval
- Tests: C++ tests under tests/ops, tests/scheduler
- Third-party: abseil, flatbuffers, googletest
- Architectural hints:
- C++ core modules providing operators (flatflow/ops), scheduling logic (flatflow/scheduler), RPC control-plane (flatflow/rpc), system hooks (flatflow/sys)
- Python bindings and utilities (flatflow/torch, nemo) layer on top
- NeMo/Megatron collections pack common NLP datasets, models, modules
- Integration with PyTorch via flatflow/torch/utils/data and profiler
- Cross-language boundary handled by pybind.cc

##### README (empty here)
- Rely on code structure and naming to infer component responsibilities

##### Detail
- Break the core into its subcomponents (ops vs scheduler vs rpc vs sys)
- Python binding boundary (pybind.cc)
- Protocols/formats (FlatBuffers for operators, gRPC or custom for RPC)
- Build tools (CMake, pip) only if relevant to the audience

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Development & Build Layer"
        CMake["CMakeLists.txt"]:::external
        SetupPy["setup.py"]:::external
        PyProject["pyproject.toml"]:::external
        Requirements["requirements.txt"]:::external
    end

    subgraph "Core Engine (C++ Shared Library)"
        subgraph "Operators"
            OpsSchemas["FlatBuffers Schemas"]:::cpp
            OpsCode["Operators Code"]:::cpp
        end
        subgraph "Scheduler"
            SchedulerAPI["scheduler.h"]:::cpp
            Partition["partition.h"]:::cpp
            Scatter["scatter.h"]:::cpp
        end
        subgraph "RPC Control-Plane"
            RPCSchemas["controlplane.fbs"]:::cpp
            RPCHeader["controlplane.h"]:::cpp
            RPCEpmty["empty.fbs"]:::cpp
            RPCPy["controlplane.py"]:::python
        end
        subgraph "System Hooks"
            SysPy["sysmodule.py"]:::python
        end
    end

    subgraph "Python Binding Layer"
        PyBind["pybind.cc"]:::cpp
    end

    subgraph "PyTorch Integration"
        DataLoader["dataloader.py"]:::python
        Dataset["dataset.py"]:::python
        Collate["_utils/collate.py"]:::python
        Profiler["profiler.py"]:::python
    end

    subgraph "High-Level Collections & Workflows"
        subgraph "NVIDIA NeMo"
            NemoDataset["dataset.py"]:::python
            NemoCollectionsData["megatron data"]:::python
            NemoModels["models"]:::python
            NemoModules["modules"]:::python
            NemoParts["parts"]:::python
        end
        subgraph "Megatron GPT"
            Bpipe["bpipe_state.py"]:::python
            Schedules["schedules.py"]:::python
        end
        Examples["examples tuning scripts"]:::external
        Experiments["experiments pipelines"]:::external
    end

    subgraph "External Infrastructure"
        Docker["Docker"]:::external
        Cluster["Compute Cluster"]:::external
        ThirdParty["Third-party libs"]:::external
    end

    %% Connections
    CMake --> OpsSchemas
    CMake --> SchedulerAPI
    SetupPy --> PyBind
    PyProject --> PyBind
    Requirements --> PyBind

    PyBind --> OpsCode
    PyBind --> SchedulerAPI
    PyBind --> RPCSchemas
    PyBind --> SysPy

    DataLoader --> PyBind
    Dataset --> PyBind
    Collate --> PyBind
    Profiler --> PyBind

    NemoDataset --> PyBind
    NemoCollectionsData --> PyBind
    NemoModels --> PyBind
    NemoModules --> PyBind
    NemoParts --> PyBind

    Bpipe --> PyBind
    Schedules --> PyBind

    SchedulerAPI --> Partition
    SchedulerAPI --> Scatter
    SchedulerAPI --> RPCSchemas
    SchedulerAPI --> Cluster

    RPCSchemas --> RPCHeader
    RPCHeader --> RPCEpmty
    RPCPy --> PyBind
    RPCPy --> Cluster

    OpsCode --> Cluster

    Examples --> NemoCollectionsData
    Examples --> NemoModels
    Examples --> PyBind

    Experiments --> Docker
    Experiments --> Cluster
    Experiments --> Examples

    ThirdParty --> OpsSchemas
    ThirdParty --> SchedulerAPI
    ThirdParty --> RPCSchemas

    %% Click Events
    click CMake "https://github.com/9rum/flatflow/blob/master/CMakeLists.txt"
    click SetupPy "https://github.com/9rum/flatflow/blob/master/setup.py"
    click PyProject "https://github.com/9rum/flatflow/blob/master/pyproject.toml"
    click Requirements "https://github.com/9rum/flatflow/blob/master/requirements.txt"

    click OpsSchemas "https://github.com/9rum/flatflow/tree/master/flatflow/ops/"
    click OpsCode "https://github.com/9rum/flatflow/tree/master/flatflow/ops/"

    click SchedulerAPI "https://github.com/9rum/flatflow/blob/master/flatflow/scheduler/scheduler.h"
    click Partition "https://github.com/9rum/flatflow/blob/master/flatflow/scheduler/internal/partition.h"
    click Scatter "https://github.com/9rum/flatflow/blob/master/flatflow/scheduler/internal/scatter.h"

    click RPCSchemas "https://github.com/9rum/flatflow/blob/master/flatflow/rpc/controlplane.fbs"
    click RPCHeader "https://github.com/9rum/flatflow/blob/master/flatflow/rpc/controlplane.h"
    click RPCEpmty "https://github.com/9rum/flatflow/blob/master/flatflow/rpc/empty.fbs"
    click RPCPy "https://github.com/9rum/flatflow/blob/master/flatflow/rpc/controlplane.py"

    click SysPy "https://github.com/9rum/flatflow/blob/master/flatflow/sys/sysmodule.py"

    click PyBind "https://github.com/9rum/flatflow/blob/master/flatflow/pybind.cc"

    click DataLoader "https://github.com/9rum/flatflow/blob/master/flatflow/torch/utils/data/dataloader.py"
    click Dataset "https://github.com/9rum/flatflow/blob/master/flatflow/torch/utils/data/dataset.py"
    click Collate "https://github.com/9rum/flatflow/blob/master/flatflow/torch/utils/data/_utils/collate.py"
    click Profiler "https://github.com/9rum/flatflow/blob/master/flatflow/torch/profiler/profiler.py"

    click NemoDataset "https://github.com/9rum/flatflow/blob/master/flatflow/nemo/core/classes/dataset.py"
    click NemoCollectionsData "https://github.com/9rum/flatflow/tree/master/flatflow/nemo/collections/nlp/data/language_modeling/megatron/"
    click NemoModels "https://github.com/9rum/flatflow/tree/master/flatflow/nemo/collections/nlp/models/language_modeling/"
    click NemoModules "https://github.com/9rum/flatflow/tree/master/flatflow/nemo/collections/nlp/modules/common/megatron/"
    click NemoParts "https://github.com/9rum/flatflow/tree/master/flatflow/nemo/collections/nlp/parts/"

    click Bpipe "https://github.com/9rum/flatflow/blob/master/flatflow/megatron/core/bpipe/bpipe_state.py"
    click Schedules "https://github.com/9rum/flatflow/blob/master/flatflow/megatron/core/pipeline_parallel/schedules.py"

    click Examples "https://github.com/9rum/flatflow/tree/master/examples/nemo/nlp/language_modeling/tuning/"
    click Experiments "https://github.com/9rum/flatflow/tree/master/experiments/"

    %% Styles
    classDef cpp fill:#bbdefb,stroke:#333,stroke-width:1px
    classDef python fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef external fill:#eeeeee,stroke:#333,stroke-width:1px
```
