# facebookresearch/flow_matching architecture

> Architecture diagram of the GitHub repository facebookresearch/flow_matching, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/flow_matching
- Repository: https://github.com/facebookresearch/flow_matching

## Overview

##### Project Overview and Purpose
- This project is a PyTorch-based library that implements Flow Matching algorithms with both continuous and discrete variants. Its core purpose is to provide the building blocks—loss functions, path sampling routines, schedulers, and solvers—for training and deploying flow matching models.
- The repository not only offers the “flow_matching” core package but also contains several examples (for image, text, and synthetic data) that demonstrate training pipelines, as well as comprehensive testing and documentation to support development and research.

##### Analyzing the File Structure
- Core Library (flow_matching):
- The main code is located in the “flow_matching” directory with submodules such as:
▪ loss: Contains various loss function implementations.
▪ path: Implements routines for modeling paths, including a “scheduler” subdirectory for scheduling and transformation utilities.
▪ solver: Provides both continuous and discrete solvers, along with Riemannian ODE solvers.
▪ utils: Offers helper functions, model wrappers, and utilities for manifold handling and other functionalities.
- Examples:
- The “examples” folder is organized by modality (image and text) as well as synthetic notebooks. This shows how the library is used in practice, setting up training routines and data preprocessing pipelines.
- Tests and Documentation:
- The “tests” directory contains unit tests for various modules, ensuring correctness.
- The “docs” folder and README give detailed usage instructions, examples, and references to the original academic paper.

##### Identifying the Main Components
- Core Modules:
- Loss Module: Implements loss functions for training Flow Matching models.
- Path Module: Handles path-based computations, including path sampling and affine/mixture variants.
▪ Also includes Schedulers and Transformation logic in its subdirectory.
- Solver Module: Contains solvers that compute model flows including discrete, ODE, and Riemannian variants.
- Utils Module: Provides necessary utilities (sampling methods, manifold definitions, model wrapping) used across core modules.
- Example Applications:
- Image Training Pipeline: Includes data transformation, model definitions (e.g., U-Net variants), and training loops.
- Text Modeling Pipeline: Contains tokenizer, data loaders, and training/evaluation scripts tailored to text data.
- Supporting Infrastructure:
- Testing Framework: Unit tests across modules.
- Documentation and CI: Automated build/deployment pipelines (GitHub Actions workflows) and documentation generation scripts.
- Configuration: Environment files and setup scripts ensure reproducibility (e.g., environment.yml, setup.py).

##### Determining Relationships and Interactions
- The core “flow_matching” components interact as follows:
- Data and Input: Passed into the example scripts/notebooks.
- Loss functions require outputs from the Solver and models influenced by computations from the Path module.
- Schedulers from the Path module transform the inputs to the solvers, which compute the required flows.
- Utils support several modules by handling data sampling, manifolds, and model wrapping.
- The examples orchestrate these relationships by:
- Using pre-built modules to train models.
- Integrating data transformation (as seen in dedicated training scripts) with core components.
- Demonstrating both continuous and discrete training regimes.

## Diagram (Mermaid)

```mermaid
graph TD
  subgraph "External Dependencies"
    direction TB
    EXT1["PyTorch (v2.1+)"]:::external
    EXT2["Python (3.9)"]:::external
    EXT3["CI/CD (GitHub Actions)"]:::external
  end

  subgraph "Example Applications"
    direction TB
    IE["Image Example Application"]:::example
    TE["Text Example Application"]:::example
  end

  subgraph "Core Library"
    direction TB
    LF["Loss Functions"]:::core
    PM["Path Module"]:::core
    ST["Schedulers & Transformations"]:::core_small
    SM["Solver Module"]:::core
    UM["Utils Module"]:::utils
    MM["Manifolds"]:::utils_small
    MW["Model Wrapping"]:::utils_small
  end

  subgraph "Supporting Infrastructure"
    direction TB
    TF["Testing Framework"]:::infra
    DC["Documentation"]:::infra
  end

  %% External Dependencies flow to Examples
  EXT1 -->|"depends_on"| IE
  EXT1 -->|"depends_on"| TE
  EXT2 -->|"depends_on"| IE
  EXT2 -->|"depends_on"| TE
  EXT3 -->|"triggers"| IE
  EXT3 -->|"triggers"| TE

  %% Examples use Core Library modules
  IE -->|"uses"| LF
  IE -->|"uses"| PM
  IE -->|"uses"| SM
  IE -->|"uses"| UM

  TE -->|"uses"| LF
  TE -->|"uses"| PM
  TE -->|"uses"| SM
  TE -->|"uses"| UM

  %% Internal Core Library Relations
  PM -->|"includes"| ST
  ST -->|"influences"| SM
  UM -->|"supports"| LF
  UM -->|"supports"| PM
  UM -->|"supports"| SM
  UM -->|"contains"| MM
  UM -->|"contains"| MW

  %% Core Library flows to Supporting Infrastructure for testing and docs
  LF -->|"validated_by"| TF
  PM -->|"validated_by"| TF
  SM -->|"validated_by"| TF
  UM -->|"validated_by"| TF
  TF -->|"documented_in"| DC

  %% Click Events for Component Mapping
  click LF "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/loss/"
  click PM "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/path/"
  click ST "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/path/scheduler/"
  click SM "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/solver/"
  click UM "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/utils/"
  click MM "https://github.com/facebookresearch/flow_matching/tree/main/flow_matching/utils/manifolds/"
  click MW "https://github.com/facebookresearch/flow_matching/blob/main/flow_matching/utils/model_wrapper.py"
  click IE "https://github.com/facebookresearch/flow_matching/tree/main/examples/image/"
  click TE "https://github.com/facebookresearch/flow_matching/tree/main/examples/text/"
  click TF "https://github.com/facebookresearch/flow_matching/tree/main/tests/"
  click DC "https://github.com/facebookresearch/flow_matching/tree/main/docs/"
  
  %% Class Definitions for Colors
  classDef core fill:#ADD8E6,stroke:#333,stroke-width:2px,color:#000;
  classDef core_small fill:#B0E0E6,stroke:#333,stroke-width:2px,color:#000;
  classDef utils fill:#90EE90,stroke:#333,stroke-width:2px,color:#000;
  classDef utils_small fill:#98FB98,stroke:#333,stroke-width:2px,color:#000;
  classDef example fill:#FFDAB9,stroke:#333,stroke-width:2px,color:#000;
  classDef infra fill:#D3D3D3,stroke:#333,stroke-width:2px,color:#000;
  classDef external fill:#FFDEAD,stroke:#333,stroke-width:2px,color:#000;
```
