# flagopen/flagcx architecture

> Architecture diagram of the GitHub repository flagopen/flagcx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/flagopen/flagcx
- Repository: https://github.com/flagopen/flagcx

## Overview

##### Project Type and Purpose
- FlagCX is a high-performance communication library focusing on abstracting and optimizing cross-chip communication. It supports multiple native backends (e.g., NCCL, IXCCL, CNCL, GLOO) via specialized adaptors and includes a service layer that handles initialization, debugging, and parameter management.
- In addition, it is integrated with upper-layer applications through plugins (for example, its PyTorch plugin).

##### Main Components to Highlight
A. Core Communication Framework (flagcx/core)
- This is the heart of the library and includes:
- Cluster and group management (cluster.cc, group.cc)
- Collective operation implementations (collectives.cc)
- Low-level networking, device, and transport abstractions (net.cc, socket.cc, transport.cc)
- Initialization, topology, and proxy settings (init.cc, topo.cc, proxy.cc)
- These modules define the communication primitives and infrastructure for message passing.

B. Adaptors (flagcx/adaptor)
- Implement platform-specific or backend-specific integration layers.
- Different adaptor files such as nccl_adaptor, cuda_adaptor, gloo_adaptor, and others indicate that for each communication library or hardware type a dedicated adaptor exists.
- Also includes bootstrap adaptors to initiate or wrap around the lower-level functionality.

C. Service Layer (flagcx/service)
- Provides higher-level utilities:
- Bootstrapping services, debugging, parameter handling, and timing utilities.
- These services likely handle the startup and run-time management of the communication operations.

D. Plugin Integration (plugin/torch)
- Contains wrapper code to integrate FlagCX with higher-level frameworks (notably PyTorch):
- Python interface via setup.py and corresponding C++ backend implementations.
- Example directories and scripts illustrate how integration and testing are handled.

E. Testing Components (test folder)
- Contains both performance tests (test/perf) and unit tests (test/unittest):
- Performance tests focus on benchmarking various collective operations.
- Unit tests confirm correctness in core, adaptor, and topology operations.

##### Relationships and Interactions
- The Core layer provides the central communication functions and serves as the intermediary for all data transfers.
- The Service Layer sits on top of the Core, offering functionality for initialization, debugging, and configuration and interacting with the Adaptor and Core layers.
- External native libraries (NCCL, IXCCL, CNCL, GLOO) should be shown as external dependencies interacting with the corresponding Adaptors.

##### Architectural Patterns and Design Principles
- Modular Design: Separate components for adaptor, core, and service imply a modular architecture where each module has a distinct responsibility.
- Abstraction and Adaptation: Adaptors abstract the differences between native collective communication libraries.
- Plugin Architecture: The plugin folder demonstrates integration capabilities for extending functionalities to other ecosystems (e.g., PyTorch).
- Extensibility: The design supports multiple backends and easy addition of new communication primitives.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Plugin Integration Layer"
        PL("Plugin Integration (PyTorch)")
    end

    subgraph "Service Layer"
        SL("Service Layer (init, debug, params)")
    end

    subgraph "Core Communication Engine"
        CORE("Core Communication Engine (clusters, collectives, network)")
    end

    subgraph "Communication Adaptors"
        ADAP("Adaptors (NCCL, IXCCL, CNCL, GLOO, etc.)")
    end

    subgraph "External Libraries"
        EXT1("NCCL")
        EXT2("IXCCL")
        EXT3("CNCL")
        EXT4("GLOO")
    end

    subgraph "Testing Components"
        TEST("Testing (perf & unittest)")
    end

    %% Relationships
    PL -->|"APIcalls"| CORE
    PL -->|"APIcalls"| ADAP
    SL -->|"init_debug"| CORE
    CORE -->|"invokes_adaptors"| ADAP
    EXT1 -->|"native_call"| ADAP
    EXT2 -->|"native_call"| ADAP
    EXT3 -->|"native_call"| ADAP
    EXT4 -->|"native_call"| ADAP

    %% Click Events for Component Mapping
    click CORE "https://github.com/flagopen/flagcx/tree/main/flagcx/core"
    click ADAP "https://github.com/flagopen/flagcx/tree/main/flagcx/adaptor"
    click SL "https://github.com/flagopen/flagcx/tree/main/flagcx/service"
    click PL "https://github.com/flagopen/flagcx/tree/main/plugin/torch"
    click TEST "https://github.com/flagopen/flagcx/tree/main/test"

    %% Styles
    classDef plugin fill:#8ecae6,stroke:#023047,stroke-width:2px,color:#000;
    classDef service fill:#219ebc,stroke:#023047,stroke-width:2px,color:#fff;
    classDef core fill:#ffb703,stroke:#9b2226,stroke-width:2px,color:#000;
    classDef adaptor fill:#fb8500,stroke:#9b2226,stroke-width:2px,color:#000;
    classDef external fill:#d62828,stroke:#ffffff,stroke-width:2px,color:#fff;
    classDef testing fill:#e5e5e5,stroke:#000000,stroke-width:2px,color:#000;

    class PL plugin;
    class SL service;
    class CORE core;
    class ADAP adaptor;
    class EXT1,EXT2,EXT3,EXT4 external;
    class TEST testing;
```
