# flagos-ai/torch-fl architecture

> Architecture diagram of the GitHub repository flagos-ai/torch-fl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/flagos-ai/torch-fl
- Repository: https://github.com/flagos-ai/torch-fl

## Overview

torch-fl is a PyTorch device plugin that presents accelerators through one logical `flagos` device. The principal workflow starts with standard PyTorch tensor operations, enters the plugin’s device and ATen dispatch layers, then selects an implementation such as portable FlagGems kernels, vendor-native kernels, compatibility boxing, or CPU fallback. Separate branches provide `torch.compile`, distributed collectives, and profiler integration. The README documents these capabilities; sampled excerpts are partial, so edges show only documented or source-supported relationships, not every platform-specific dispatch detail.

Main entry points: [`csrc/aten/register.cc`](https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/register.cc) (ATen dispatcher), [`torch_fl/__init__.py`](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/__init__.py) (Plugin integration), [`torch_fl/flagos/__init__.py`](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/flagos/__init__.py) (FlagOS device), [`csrc/runtime/functions.cc`](https://github.com/flagos-ai/torch-fl/blob/main/csrc/runtime/functions.cc) (Device runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Device interface"]
  node_plugin["Plugin integration<br/>[__init__.py]"]
  node_device["FlagOS device<br/>[__init__.py]"]
  node_runtime["Device runtime<br/>[functions.cc]"]
  node_platform_runtime["Vendor runtimes"]
end

subgraph group_execution["Operator execution"]
  node_dispatcher["ATen dispatcher<br/>[register.cc]"]
  node_boxing["Compatibility boxing<br/>[device_boxing.h]"]
  node_flaggems["FlagGems kernels"]
  node_native["Vendor kernels"]
  node_fallback["CPU fallback<br/>[fallback.cc]"]
end

subgraph group_compile["Compilation"]
  node_compile_api["Compile API<br/>[__init__.py]"]
  node_inductor["Inductor backend"]
  node_device_interface["Compile device interface"]
  node_tileops["TileOps runtime<br/>[runtime.py]"]
end

subgraph group_parallel["Distributed execution"]
  node_distributed_api["Distributed API<br/>[distributed.py]"]
  node_process_group["FlagOS process group<br/>[process_group.py]"]
  node_collectives["Native collectives"]
end

subgraph group_observability["Profiling"]
  node_profiler_api["Profiler integration"]
  node_device_tracer["Device tracing<br/>[device_tracer.h]"]
end

node_user(("PyTorch user"))
node_pytorch["PyTorch APIs"]
node_result["Tensor result"]

node_user -->|"invokes"| node_pytorch
node_pytorch -->|"loads integration"| node_plugin
node_plugin -->|"exposes"| node_device
node_device -->|"uses runtime"| node_runtime
node_runtime -->|"calls"| node_platform_runtime
node_pytorch -->|"dispatches operations"| node_dispatcher
node_dispatcher -->|"routes kernels"| node_flaggems
node_dispatcher -->|"routes kernels"| node_native
node_dispatcher -->|"routes operations"| node_boxing
node_dispatcher -->|"falls back"| node_fallback
node_flaggems -->|"produces"| node_result
node_native -->|"produces"| node_result
node_boxing -->|"produces"| node_result
node_fallback -->|"produces"| node_result
node_user -->|"requests compilation"| node_compile_api
node_compile_api -->|"invokes"| node_inductor
node_inductor -->|"registers device"| node_device_interface
node_user -->|"requests collectives"| node_distributed_api
node_distributed_api -->|"uses"| node_process_group
node_process_group -->|"dispatches collectives"| node_collectives
node_user -->|"starts profiling"| node_profiler_api
node_profiler_api -->|"controls tracing"| node_device_tracer

click node_plugin "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/__init__.py"
click node_device "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/flagos/__init__.py"
click node_runtime "https://github.com/flagos-ai/torch-fl/blob/main/csrc/runtime/functions.cc"
click node_platform_runtime "https://github.com/flagos-ai/torch-fl/tree/main/csrc/runtime/accelerator"
click node_dispatcher "https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/register.cc"
click node_boxing "https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/device_boxing.h"
click node_flaggems "https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/flaggems_cpp_kernels.cc"
click node_native "https://github.com/flagos-ai/torch-fl/tree/main/csrc/aten/backends/ascend"
click node_fallback "https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/fallback.cc"
click node_compile_api "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/__init__.py"
click node_inductor "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/inductor_backend.py"
click node_device_interface "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/device_interface.py"
click node_tileops "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/tileops/runtime.py"
click node_distributed_api "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/distributed.py"
click node_process_group "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/comm/process_group.py"
click node_collectives "https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/csrc/dataparallel_comm.cc"
click node_profiler_api "https://github.com/flagos-ai/torch-fl/blob/main/csrc/profiler/flagos_kineto_profiler.cc"
click node_device_tracer "https://github.com/flagos-ai/torch-fl/blob/main/csrc/profiler/device_tracer.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_plugin,node_device,node_runtime,node_platform_runtime,node_user toneBlue
class node_dispatcher,node_boxing,node_flaggems,node_native,node_fallback toneAmber
class node_compile_api,node_inductor,node_device_interface,node_tileops,node_pytorch toneMint
class node_distributed_api,node_process_group,node_collectives toneRose
class node_profiler_api,node_device_tracer,node_result toneIndigo
```

## Components

### Device interface

- [Plugin integration](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/__init__.py): `torch_fl/__init__.py`
- [FlagOS device](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/flagos/__init__.py): `torch_fl/flagos/__init__.py`
- [Device runtime](https://github.com/flagos-ai/torch-fl/blob/main/csrc/runtime/functions.cc): `csrc/runtime/functions.cc`
- [Vendor runtimes](https://github.com/flagos-ai/torch-fl/tree/main/csrc/runtime/accelerator): `csrc/runtime/accelerator`

### Operator execution

- [ATen dispatcher](https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/register.cc): `csrc/aten/register.cc`
- [Compatibility boxing](https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/device_boxing.h): `csrc/aten/device_boxing.h`
- [FlagGems kernels](https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/flaggems_cpp_kernels.cc): `csrc/aten/flaggems_cpp_kernels.cc`
- [Vendor kernels](https://github.com/flagos-ai/torch-fl/tree/main/csrc/aten/backends/ascend): `csrc/aten/backends/ascend`
- [CPU fallback](https://github.com/flagos-ai/torch-fl/blob/main/csrc/aten/fallback.cc): `csrc/aten/fallback.cc`

### Compilation

- [Compile API](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/__init__.py): `torch_fl/compile/__init__.py`
- [Inductor backend](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/inductor_backend.py): `torch_fl/compile/inductor_backend.py`
- [Compile device interface](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/compile/device_interface.py): `torch_fl/compile/device_interface.py`
- [TileOps runtime](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/tileops/runtime.py): `torch_fl/tileops/runtime.py`

### Distributed execution

- [Distributed API](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/distributed.py): `torch_fl/distributed.py`
- [FlagOS process group](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/comm/process_group.py): `torch_fl/comm/process_group.py`
- [Native collectives](https://github.com/flagos-ai/torch-fl/blob/main/torch_fl/csrc/dataparallel_comm.cc): `torch_fl/csrc/dataparallel_comm.cc`

### Profiling

- [Profiler integration](https://github.com/flagos-ai/torch-fl/blob/main/csrc/profiler/flagos_kineto_profiler.cc): `csrc/profiler/flagos_kineto_profiler.cc`
- [Device tracing](https://github.com/flagos-ai/torch-fl/blob/main/csrc/profiler/device_tracer.h): `csrc/profiler/device_tracer.h`

### Other components

- **PyTorch user**
- **PyTorch APIs**
- **Tensor result**

## Connections

- PyTorch user → PyTorch APIs: invokes
- PyTorch APIs → Plugin integration: loads integration
- Plugin integration → FlagOS device: exposes
- FlagOS device → Device runtime: uses runtime
- Device runtime → Vendor runtimes: calls
- PyTorch APIs → ATen dispatcher: dispatches operations
- ATen dispatcher → FlagGems kernels: routes kernels
- ATen dispatcher → Vendor kernels: routes kernels
- ATen dispatcher → Compatibility boxing: routes operations
- ATen dispatcher → CPU fallback: falls back
- FlagGems kernels → Tensor result: produces
- Vendor kernels → Tensor result: produces
- Compatibility boxing → Tensor result: produces
- CPU fallback → Tensor result: produces
- PyTorch user → Compile API: requests compilation
- Compile API → Inductor backend: invokes
- Inductor backend → Compile device interface: registers device
- PyTorch user → Distributed API: requests collectives
- Distributed API → FlagOS process group: uses
- FlagOS process group → Native collectives: dispatches collectives
- PyTorch user → Profiler integration: starts profiling
- Profiler integration → Device tracing: controls tracing