# openinferencefabric/oif architecture

> Architecture diagram of the GitHub repository openinferencefabric/oif, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openinferencefabric/oif
- Repository: https://github.com/openinferencefabric/oif

## Overview

OIF is a GPU inference platform that installs Kubernetes-based infrastructure for NVIDIA and AMD, then lets users schedule workloads, serve models, benchmark them, and observe fleet health. The principal user workflow begins with the `oif` CLI: inspect and configure a host, install the platform, submit GPU workloads through Run:ai/KAI, and use serving and benchmark tooling to produce performance results. Vendor-aware placement and accelerator topology are distinct scheduling stages; observability is a parallel operational capability. README statements support the broad platform relationships; edges from sampled controller code are limited to its visible Kubernetes API interactions.

Main entry points: [`installer/stages`](https://github.com/openinferencefabric/oif/tree/main/installer/stages) (Installer stages), [`bench/collect.py`](https://github.com/openinferencefabric/oif/blob/main/bench/collect.py) (Benchmark runner), [`platform/gpu-amd`](https://github.com/openinferencefabric/oif/tree/main/platform/gpu-amd) (AMD GPU Operator), [`platform/runai`](https://github.com/openinferencefabric/oif/tree/main/platform/runai) (Run:ai and KAI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_install["Installation and platform"]
  node_cli["OIF CLI"]
  node_installer["Installer stages"]
  node_kubernetes["Kubernetes"]
  node_nvidia["NVIDIA GPU Operator"]
  node_amd["AMD GPU Operator"]
end

subgraph group_schedule["Workload scheduling"]
  node_runai["Run:ai and KAI"]
  node_vendorpool["Vendor pool admission<br/>[main.go]"]
  node_topology["Topology labeling<br/>[main.go]"]
  node_domainview["Accelerator domains<br/>[domains.go]"]
  node_aijob["AIJob controller<br/>[main.go]"]
  node_aijoblogic["Placement policy logic<br/>[aijob.go]"]
  node_amdshare["AMD GPU sharing<br/>[main.go]"]
  node_runaiapi["Run:ai API client<br/>[runai_api.py]"]
end

subgraph group_serve["Inference and benchmarking"]
  node_dynamo["Dynamo serving"]
  node_rocmimages["ROCm serving images"]
  node_bench["Benchmark runner<br/>[collect.py]"]
  node_report["Benchmark reports<br/>[report.py]"]
end

subgraph group_operate["Observability and state"]
  node_prometheus["Prometheus metrics"]
  node_grafana["Grafana dashboards"]
  node_database[("Platform database")]
end

node_user(("Platform user"))
node_clusteradmin(("Cluster administrator"))

node_user -->|"runs commands"| node_cli
node_cli -->|"installs platform"| node_installer
node_installer -->|"provisions cluster"| node_kubernetes
node_installer -->|"installs operator"| node_nvidia
node_installer -->|"installs operator"| node_amd
node_installer -->|"configures orchestration"| node_runai
node_clusteradmin -.->|"manages projects"| node_runaiapi
node_aijob -.->|"uses placement logic"| node_aijoblogic
node_aijob -->|"reads and writes API"| node_kubernetes
node_topology -->|"labels nodes"| node_kubernetes
node_domainview -.->|"summarizes domains"| node_topology
node_vendorpool -.->|"pins vendor pool"| node_runai
node_amdshare -.->|"shares AMD devices"| node_amd
node_runai -->|"schedules serving"| node_dynamo
node_dynamo -->|"uses ROCm build"| node_rocmimages
node_user -->|"runs benchmark"| node_bench
node_bench -->|"benchmarks endpoint"| node_dynamo
node_bench -->|"produces report"| node_report
node_nvidia -->|"exports GPU metrics"| node_prometheus
node_amd -->|"exports device metrics"| node_prometheus
node_prometheus -->|"feeds dashboards"| node_grafana
node_installer -.->|"installs platform state"| node_database

click node_cli "https://github.com/openinferencefabric/oif/blob/main/oif"
click node_installer "https://github.com/openinferencefabric/oif/tree/main/installer/stages"
click node_kubernetes "https://github.com/openinferencefabric/oif/tree/main/platform/kubernetes"
click node_nvidia "https://github.com/openinferencefabric/oif/tree/main/platform/gpu-nvidia"
click node_amd "https://github.com/openinferencefabric/oif/tree/main/platform/gpu-amd"
click node_runai "https://github.com/openinferencefabric/oif/tree/main/platform/runai"
click node_vendorpool "https://github.com/openinferencefabric/oif/blob/main/images/oif-vendor-pool/main.go"
click node_topology "https://github.com/openinferencefabric/oif/blob/main/images/oif-topology/main.go"
click node_domainview "https://github.com/openinferencefabric/oif/blob/main/images/oif-topology/domains.go"
click node_aijob "https://github.com/openinferencefabric/oif/blob/main/images/oif-aijob/main.go"
click node_aijoblogic "https://github.com/openinferencefabric/oif/blob/main/images/oif-aijob/aijob.go"
click node_amdshare "https://github.com/openinferencefabric/oif/blob/main/images/oif-amd-share/main.go"
click node_dynamo "https://github.com/openinferencefabric/oif/tree/main/platform/dynamo"
click node_rocmimages "https://github.com/openinferencefabric/oif/tree/main/images/dynamo-vllm-rocm"
click node_bench "https://github.com/openinferencefabric/oif/blob/main/bench/collect.py"
click node_report "https://github.com/openinferencefabric/oif/blob/main/bench/report.py"
click node_prometheus "https://github.com/openinferencefabric/oif/tree/main/platform/observability"
click node_grafana "https://github.com/openinferencefabric/oif/tree/main/platform/observability/dashboards"
click node_database "https://github.com/openinferencefabric/oif/tree/main/platform/database"
click node_runaiapi "https://github.com/openinferencefabric/oif/blob/main/installer/tools/runai_api.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_installer,node_kubernetes,node_nvidia,node_amd,node_user toneBlue
class node_runai,node_vendorpool,node_topology,node_domainview,node_aijob,node_aijoblogic,node_amdshare,node_runaiapi toneAmber
class node_dynamo,node_rocmimages,node_bench,node_report toneMint
class node_prometheus,node_grafana,node_database toneRose
class node_clusteradmin toneIndigo
```

## Components

### Installation and platform

- [OIF CLI](https://github.com/openinferencefabric/oif/blob/main/oif): `oif`
- [Installer stages](https://github.com/openinferencefabric/oif/tree/main/installer/stages): `installer/stages`
- [Kubernetes](https://github.com/openinferencefabric/oif/tree/main/platform/kubernetes): `platform/kubernetes`
- [NVIDIA GPU Operator](https://github.com/openinferencefabric/oif/tree/main/platform/gpu-nvidia): `platform/gpu-nvidia`
- [AMD GPU Operator](https://github.com/openinferencefabric/oif/tree/main/platform/gpu-amd): `platform/gpu-amd`

### Workload scheduling

- [Run:ai and KAI](https://github.com/openinferencefabric/oif/tree/main/platform/runai): `platform/runai`
- [Vendor pool admission](https://github.com/openinferencefabric/oif/blob/main/images/oif-vendor-pool/main.go): `images/oif-vendor-pool/main.go`
- [Topology labeling](https://github.com/openinferencefabric/oif/blob/main/images/oif-topology/main.go): `images/oif-topology/main.go`
- [Accelerator domains](https://github.com/openinferencefabric/oif/blob/main/images/oif-topology/domains.go): `images/oif-topology/domains.go`
- [AIJob controller](https://github.com/openinferencefabric/oif/blob/main/images/oif-aijob/main.go): `images/oif-aijob/main.go`
- [Placement policy logic](https://github.com/openinferencefabric/oif/blob/main/images/oif-aijob/aijob.go): `images/oif-aijob/aijob.go`
- [AMD GPU sharing](https://github.com/openinferencefabric/oif/blob/main/images/oif-amd-share/main.go): `images/oif-amd-share/main.go`
- [Run:ai API client](https://github.com/openinferencefabric/oif/blob/main/installer/tools/runai_api.py): `installer/tools/runai_api.py`

### Inference and benchmarking

- [Dynamo serving](https://github.com/openinferencefabric/oif/tree/main/platform/dynamo): `platform/dynamo`
- [ROCm serving images](https://github.com/openinferencefabric/oif/tree/main/images/dynamo-vllm-rocm): `images/dynamo-vllm-rocm`
- [Benchmark runner](https://github.com/openinferencefabric/oif/blob/main/bench/collect.py): `bench/collect.py`
- [Benchmark reports](https://github.com/openinferencefabric/oif/blob/main/bench/report.py): `bench/report.py`

### Observability and state

- [Prometheus metrics](https://github.com/openinferencefabric/oif/tree/main/platform/observability): `platform/observability`
- [Grafana dashboards](https://github.com/openinferencefabric/oif/tree/main/platform/observability/dashboards): `platform/observability/dashboards`
- [Platform database](https://github.com/openinferencefabric/oif/tree/main/platform/database): `platform/database`

### Other components

- **Platform user**
- **Cluster administrator**

## Connections

- Platform user → OIF CLI: runs commands (evidence: `README.md`)
- OIF CLI → Installer stages: installs platform (evidence: `README.md`)
- Installer stages → Kubernetes: provisions cluster (evidence: `README.md`)
- Installer stages → NVIDIA GPU Operator: installs operator (evidence: `README.md`)
- Installer stages → AMD GPU Operator: installs operator (evidence: `README.md`)
- Installer stages → Run:ai and KAI: configures orchestration (evidence: `README.md`)
- Cluster administrator → Run:ai API client: manages projects
- AIJob controller → Placement policy logic: uses placement logic
- AIJob controller → Kubernetes: reads and writes API (evidence: `images/oif-aijob/main.go`)
- Topology labeling → Kubernetes: labels nodes (evidence: `images/oif-topology/main.go`)
- Accelerator domains → Topology labeling: summarizes domains
- Vendor pool admission → Run:ai and KAI: pins vendor pool
- AMD GPU sharing → AMD GPU Operator: shares AMD devices
- Run:ai and KAI → Dynamo serving: schedules serving (evidence: `README.md`)
- Dynamo serving → ROCm serving images: uses ROCm build (evidence: `README.md`)
- Platform user → Benchmark runner: runs benchmark (evidence: `README.md`)
- Benchmark runner → Dynamo serving: benchmarks endpoint (evidence: `README.md`)
- Benchmark runner → Benchmark reports: produces report (evidence: `README.md`)
- NVIDIA GPU Operator → Prometheus metrics: exports GPU metrics (evidence: `README.md`)
- AMD GPU Operator → Prometheus metrics: exports device metrics (evidence: `README.md`)
- Prometheus metrics → Grafana dashboards: feeds dashboards (evidence: `README.md`)
- Installer stages → Platform database: installs platform state
