# roboflow/inference architecture

> Architecture diagram of the GitHub repository roboflow/inference, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roboflow/inference
- Repository: https://github.com/roboflow/inference

## Overview

Roboflow Inference is a self-hosted computer-vision server for model predictions, composable Workflows, and camera/video processing. The principal path is a client request to the server API, through inference/model handling, to a prediction response; Workflows extend this with multi-step model and business-logic execution. A separate stream path manages video pipelines and dispatches results. The README documents capabilities beyond the excerpts; dashed links mark relationships whose implementation was not shown, rather than asserting specific calls.

Main entry points: [`inference/core/interfaces/http/builder/routes.py`](https://github.com/roboflow/inference/blob/main/inference/core/interfaces/http/builder/routes.py) (HTTP Routes), [`inference/core/managers/base.py`](https://github.com/roboflow/inference/blob/main/inference/core/managers/base.py) (Model Manager), [`inference/core/registries/roboflow.py`](https://github.com/roboflow/inference/blob/main/inference/core/registries/roboflow.py) (Model Registry), [`workflows/roboflow_workflows/execution_engine/entities/engine.py`](https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/entities/engine.py) (Workflow Engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and Requests"]
  node_api_routes["HTTP Routes<br/>[routes.py]"]
  node_request_types["Inference Requests<br/>[inference.py]"]
end

subgraph group_models["Model Inference"]
  node_model_manager["Model Manager<br/>[base.py]"]
  node_model_registry["Model Registry<br/>[roboflow.py]"]
  node_model_families["Model Families"]
  node_model_assets[("Model Cache")]
end

subgraph group_workflows["Workflow Execution"]
  node_workflow_api["Workflow API Bindings"]
  node_workflow_errors["Step Error Handling"]
  node_workflow_engine["Workflow Engine<br/>[engine.py]"]
  node_execution_data["Execution Data<br/>[manager.py]"]
  node_dynamic_batches["Dynamic Batches"]
end

subgraph group_streams["Video and Streams"]
  node_stream_manager["Stream Manager"]
  node_legacy_pipeline["Inference Pipeline Host"]
  node_video_pipeline["Video Pipeline<br/>[pipeline.py]"]
  node_video_sources["Video Sources<br/>[video_source.py]"]
  node_stream_store["Stream Manager Service<br/>[app.py]"]
end

subgraph group_platform["Platform Services"]
  node_roboflow_api["Roboflow Services<br/>[roboflow_api.py]"]
  node_configuration["Runtime Configuration<br/>[env.py]"]
  node_telemetry["Telemetry<br/>[telemetry.py]"]
  node_server_logs["Server Logging<br/>[logger.py]"]
  node_exceptions["Error Types<br/>[exceptions.py]"]
end

node_client(("API Client"))

node_client -->|"sends requests"| node_api_routes
node_api_routes -.->|"validates as"| node_request_types
node_api_routes -.->|"dispatches inference"| node_model_manager
node_model_manager -.->|"resolves models"| node_model_registry
node_model_registry -->|"fetches model data"| node_roboflow_api
node_model_manager -.->|"runs models"| node_model_families
node_model_registry -.->|"uses cached metadata"| node_model_assets
node_api_routes -.->|"runs Workflows"| node_workflow_engine
node_workflow_engine -.->|"manages execution data"| node_execution_data
node_execution_data -->|"delegates batching"| node_dynamic_batches
node_workflow_api -.->|"provides models"| node_model_manager
node_workflow_engine -.->|"handles step errors"| node_workflow_errors
node_client -.->|"manages video streams"| node_stream_store
node_stream_store -.->|"routes commands"| node_stream_manager
node_stream_manager -.->|"initializes host"| node_legacy_pipeline
node_legacy_pipeline -->|"extends pipeline"| node_video_pipeline
node_stream_manager -->|"starts pipeline"| node_video_pipeline
node_video_pipeline -.->|"collects frames"| node_video_sources
node_configuration -.->|"configures server"| node_api_routes
node_api_routes -.->|"records usage"| node_telemetry
node_api_routes -.->|"logs requests"| node_server_logs
node_api_routes -.->|"maps errors"| node_exceptions

click node_api_routes "https://github.com/roboflow/inference/blob/main/inference/core/interfaces/http/builder/routes.py"
click node_request_types "https://github.com/roboflow/inference/blob/main/inference/core/entities/requests/inference.py"
click node_model_manager "https://github.com/roboflow/inference/blob/main/inference/core/managers/base.py"
click node_model_registry "https://github.com/roboflow/inference/blob/main/inference/core/registries/roboflow.py"
click node_model_families "https://github.com/roboflow/inference/tree/main/inference/models"
click node_model_assets "https://github.com/roboflow/inference/tree/main/inference/core/cache"
click node_roboflow_api "https://github.com/roboflow/inference/blob/main/inference/core/roboflow_api.py"
click node_workflow_api "https://github.com/roboflow/inference/blob/main/inference/core/interfaces/workflows_models_provider.py"
click node_workflow_errors "https://github.com/roboflow/inference/blob/main/inference/core/interfaces/workflows_step_error_handlers.py"
click node_workflow_engine "https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/entities/engine.py"
click node_execution_data "https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py"
click node_dynamic_batches "https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/dynamic_batches_manager.py"
click node_stream_manager "https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py"
click node_legacy_pipeline "https://github.com/roboflow/inference/blob/main/inference/core/interfaces/legacy_stream/inference_pipeline.py"
click node_video_pipeline "https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream/pipeline.py"
click node_video_sources "https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/camera/video_source.py"
click node_stream_store "https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream_manager/manager_app/app.py"
click node_configuration "https://github.com/roboflow/inference/blob/main/inference/core/env.py"
click node_telemetry "https://github.com/roboflow/inference/blob/main/inference/core/telemetry.py"
click node_server_logs "https://github.com/roboflow/inference/blob/main/inference/core/logger.py"
click node_exceptions "https://github.com/roboflow/inference/blob/main/inference/core/exceptions.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api_routes,node_request_types,node_client toneBlue
class node_model_manager,node_model_registry,node_model_families,node_model_assets toneAmber
class node_workflow_api,node_workflow_errors,node_workflow_engine,node_execution_data,node_dynamic_batches toneMint
class node_stream_manager,node_legacy_pipeline,node_video_pipeline,node_video_sources,node_stream_store toneRose
class node_roboflow_api,node_configuration,node_telemetry,node_server_logs,node_exceptions toneIndigo
```

## Components

### API and Requests

- [HTTP Routes](https://github.com/roboflow/inference/blob/main/inference/core/interfaces/http/builder/routes.py): `inference/core/interfaces/http/builder/routes.py`
- [Inference Requests](https://github.com/roboflow/inference/blob/main/inference/core/entities/requests/inference.py): `inference/core/entities/requests/inference.py`

### Model Inference

- [Model Manager](https://github.com/roboflow/inference/blob/main/inference/core/managers/base.py): `inference/core/managers/base.py`
- [Model Registry](https://github.com/roboflow/inference/blob/main/inference/core/registries/roboflow.py): `inference/core/registries/roboflow.py`
- [Model Families](https://github.com/roboflow/inference/tree/main/inference/models): `inference/models`
- [Model Cache](https://github.com/roboflow/inference/tree/main/inference/core/cache): `inference/core/cache`

### Workflow Execution

- [Workflow API Bindings](https://github.com/roboflow/inference/blob/main/inference/core/interfaces/workflows_models_provider.py): `inference/core/interfaces/workflows_models_provider.py`
- [Step Error Handling](https://github.com/roboflow/inference/blob/main/inference/core/interfaces/workflows_step_error_handlers.py): `inference/core/interfaces/workflows_step_error_handlers.py`
- [Workflow Engine](https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/entities/engine.py): `workflows/roboflow_workflows/execution_engine/entities/engine.py`
- [Execution Data](https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py): `workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py`
- [Dynamic Batches](https://github.com/roboflow/inference/blob/main/workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/dynamic_batches_manager.py): `workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/dynamic_batches_manager.py`

### Video and Streams

- [Stream Manager](https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py): `stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py`
- [Inference Pipeline Host](https://github.com/roboflow/inference/blob/main/inference/core/interfaces/legacy_stream/inference_pipeline.py): `inference/core/interfaces/legacy_stream/inference_pipeline.py`
- [Video Pipeline](https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream/pipeline.py): `stream_vision/streamvision/stream/pipeline.py`
- [Video Sources](https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/camera/video_source.py): `stream_vision/streamvision/camera/video_source.py`
- [Stream Manager Service](https://github.com/roboflow/inference/blob/main/stream_vision/streamvision/stream_manager/manager_app/app.py): `stream_vision/streamvision/stream_manager/manager_app/app.py`

### Platform Services

- [Roboflow Services](https://github.com/roboflow/inference/blob/main/inference/core/roboflow_api.py): `inference/core/roboflow_api.py`
- [Runtime Configuration](https://github.com/roboflow/inference/blob/main/inference/core/env.py): `inference/core/env.py`
- [Telemetry](https://github.com/roboflow/inference/blob/main/inference/core/telemetry.py): `inference/core/telemetry.py`
- [Server Logging](https://github.com/roboflow/inference/blob/main/inference/core/logger.py): `inference/core/logger.py`
- [Error Types](https://github.com/roboflow/inference/blob/main/inference/core/exceptions.py): `inference/core/exceptions.py`

### Other components

- **API Client**

## Connections

- API Client → HTTP Routes: sends requests (evidence: `README.md`)
- HTTP Routes → Inference Requests: validates as
- HTTP Routes → Model Manager: dispatches inference
- Model Manager → Model Registry: resolves models (evidence: `inference/core/managers/base.py`)
- Model Registry → Roboflow Services: fetches model data (evidence: `inference/core/registries/roboflow.py`)
- Model Manager → Model Families: runs models
- Model Registry → Model Cache: uses cached metadata (evidence: `inference/core/registries/roboflow.py`)
- HTTP Routes → Workflow Engine: runs Workflows
- Workflow Engine → Execution Data: manages execution data
- Execution Data → Dynamic Batches: delegates batching (evidence: `workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py`)
- Workflow API Bindings → Model Manager: provides models (evidence: `inference/core/interfaces/workflows_models_provider.py`)
- Workflow Engine → Step Error Handling: handles step errors
- API Client → Stream Manager Service: manages video streams
- Stream Manager Service → Stream Manager: routes commands
- Stream Manager → Inference Pipeline Host: initializes host
- Inference Pipeline Host → Video Pipeline: extends pipeline (evidence: `inference/core/interfaces/legacy_stream/inference_pipeline.py`)
- Stream Manager → Video Pipeline: starts pipeline (evidence: `stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py`)
- Video Pipeline → Video Sources: collects frames (evidence: `stream_vision/streamvision/stream/pipeline.py`)
- Runtime Configuration → HTTP Routes: configures server (evidence: `inference/core/interfaces/http/builder/routes.py`)
- HTTP Routes → Telemetry: records usage
- HTTP Routes → Server Logging: logs requests
- HTTP Routes → Error Types: maps errors
