How roboflow/inference works
Roboflow Inference is a self-hosted computer-vision server for model predictions, composable Workflows, and camera/video processing. The principal path is a client request to the server API, through inference/model handling, to a prediction response; Workflows extend this with multi-step model and business-logic execution. A separate stream path manages video pipelines and dispatches results. The README documents capabilities beyond the excerpts; dashed links mark relationships whose implementation was not shown, rather than asserting specific calls.
Main entry points: routes.py HTTP Routes · base.py Model Manager · roboflow.py Model Registry · engine.py Workflow Engine
Components 22
API and Requests
- HTTP Routes
inference/core/interfaces/http/builder/routes.py - Inference Requests
inference/core/entities/requests/inference.py
Model Inference
- Model Manager
inference/core/managers/base.py - Model Registry
inference/core/registries/roboflow.py - Model Families
inference/models - Model Cache
inference/core/cache
Workflow Execution
- Workflow API Bindings
inference/core/interfaces/workflows_models_provider.py - Step Error Handling
inference/core/interfaces/workflows_step_error_handlers.py - Workflow Engine
workflows/roboflow_workflows/execution_engine/entities/engine.py - Execution Data
workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py - Dynamic Batches
workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/dynamic_batches_manager.py
Video and Streams
- Stream Manager
stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py - Inference Pipeline Host
inference/core/interfaces/legacy_stream/inference_pipeline.py - Video Pipeline
stream_vision/streamvision/stream/pipeline.py - Video Sources
stream_vision/streamvision/camera/video_source.py - Stream Manager Service
stream_vision/streamvision/stream_manager/manager_app/app.py
Platform Services
- Roboflow Services
inference/core/roboflow_api.py - Runtime Configuration
inference/core/env.py - Telemetry
inference/core/telemetry.py - Server Logging
inference/core/logger.py - Error Types
inference/core/exceptions.py
Other components
- API Client
Connections 22
- API Client → HTTP Routes: sends requests
README.md - HTTP Routes → Inference Requests: validates as
- HTTP Routes → Model Manager: dispatches inference
- Model Manager → Model Registry: resolves models
inference/core/managers/base.py - Model Registry → Roboflow Services: fetches model data
inference/core/registries/roboflow.py - Model Manager → Model Families: runs models
- Model Registry → Model Cache: uses cached metadata
inference/core/registries/roboflow.py - HTTP Routes → Workflow Engine: runs Workflows
- Workflow Engine → Execution Data: manages execution data
- Execution Data → Dynamic Batches: delegates batching
workflows/roboflow_workflows/execution_engine/v1/executor/execution_data_manager/manager.py - Workflow API Bindings → Model Manager: provides models
inference/core/interfaces/workflows_models_provider.py - Workflow Engine → Step Error Handling: handles step errors
- API Client → Stream Manager Service: manages video streams
- Stream Manager Service → Stream Manager: routes commands
- Stream Manager → Inference Pipeline Host: initializes host
- Inference Pipeline Host → Video Pipeline: extends pipeline
inference/core/interfaces/legacy_stream/inference_pipeline.py - Stream Manager → Video Pipeline: starts pipeline
stream_vision/streamvision/stream_manager/manager_app/inference_pipeline_manager.py - Video Pipeline → Video Sources: collects frames
stream_vision/streamvision/stream/pipeline.py - Runtime Configuration → HTTP Routes: configures server
inference/core/interfaces/http/builder/routes.py - HTTP Routes → Telemetry: records usage
- HTTP Routes → Server Logging: logs requests
- HTTP Routes → Error Types: maps errors