# mounikamandava011/batch-inference-engine architecture

> Architecture diagram of the GitHub repository mounikamandava011/batch-inference-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mounikamandava011/batch-inference-engine
- Repository: https://github.com/mounikamandava011/batch-inference-engine

## Overview

Batch Inference Engine accepts local JSON prompt batches as asynchronous jobs, processes each prompt through bounded workers and a configurable inference provider, persists progress and results, then exposes status and ordered streaming downloads. The graph follows the sampled routes, coordinator, pipeline, executor, and repository calls; major documented capabilities such as backpressure, retries, SQLite durability, and provider pacing are represented. Provider choice is wired at startup. Other than README-described behavior, unsampled modules are not used to infer additional call relationships.

Main entry points: [`app/main.py`](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/main.py) (App setup), [`app/api/routes_jobs.py`](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/routes_jobs.py) (Job endpoints), [`app/repositories/job_repository.py`](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/repositories/job_repository.py) (Job repository), [`app/services/coordinator.py`](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/coordinator.py) (Job coordinator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["REST API"]
  node_app["App setup<br/>[main.py]"]
  node_job_routes["Job endpoints<br/>[routes_jobs.py]"]
  node_ops_routes["Health endpoint<br/>[routes_ops.py]"]
  node_schemas["API schemas<br/>[schemas.py]"]
  node_paths["Input path guard<br/>[paths.py]"]
  node_settings["Runtime settings<br/>[config.py]"]
end

subgraph group_jobs["Job Processing"]
  node_coordinator["Job coordinator<br/>[coordinator.py]"]
  node_pipeline["Stream pipeline<br/>[pipeline.py]"]
  node_models["Job and item models<br/>[models.py]"]
end

subgraph group_inference["Inference"]
  node_executor["Inference executor<br/>[executor.py]"]
  node_pacer["Request pacing<br/>[rate_limit.py]"]
  node_provider_contract["Provider contract<br/>[base.py]"]
  node_do_provider["DigitalOcean client<br/>[digitalocean.py]"]
  node_fake_provider["Fake client<br/>[fake.py]"]
end

subgraph group_persistence["Persistence"]
  node_repository["Job repository<br/>[job_repository.py]"]
  node_database[("SQLite database<br/>[db.py]")]
end

node_caller(("API caller"))

node_caller -->|"submits, checks, downloads"| node_job_routes
node_caller -->|"checks health"| node_ops_routes
node_app -->|"registers routes"| node_job_routes
node_app -->|"registers routes"| node_ops_routes
node_job_routes -->|"validates and responds"| node_schemas
node_job_routes -->|"resolves input"| node_paths
node_job_routes -->|"creates and reads jobs"| node_repository
node_job_routes -->|"starts job"| node_coordinator
node_app -->|"loads configuration"| node_settings
node_app -->|"connects database"| node_database
node_app -->|"creates repository"| node_repository
node_app -->|"configures coordinator"| node_coordinator
node_app -->|"selects provider"| node_do_provider
node_app -->|"selects provider"| node_fake_provider
node_coordinator -->|"updates job state"| node_repository
node_coordinator -->|"starts producer, workers, writer"| node_pipeline
node_coordinator -->|"shares with workers"| node_executor
node_pipeline -->|"records ingestion and results"| node_repository
node_pipeline -->|"builds work and results"| node_models
node_executor -->|"calls completion"| node_provider_contract
node_executor -->|"paces requests"| node_pacer
node_executor -->|"returns item result"| node_models
node_do_provider -->|"implements contract"| node_provider_contract
node_repository -.->|"reads and writes"| node_database
node_job_routes -->|"pages ordered results"| node_repository
node_repository -->|"maps records"| node_models

click node_app "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/main.py"
click node_job_routes "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/routes_jobs.py"
click node_ops_routes "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/routes_ops.py"
click node_schemas "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/schemas.py"
click node_paths "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/core/paths.py"
click node_coordinator "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/coordinator.py"
click node_pipeline "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/pipeline.py"
click node_models "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/domain/models.py"
click node_executor "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/executor.py"
click node_pacer "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/rate_limit.py"
click node_provider_contract "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/base.py"
click node_do_provider "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/digitalocean.py"
click node_fake_provider "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/fake.py"
click node_settings "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/core/config.py"
click node_repository "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/repositories/job_repository.py"
click node_database "https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/repositories/db.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_job_routes,node_ops_routes,node_schemas,node_paths,node_settings toneBlue
class node_coordinator,node_pipeline,node_models toneAmber
class node_executor,node_pacer,node_provider_contract,node_do_provider,node_fake_provider,node_caller toneMint
class node_repository,node_database toneRose
```

## Components

### REST API

- [App setup](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/main.py): `app/main.py`
- [Job endpoints](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/routes_jobs.py): `app/api/routes_jobs.py`
- [Health endpoint](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/routes_ops.py): `app/api/routes_ops.py`
- [API schemas](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/api/schemas.py): `app/api/schemas.py`
- [Input path guard](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/core/paths.py): `app/core/paths.py`
- [Runtime settings](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/core/config.py): `app/core/config.py`

### Job Processing

- [Job coordinator](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/coordinator.py): `app/services/coordinator.py`
- [Stream pipeline](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/pipeline.py): `app/services/pipeline.py`
- [Job and item models](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/domain/models.py): `app/domain/models.py`

### Inference

- [Inference executor](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/executor.py): `app/services/executor.py`
- [Request pacing](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/services/rate_limit.py): `app/services/rate_limit.py`
- [Provider contract](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/base.py): `app/inference/base.py`
- [DigitalOcean client](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/digitalocean.py): `app/inference/digitalocean.py`
- [Fake client](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/inference/fake.py): `app/inference/fake.py`

### Persistence

- [Job repository](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/repositories/job_repository.py): `app/repositories/job_repository.py`
- [SQLite database](https://github.com/mounikamandava011/batch-inference-engine/blob/main/app/repositories/db.py): `app/repositories/db.py`

### Other components

- **API caller**

## Connections

- API caller → Job endpoints: submits, checks, downloads (evidence: `README.md`)
- API caller → Health endpoint: checks health (evidence: `README.md`)
- App setup → Job endpoints: registers routes (evidence: `app/main.py`)
- App setup → Health endpoint: registers routes (evidence: `app/main.py`)
- Job endpoints → API schemas: validates and responds (evidence: `app/api/routes_jobs.py`)
- Job endpoints → Input path guard: resolves input (evidence: `app/api/routes_jobs.py`)
- Job endpoints → Job repository: creates and reads jobs (evidence: `app/api/routes_jobs.py`)
- Job endpoints → Job coordinator: starts job (evidence: `app/api/routes_jobs.py`)
- App setup → Runtime settings: loads configuration (evidence: `app/main.py`)
- App setup → SQLite database: connects database (evidence: `app/main.py`)
- App setup → Job repository: creates repository (evidence: `app/main.py`)
- App setup → Job coordinator: configures coordinator (evidence: `app/main.py`)
- App setup → DigitalOcean client: selects provider (evidence: `app/main.py`)
- App setup → Fake client: selects provider (evidence: `app/main.py`)
- Job coordinator → Job repository: updates job state (evidence: `app/services/coordinator.py`)
- Job coordinator → Stream pipeline: starts producer, workers, writer (evidence: `app/services/coordinator.py`)
- Job coordinator → Inference executor: shares with workers (evidence: `app/services/coordinator.py`)
- Stream pipeline → Job repository: records ingestion and results (evidence: `app/services/pipeline.py`)
- Stream pipeline → Job and item models: builds work and results (evidence: `app/services/pipeline.py`)
- Inference executor → Provider contract: calls completion (evidence: `app/services/executor.py`)
- Inference executor → Request pacing: paces requests (evidence: `app/services/executor.py`)
- Inference executor → Job and item models: returns item result (evidence: `app/services/executor.py`)
- DigitalOcean client → Provider contract: implements contract (evidence: `app/inference/digitalocean.py`)
- Job repository → SQLite database: reads and writes
- Job endpoints → Job repository: pages ordered results (evidence: `app/api/routes_jobs.py`)
- Job repository → Job and item models: maps records (evidence: `app/repositories/job_repository.py`)
