# sh4shv4t/inference-forge architecture

> Architecture diagram of the GitHub repository sh4shv4t/inference-forge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sh4shv4t/inference-forge
- Repository: https://github.com/sh4shv4t/inference-forge

## Overview

Inference Forge is an async support-ticket classification service. Clients submit ticket batches to the FastAPI API, which validates input, deduplicates tickets, estimates complexity and ETA, then orchestrates priority-aware adaptive batching and Sarvam inference. Results and job progress are stored in Redis and exposed through polling and SSE; partial failures are isolated. Supporting layers provide circuit breaking, retries, caching, configuration, logging, and metrics. The README documents the overall processing stages; some route-to-pipeline details are not visible in the excerpts, so those documented links cite the README.

Main entry points: [`src/inference_forge/api/routes.py`](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/api/routes.py) (API routes), [`src/inference_forge/pipeline/caller.py`](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/caller.py) (Sarvam caller), [`src/inference_forge/main.py`](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/main.py) (Application setup), [`src/inference_forge/pipeline/cache.py`](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/cache.py) (Ticket deduplication)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and responses"]
  node_app["Application setup<br/>[main.py]"]
  node_routes["API routes<br/>[routes.py]"]
  node_schemas["Request schemas<br/>[schemas.py]"]
end

subgraph group_pipeline["Inference pipeline"]
  node_dedup["Ticket deduplication<br/>[cache.py]"]
  node_estimator["Complexity and ETA<br/>[estimator.py]"]
  node_batcher["Priority batcher<br/>[batcher.py]"]
  node_caller["Sarvam caller<br/>[caller.py]"]
  node_breaker["Circuit breaker<br/>[circuit_breaker.py]"]
end

subgraph group_state["State and persistence"]
  node_jobstore["Job state and events<br/>[job_store.py]"]
  node_redis[("Redis")]
end

subgraph group_operations["Configuration and observability"]
  node_config["Runtime settings<br/>[config.py]"]
  node_logger["Structured logging<br/>[logger.py]"]
  node_metrics["Metrics and health<br/>[metrics.py]"]
end

node_client(("API client"))
node_sarvam{{"Sarvam AI API"}}

node_client -->|"submits and queries"| node_routes
node_app -->|"registers routes"| node_routes
node_app -->|"reads settings"| node_config
node_app -->|"creates cache"| node_dedup
node_app -->|"creates store"| node_jobstore
node_app -->|"creates caller"| node_caller
node_routes -->|"uses models"| node_schemas
node_routes -->|"checks tickets"| node_dedup
node_routes -->|"classifies and estimates"| node_estimator
node_routes -->|"orchestrates batches"| node_batcher
node_routes -->|"processes batches"| node_caller
node_routes -->|"updates and reads jobs"| node_jobstore
node_routes -->|"serves metrics"| node_metrics
node_batcher -->|"classifies tickets"| node_estimator
node_caller -->|"reads and caches results"| node_dedup
node_caller -->|"guards API calls"| node_breaker
node_caller -->|"sends inference requests"| node_sarvam
node_dedup -->|"reads and writes cache"| node_redis
node_jobstore -->|"stores jobs and publishes events"| node_redis
node_routes -->|"streams job events"| node_jobstore
node_caller -->|"records API metrics"| node_metrics
node_dedup -->|"records cache metrics"| node_metrics
node_breaker -->|"records breaker state"| node_metrics
node_routes -->|"logs job activity"| node_logger
node_schemas -->|"uses validation limits"| node_config

click node_app "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/main.py"
click node_routes "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/api/routes.py"
click node_schemas "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/api/schemas.py"
click node_dedup "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/cache.py"
click node_estimator "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/estimator.py"
click node_batcher "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/batcher.py"
click node_caller "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/caller.py"
click node_breaker "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/circuit_breaker.py"
click node_jobstore "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/job_store.py"
click node_config "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/config.py"
click node_logger "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/observability/logger.py"
click node_metrics "https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/observability/metrics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_routes,node_schemas,node_client toneBlue
class node_dedup,node_estimator,node_batcher,node_caller,node_breaker toneAmber
class node_jobstore,node_redis,node_sarvam toneMint
class node_config,node_logger,node_metrics toneRose
```

## Components

### API and responses

- [Application setup](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/main.py): `src/inference_forge/main.py`
- [API routes](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/api/routes.py): `src/inference_forge/api/routes.py`
- [Request schemas](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/api/schemas.py): `src/inference_forge/api/schemas.py`

### Inference pipeline

- [Ticket deduplication](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/cache.py): `src/inference_forge/pipeline/cache.py`
- [Complexity and ETA](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/estimator.py): `src/inference_forge/pipeline/estimator.py`
- [Priority batcher](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/batcher.py): `src/inference_forge/pipeline/batcher.py`
- [Sarvam caller](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/caller.py): `src/inference_forge/pipeline/caller.py`
- [Circuit breaker](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/circuit_breaker.py): `src/inference_forge/pipeline/circuit_breaker.py`

### State and persistence

- [Job state and events](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/pipeline/job_store.py): `src/inference_forge/pipeline/job_store.py`
- **Redis**

### Configuration and observability

- [Runtime settings](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/config.py): `src/inference_forge/config.py`
- [Structured logging](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/observability/logger.py): `src/inference_forge/observability/logger.py`
- [Metrics and health](https://github.com/sh4shv4t/inference-forge/blob/main/src/inference_forge/observability/metrics.py): `src/inference_forge/observability/metrics.py`

### Other components

- **API client**
- **Sarvam AI API**

## Connections

- API client → API routes: submits and queries (evidence: `README.md`)
- Application setup → API routes: registers routes (evidence: `src/inference_forge/main.py`)
- Application setup → Runtime settings: reads settings (evidence: `src/inference_forge/main.py`)
- Application setup → Ticket deduplication: creates cache (evidence: `src/inference_forge/main.py`)
- Application setup → Job state and events: creates store (evidence: `src/inference_forge/main.py`)
- Application setup → Sarvam caller: creates caller (evidence: `src/inference_forge/main.py`)
- API routes → Request schemas: uses models (evidence: `src/inference_forge/api/routes.py`)
- API routes → Ticket deduplication: checks tickets (evidence: `src/inference_forge/api/routes.py`)
- API routes → Complexity and ETA: classifies and estimates (evidence: `README.md`)
- API routes → Priority batcher: orchestrates batches (evidence: `README.md`)
- API routes → Sarvam caller: processes batches (evidence: `README.md`)
- API routes → Job state and events: updates and reads jobs (evidence: `src/inference_forge/api/routes.py`)
- API routes → Metrics and health: serves metrics (evidence: `src/inference_forge/api/routes.py`)
- Priority batcher → Complexity and ETA: classifies tickets (evidence: `src/inference_forge/pipeline/batcher.py`)
- Sarvam caller → Ticket deduplication: reads and caches results (evidence: `src/inference_forge/pipeline/caller.py`)
- Sarvam caller → Circuit breaker: guards API calls (evidence: `src/inference_forge/pipeline/caller.py`)
- Sarvam caller → Sarvam AI API: sends inference requests (evidence: `src/inference_forge/pipeline/caller.py`)
- Ticket deduplication → Redis: reads and writes cache (evidence: `src/inference_forge/pipeline/cache.py`)
- Job state and events → Redis: stores jobs and publishes events (evidence: `src/inference_forge/pipeline/job_store.py`)
- API routes → Job state and events: streams job events (evidence: `README.md`)
- Sarvam caller → Metrics and health: records API metrics (evidence: `src/inference_forge/pipeline/caller.py`)
- Ticket deduplication → Metrics and health: records cache metrics (evidence: `src/inference_forge/pipeline/cache.py`)
- Circuit breaker → Metrics and health: records breaker state (evidence: `src/inference_forge/pipeline/circuit_breaker.py`)
- API routes → Structured logging: logs job activity (evidence: `src/inference_forge/api/routes.py`)
- Request schemas → Runtime settings: uses validation limits (evidence: `src/inference_forge/api/schemas.py`)
