How mounikamandava011/batch-inference-engine works
Batch Inference Engine accepts local JSON prompt batches as asynchronous jobs, processes each prompt through bounded workers and a configurable inference provider, persists progress and results, then exposes status and ordered streaming downloads. The graph follows the sampled routes, coordinator, pipeline, executor, and repository calls; major documented capabilities such as backpressure, retries, SQLite durability, and provider pacing are represented. Provider choice is wired at startup. Other than README-described behavior, unsampled modules are not used to infer additional call relationships.
Main entry points: main.py App setup · routes_jobs.py Job endpoints · job_repository.py Job repository · coordinator.py Job coordinator
Components 17
REST API
- App setup
app/main.py - Job endpoints
app/api/routes_jobs.py - Health endpoint
app/api/routes_ops.py - API schemas
app/api/schemas.py - Input path guard
app/core/paths.py - Runtime settings
app/core/config.py
Job Processing
- Job coordinator
app/services/coordinator.py - Stream pipeline
app/services/pipeline.py - Job and item models
app/domain/models.py
Inference
- Inference executor
app/services/executor.py - Request pacing
app/services/rate_limit.py - Provider contract
app/inference/base.py - DigitalOcean client
app/inference/digitalocean.py - Fake client
app/inference/fake.py
Persistence
- Job repository
app/repositories/job_repository.py - SQLite database
app/repositories/db.py
Other components
- API caller
Connections 26
- API caller → Job endpoints: submits, checks, downloads
README.md - API caller → Health endpoint: checks health
README.md - App setup → Job endpoints: registers routes
app/main.py - App setup → Health endpoint: registers routes
app/main.py - Job endpoints → API schemas: validates and responds
app/api/routes_jobs.py - Job endpoints → Input path guard: resolves input
app/api/routes_jobs.py - Job endpoints → Job repository: creates and reads jobs
app/api/routes_jobs.py - Job endpoints → Job coordinator: starts job
app/api/routes_jobs.py - App setup → Runtime settings: loads configuration
app/main.py - App setup → SQLite database: connects database
app/main.py - App setup → Job repository: creates repository
app/main.py - App setup → Job coordinator: configures coordinator
app/main.py - App setup → DigitalOcean client: selects provider
app/main.py - App setup → Fake client: selects provider
app/main.py - Job coordinator → Job repository: updates job state
app/services/coordinator.py - Job coordinator → Stream pipeline: starts producer, workers, writer
app/services/coordinator.py - Job coordinator → Inference executor: shares with workers
app/services/coordinator.py - Stream pipeline → Job repository: records ingestion and results
app/services/pipeline.py - Stream pipeline → Job and item models: builds work and results
app/services/pipeline.py - Inference executor → Provider contract: calls completion
app/services/executor.py - Inference executor → Request pacing: paces requests
app/services/executor.py - Inference executor → Job and item models: returns item result
app/services/executor.py - DigitalOcean client → Provider contract: implements contract
app/inference/digitalocean.py - Job repository → SQLite database: reads and writes
- Job endpoints → Job repository: pages ordered results
app/api/routes_jobs.py - Job repository → Job and item models: maps records
app/repositories/job_repository.py