How ainize-team/llm-fastapi works
Project Type and Purpose
• This is a full‐stack backend service for “Serving Large Language Model Using FastAPI and Celery.”
• It exposes an HTTP/REST API (FastAPI) that enqueues model‐inference jobs into a Celery task queue.
Project Structure
• The root contains configuration files (pyproject.toml, setup.cfg, requirements-dev.txt).
• No separate “frontend” directory—this is purely a backend service.
• Expect folders (not shown here) under the package for:
– The FastAPI app (e.g. app/main.py, app/routes.py)
– Celery configuration and worker code (e.g. app/tasks.py)
– Model‐loading utilities and any cache or storage modules
Core System Components
a. Client / API Consumer
b. FastAPI Application
c. Celery Task Queue
• Broker (e.g. Redis or RabbitMQ)
• Result Backend (e.g. Redis, PostgreSQL)
d. Celery Worker(s)
e. LLM Model Storage & Hosting
• Local filesystem or external object store
f. Optional: Logging/Monitoring service (e.g. Prometheus, Sentry)
Component Interactions (Data Flow)
1. API Client → HTTP Request → FastAPI
2. FastAPI handler
• Validates input and enqueues a Celery task via the broker
• Returns job ID or blocks until result (sync vs. async pattern)
3. Broker holds the task message
4. Celery Worker picks up the task
• Loads the LLM (from local filesystem or object store)
• Runs inference, stores result in the result backend
5. FastAPI polls or fetches result from the result backend
6. FastAPI returns the inference response to the client
Architectural Patterns & Principles
• Asynchronous Task Queue: decouples HTTP request from heavy compute
• Separation of Concerns: API layer vs. inference worker layer
• Scalability: multiple Celery workers for horizontal scaling
• Configuration-as-Code: pyproject.toml, setup.cfg, .pre-commit, CI workflows
Technologies & Frameworks to Annotate
• FastAPI (HTTP server, Pydantic models, uvicorn)
• Celery (task queue, worker processes)
• Broker: Redis or RabbitMQ
• Result Backend: Redis, RDBMS, or custom store
• Python Packaging: pyproject.toml, setup.cfg
• Dev Tools: flake8, pre-commit, GitHub Actions