How azizamari/llm_deploy works
Project Type and Purpose
• This is a containerized, backend‐focused “microservices” demo for deploying an LLM behind an HTTP API.
• It uses FastAPI as the HTTP frontend service, Celery + Redis for async task processing, and MLflow for model registry and tracking.
• Docker Compose ties everything together into separate containers.
Main Components
a. FastAPI App
• Container: fastapi_app
• Exposes HTTP endpoints on port 8000
• Reads env vars (REDIS_HOST/PORT, MLFLOW_TRACKING_URI)
b. Celery Worker
• Container: celery_worker
• Connects to Redis for broker & result backend
• Pulls tasks (e.g., LLM inference) from the queue
c. Redis
• Container: redis (defined in docker-compose.yaml)
• Acts as both Celery broker and result backend
d. MLflow Server
• Container: mlflow_setup
• Provides model registry & tracking UI on port 5000
e. External Hugging Face Model Repo
• An external service where the LLM is sourced (auth via HUGGING_FACE token)
Interactions & Data Flows
• Client → (HTTP) → FastAPI
• FastAPI → (publish task) → Redis (broker)
• Celery Worker → (consume task) → Redis
• Celery Worker → (load/infer) →
• Model retrieved from MLflow registry (mlflow_setup)
• If not in registry, downloads from Hugging Face
• Celery Worker → (store result) → Redis (result backend)
• FastAPI → (poll or callback) → Redis → returns inference result to client
Architectural Patterns & Principles
• Microservices: each major function (API, worker, model registry) is its own container
• Asynchronous Task Queue: decouples request from inference work
• Single Responsibility: fastapi_app has no long-running compute, celery_worker has no HTTP endpoints
• Configuration via environment variables (12-factor principles)
Must-Include Elements
• Clear, unique labels on every component