# azizamari/llm_deploy architecture

> Architecture diagram of the GitHub repository azizamari/llm_deploy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azizamari/llm_deploy
- Repository: https://github.com/azizamari/llm_deploy

## Overview

##### Project Type and Purpose
- This is a containerized, backend‐focused “microservices” demo for deploying an LLM behind an HTTP API.
- It uses FastAPI as the HTTP frontend service, Celery + Redis for async task processing, and MLflow for model registry and tracking.
- Docker Compose ties everything together into separate containers.

##### Main Components
a. FastAPI App
- Container: fastapi_app
- Exposes HTTP endpoints on port 8000
- Reads env vars (REDIS_HOST/PORT, MLFLOW_TRACKING_URI)
b. Celery Worker
- Container: celery_worker
- Connects to Redis for broker & result backend
- Pulls tasks (e.g., LLM inference) from the queue
c. Redis
- Container: redis (defined in docker-compose.yaml)
- Acts as both Celery broker and result backend
d. MLflow Server
- Container: mlflow_setup
- Provides model registry & tracking UI on port 5000
e. External Hugging Face Model Repo
- An external service where the LLM is sourced (auth via HUGGING_FACE token)

##### Interactions & Data Flows
- Client → (HTTP) → FastAPI
- FastAPI → (publish task) → Redis (broker)
- Celery Worker → (consume task) → Redis
- Celery Worker → (load/infer) →
- Model retrieved from MLflow registry (mlflow_setup)
- If not in registry, downloads from Hugging Face
- Celery Worker → (store result) → Redis (result backend)
- FastAPI → (poll or callback) → Redis → returns inference result to client

##### Architectural Patterns & Principles
- Microservices: each major function (API, worker, model registry) is its own container
- Asynchronous Task Queue: decouples request from inference work
- Single Responsibility: fastapi_app has no long-running compute, celery_worker has no HTTP endpoints
- Configuration via environment variables (12-factor principles)

##### Must-Include Elements
- Clear, unique labels on every component

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Client
    Client["Client"]:::external

    %% Docker Network Boundary
    subgraph "Docker Network"
        direction TB
        fastapi_app["FastAPI App\n(container: fastapi_app)\nport:8000\nenv: REDIS_HOST/PORT,\nMLFLOW_TRACKING_URI"]:::api
        celery["Celery Worker\n(container: celery_worker)\nenv: REDIS_HOST/PORT"]:::worker
        redis["Redis\n(container: redis)\nport:6379\nbroker & result backend"]:::datastore
        mlflow["MLflow Server\n(container: mlflow_setup)\nport:5000"]:::datastore
        compose["docker-compose.yaml"]:::datastore
    end

    %% Configuration Files
    subgraph "Configuration"
        direction TB
        config_py["config.py"]:::datastore
        env_dev[".env.dev"]:::datastore
        env_prod[".env.prod"]:::datastore
    end

    %% External Hugging Face
    huggingface["Hugging Face Model Repo"]:::external

    %% Interactions
    Client -->|"HTTP POST /infer"| fastapi_app
    fastapi_app -->|"publish task"| redis
    redis -->|"task queued"| celery
    celery -->|"GET /models/latest"| mlflow
    mlflow -->|"model data"| celery
    celery -->|"download if not found"| huggingface
    celery -->|"store result"| redis
    fastapi_app -->|"poll result"| redis
    redis -->|"response data"| fastapi_app

    %% Click Events
    click fastapi_app "https://github.com/azizamari/llm_deploy/tree/main/fastapi_app/"
    click celery "https://github.com/azizamari/llm_deploy/tree/main/celery_worker/"
    click redis "https://github.com/azizamari/llm_deploy/blob/main/docker-compose.yaml"
    click mlflow "https://github.com/azizamari/llm_deploy/tree/main/mlflow_setup/"
    click compose "https://github.com/azizamari/llm_deploy/blob/main/docker-compose.yaml"
    click config_py "https://github.com/azizamari/llm_deploy/blob/main/config.py"
    click env_dev "https://github.com/azizamari/llm_deploy/blob/main/.env.dev"
    click env_prod "https://github.com/azizamari/llm_deploy/blob/main/.env.prod"

    %% Styles
    classDef api fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef worker fill:#90EE90,stroke:#333,stroke-width:1px
    classDef datastore fill:#FFD580,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
```
