# winstxnhdw/nllb-api architecture

> Architecture diagram of the GitHub repository winstxnhdw/nllb-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/winstxnhdw/nllb-api
- Repository: https://github.com/winstxnhdw/nllb-api

## Overview

##### Project Type and Purpose
- This is a microservice-style REST API for machine translation built on Meta’s NLLB distilled 1.3B 8-bit quantised model.
- Implements language detection and translation endpoints, served via FastAPI/uvicorn.
- Provides a Rust client crate (and Python packaging wrapper) for easy integration.

##### File Structure
- Top‐level artifacts:
- Dockerfiles & Makefile for container build and dev workflow
- `client/` folder: Rust crate (with Cargo.toml, build.rs) plus Python packaging metadata
- `server/` folder: Python FastAPI application with subfolders for API endpoints, feature logic, guards, lifespan hooks, schemas, utils, and type definitions
- `tests/`: pytest tests for API and utils
- Lifespan hooks load models and register the service in Consul at startup.
- CI/CD workflows (GitHub Actions) handle building, testing, CUDA builds, linting, and deployment to Hugging Face Spaces.

##### README for Additional Insights
- Tech stack: Python (FastAPI), Rust (client), CTranslate2 (inference engine), Hugging Face Hub (model download), Consul (service discovery), Docker.
- Exposes endpoints:
- `/api/v4/language` → language detection
- `/api/v4/translator` (+ `/stream`) → translation (SSE for streaming)
- Environment-driven config for ports, CORS, threading (`OMP_NUM_THREADS`, `TRANSLATOR_THREADS`), worker count, and optional CUDA support.
- CI/CD pushes container images to GHCR and auto-deploys to Hugging Face Spaces.

##### Main Components & Relationships
a. Client Library
- Rust crate + Python wrapper exposing `AsyncTranslatorClient`
- Makes HTTP(S) requests to the API server
b. API Server (Python)
- FastAPI app (`server/app.py`)
- API layer (`server/api/v4/...`)
- Feature/business logic (`server/features/...`)
- Lifespan hooks (`server/lifespans/...`): load models, register with Consul
- Guards (`server/guards/...`): secret-based access control
- Schemas (`server/schemas/v1/...`): Pydantic request/response models
- Utilities (`server/utils/...`): downloading from HF, internet check
- Typings & typedefs for CTranslate2, tokenizers, FastText bindings
c. Inference Engine
- CTranslate2 native library invoked via Python bindings
d. Model Repository
- Hugging Face Hub for downloading the quantised NLLB model
e. External Services
- Consul for service registration/discovery
- (Optional) GPU/CUDA if built with the `USE_CUDA` flag
f. Infrastructure & CI/CD
- Docker build (CPU vs CUDA)
- GitHub Actions workflows for build/test/deploy
- Container Registry (GHCR) → Hugging Face Spaces

##### Data Flow & Interaction
- Client → HTTP request → API Server
- Server startup → download model from HF Hub → load into CTranslate2
- Request enters API layer → feature logic → CTranslate2 inference → response
- Lifespan hook registers service address in Consul
- CI/CD pipeline → Docker image → deploy to Spaces

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Clients & Services
    subgraph "External"
        direction TB
        Client["Rust/Python Client"]:::external
        HFHub["Hugging Face Hub"]:::external
        Consul["Consul Service Discovery"]:::external
    end

    %% Container Boundary with Environment Variables
    subgraph "Docker Container\n(env: OMP_NUM_THREADS, TRANSLATOR_THREADS, USE_CUDA)" 
        direction TB
        %% FastAPI Server Components
        subgraph "FastAPI Server" 
            direction TB
            App["FastAPI App\n(server/app.py)"]:::internal
            Config["Configuration Module\n(server/config.py)"]:::internal

            subgraph "Lifespans"
                direction TB
                LoadLangModel["Load Language Detection Model\n(server/lifespans/load_language_detection_model.py)"]:::internal
                LoadTransModel["Load Translator Model\n(server/lifespans/load_translator_model.py)"]:::internal
                ConsulReg["Consul Registration\n(server/lifespans/consul_register.py)"]:::internal
            end

            subgraph "API Layer"
                direction TB
                Health["Health Endpoint\n(server/api/health.py)"]:::internal
                LangDetectAPI["/api/v4/language\n(server/api/v4/language.py)"]:::internal
                TranslateAPI["/api/v4/translator\n(server/api/v4/translator.py)"]:::internal
            end

            subgraph "Feature Logic"
                direction TB
                LangDetector["Language Detector\n(server/features/language_detector.py)"]:::internal
                Translator["Translator\n(server/features/translator.py)"]:::internal
            end

            subgraph "Guards & Utils"
                direction TB
                SecretGuard["Secret Guard\n(server/guards/secret.py)"]:::internal
                InternetUtil["Internet Check\n(server/utils/has_internet_access.py)"]:::internal
                HFDownload["HF Download Helpers\n(server/utils/huggingface_download.py)"]:::internal
                HFFileDL["HF File Download Helpers\n(server/utils/huggingface_file_download.py)"]:::internal
            end

            subgraph "Schemas & TypeDefs"
                direction TB
                LanguageSchema["Language Schema\n(server/schemas/v1/language.py)"]:::internal
                TokensSchema["Tokens Schema\n(server/schemas/v1/tokens.py)"]:::internal
                TranslationSchema["Translation Schema\n(server/schemas/v1/translation.py)"]:::internal
                TranslatedSchema["Translated Output Schema\n(server/schemas/v1/translated.py)"]:::internal
                ComputeTypes["Compute Types\n(server/typedefs/compute_types.py)"]:::internal
                ConfidenceTypes["Confidence Types\n(server/typedefs/confidence.py)"]:::internal
                Devices["Device Enums\n(server/typedefs/devices.py)"]:::internal
                Languages["Language Enums\n(server/typedefs/language.py)"]:::internal
                StateTypes["State Types\n(server/typedefs/state.py)"]:::internal
                TypingsCT2["CTranslate2 Typings\n(server/typings/ctranslate2/__init__.pyi)"]:::internal
                TypingsFT["FastText Typings\n(server/typings/fasttext_pybind/__init__.pyi)"]:::internal
                TypingsTok["Tokenizers Typings\n(server/typings/tokenizers/__init__.pyi)"]:::internal
            end
        end

        CTranslate2["CTranslate2 Engine"]:::infra
        DockerDef["Container Definitions\n(Dockerfile, Dockerfile.build, .dockerignore, Makefile)"]:::infra
        CI_CD["CI/CD Workflows\n(.github/workflows)"]:::infra
    end

    %% CI/CD Pipeline
    CI_CD -->|build/test| DockerDef
    DockerDef -->|push| GHCR["GitHub Container Registry"]:::infra
    GHCR -->|deploy| Spaces["Hugging Face Spaces"]:::infra

    %% Data & Control Flows
    Client -->|HTTP Request| App
    App -->|uses| Config
    App -->|startup| LoadLangModel
    App -->|startup| LoadTransModel
    App -->|startup| ConsulReg
    ConsulReg -->|registers| Consul
    App -->|calls| Health
    App -->|calls| LangDetectAPI
    LangDetectAPI -->|invokes| LangDetector
    App -->|calls| TranslateAPI
    TranslateAPI -->|invokes| Translator
    LangDetector -->|requests| CTranslate2
    Translator -->|requests| CTranslate2
    LoadLangModel -->|downloads model| HFHub
    LoadTransModel -->|downloads model| HFHub
    App -->|uses| SecretGuard
    App -->|uses| InternetUtil
    App -->|uses| HFDownload
    App -->|uses| HFFileDL
    App -->|defines| LanguageSchema
    App -->|defines| TokensSchema
    App -->|defines| TranslationSchema
    App -->|defines| TranslatedSchema
    App -->|defines| ComputeTypes
    App -->|defines| ConfidenceTypes
    App -->|defines| Devices
    App -->|defines| Languages
    App -->|defines| StateTypes
    App -->|defines| TypingsCT2
    App -->|defines| TypingsFT
    App -->|defines| TypingsTok

    %% Styling
    classDef external fill:#ffe4e1,stroke:#cc0000;
    classDef internal fill:#e0f7fa,stroke:#006064;
    classDef infra fill:#f1f8e9,stroke:#33691e;

    %% Click Events
    click Client "https://github.com/winstxnhdw/nllb-api/tree/main/client/"
    click App "https://github.com/winstxnhdw/nllb-api/blob/main/server/app.py"
    click Config "https://github.com/winstxnhdw/nllb-api/blob/main/server/config.py"
    click Health "https://github.com/winstxnhdw/nllb-api/blob/main/server/api/health.py"
    click LangDetectAPI "https://github.com/winstxnhdw/nllb-api/blob/main/server/api/v4/language.py"
    click TranslateAPI "https://github.com/winstxnhdw/nllb-api/blob/main/server/api/v4/translator.py"
    click LangDetector "https://github.com/winstxnhdw/nllb-api/blob/main/server/features/language_detector.py"
    click Translator "https://github.com/winstxnhdw/nllb-api/blob/main/server/features/translator.py"
    click ConsulReg "https://github.com/winstxnhdw/nllb-api/blob/main/server/lifespans/consul_register.py"
    click LoadLangModel "https://github.com/winstxnhdw/nllb-api/blob/main/server/lifespans/load_language_detection_model.py"
    click LoadTransModel "https://github.com/winstxnhdw/nllb-api/blob/main/server/lifespans/load_translator_model.py"
    click SecretGuard "https://github.com/winstxnhdw/nllb-api/blob/main/server/guards/secret.py"
    click LanguageSchema "https://github.com/winstxnhdw/nllb-api/blob/main/server/schemas/v1/language.py"
    click TokensSchema "https://github.com/winstxnhdw/nllb-api/blob/main/server/schemas/v1/tokens.py"
    click TranslationSchema "https://github.com/winstxnhdw/nllb-api/blob/main/server/schemas/v1/translation.py"
    click TranslatedSchema "https://github.com/winstxnhdw/nllb-api/blob/main/server/schemas/v1/translated.py"
    click ComputeTypes "https://github.com/winstxnhdw/nllb-api/blob/main/server/typedefs/compute_types.py"
    click ConfidenceTypes "https://github.com/winstxnhdw/nllb-api/blob/main/server/typedefs/confidence.py"
    click Devices "https://github.com/winstxnhdw/nllb-api/blob/main/server/typedefs/devices.py"
    click Languages "https://github.com/winstxnhdw/nllb-api/blob/main/server/typedefs/language.py"
    click StateTypes "https://github.com/winstxnhdw/nllb-api/blob/main/server/typedefs/state.py"
    click TypingsCT2 "https://github.com/winstxnhdw/nllb-api/blob/main/server/typings/ctranslate2/__init__.pyi"
    click TypingsFT "https://github.com/winstxnhdw/nllb-api/blob/main/server/typings/fasttext_pybind/__init__.pyi"
    click TypingsTok "https://github.com/winstxnhdw/nllb-api/blob/main/server/typings/tokenizers/__init__.pyi"
    click InternetUtil "https://github.com/winstxnhdw/nllb-api/blob/main/server/utils/has_internet_access.py"
    click HFDownload "https://github.com/winstxnhdw/nllb-api/blob/main/server/utils/huggingface_download.py"
    click HFFileDL "https://github.com/winstxnhdw/nllb-api/blob/main/server/utils/huggingface_file_download.py"
    click DockerDef "https://github.com/winstxnhdw/nllb-api/tree/main/Dockerfile"
    click CI_CD "https://github.com/winstxnhdw/nllb-api/tree/main/.github/workflows"
    click GHCR "https://github.com/winstxnhdw/nllb-api/tree/main/client/"
    click Spaces "https://github.com/winstxnhdw/nllb-api/tree/main/tests/"
```
