# transformerlab/transformerlab-api architecture

> Architecture diagram of the GitHub repository transformerlab/transformerlab-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/transformerlab/transformerlab-api
- Repository: https://github.com/transformerlab/transformerlab-api

## Overview

##### Main Components
- API Server: The core of the system is the API layer (see api.py) which handles incoming requests.
- Routers & Endpoints: Under the transformerlab/routers directory, various endpoints (e.g., for experiments, models, prompts, plugins, training, etc.) are defined. Represent these logical “modules” or “services” that route API calls.
- Plugin System: The directory transformerlab/plugins (and the complementary transformerlab/plugin_sdk) indicates a modular plugin architecture. Include a component that represents the plugin loader/harness and note that these plugins extend functionality (such as evals, training, generation, etc.).
- Data/Models & Database Module: The transformerlab/db.py file and shared models under transformerlab/shared/models show data management. This component represents database interactions and model management.
- Docker Deployment: The docker folder (with separate directories for CPU and GPU) contains Dockerfiles and deployment scripts. Represent these as deployment containers (showing differences for GPU vs. CPU environments), emphasizing that containerization is used to standardize execution.
- Cloud Deployment Scripts: The “scripts/aws” and “scripts/azure” directories (including Terraform files) indicate infrastructure automation for cloud deployments. Show these as components interacting with cloud provider APIs.
- Testing & CI/CD: The .github/workflows, pytest.ini, and test directories indicate automated testing and continuous integration processes.
- Dependency & Environment Management: Note that the project uses Conda (see install.sh), pip requirements (requirements.in, requirements-uv.txt, etc.), and custom setup scripts. This is an external configuration/management aspect that supports the entire system.

- Routers/endpoints that expose various functions.
- The plugin system, showing how the API dynamically loads and uses plugin functionality.
- The database/data storage component, noting read/write operations.
- Separate containers (or nodes) for CPU and GPU deployments. Indicate that both use the API service but with specialized Docker images for GPU-accelerated tasks.
- A bidirectional relationship between deployment scripts/cloud automation and the API server, indicating that changes (e.g., deployments and scaling) are managed via Terraform/Cloud scripts.
- CI/CD pipeline as a separate “integration” layer that automatically tests code changes and triggers deployments.

##### Important Architectural Patterns
- Modular Design: The clear separation between the core API, the routers, and the plugin ecosystem shows an “extension by plugins” pattern.
- Containerization and Infrastructure as Code (IaC): Using Docker (with separate variants for GPU/CPU) and Terraform scripts for cloud (AWS/Azure) is key.
- Environment & Dependency Management: The use of Conda and pip (with explicit handling for GPU vs. non-GPU requirements) ensures reproducibility.

##### Relevant Technologies and Frameworks
- Python (the primary development language)
- Docker (containerization with CPU and GPU variants)
- Conda for environment management
- Terraform automation for cloud deployment (present in the scripts/azure directory)
- CI/CD Tools from GitHub Workflows (Ruff, Pytest, CodeQL)
- The plugin architecture is evident in the structure under transformerlab/plugins and plugin_sdk

##### Elements
- Clear labels for every component (e.g., “API Server,” “Router - Plugins,” “Database/Model Store,” “Docker Container (GPU)”, “Docker Container (CPU)”, “Terraform Cloud Automation”, “CI/CD Pipeline”).
- From clients to the API server.
- From the API server to various routers and plugins.
- Use rounded rectangles for API and microservices.
- Use cubes or database symbols for persistence layers.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        CL["Client Requests"]:::client
    end

    %% Core API Layer
    subgraph "Core API Layer"
        AP["API Server"]:::api
        click AP "https://github.com/transformerlab/transformerlab-api/blob/main/api.py"
    end

    %% Service Modules
    subgraph "Service Modules"
        RE["Routers & Endpoints"]:::service
        click RE "https://github.com/transformerlab/transformerlab-api/tree/main/transformerlab/routers"
        
        DS["Database/Model Store"]:::db
        click DS "https://github.com/transformerlab/transformerlab-api/blob/main/transformerlab/db.py"
        click DS "https://github.com/transformerlab/transformerlab-api/tree/main/transformerlab/shared/models"
        
        subgraph "Plugin System"
            P1["Plugins"]:::plugin
            click P1 "https://github.com/transformerlab/transformerlab-api/tree/main/transformerlab/plugins"
            P2["Plugin SDK"]:::plugin
            click P2 "https://github.com/transformerlab/transformerlab-api/tree/main/transformerlab/plugin_sdk"
        end
    end

    %% Deployment & Automation
    subgraph "Deployment & Automation"
        subgraph "Container Deployment"
            DC_CPU["Docker Container (CPU)"]:::docker
            click DC_CPU "https://github.com/transformerlab/transformerlab-api/tree/main/docker/cpu"
            DC_GPU["Docker Container (GPU)"]:::docker
            click DC_GPU "https://github.com/transformerlab/transformerlab-api/tree/main/docker/gpu/nvidia"
        end
        subgraph "Cloud Automation"
            AWS["Cloud Deployment (AWS)"]:::cloud
            click AWS "https://github.com/transformerlab/transformerlab-api/tree/main/scripts/aws"
            AZ["Cloud Deployment (Azure)"]:::cloud
            click AZ "https://github.com/transformerlab/transformerlab-api/tree/main/scripts/azure"
        end
        CICD["CI/CD Pipeline"]:::cicd
        click CICD "https://github.com/transformerlab/transformerlab-api/tree/main/.github/workflows"
        click CICD "https://github.com/transformerlab/transformerlab-api/blob/main/pytest.ini"
        click CICD "https://github.com/transformerlab/transformerlab-api/tree/main/test"
    end

    %% Environment Management
    subgraph "Environment Management"
        EM1["install.sh"]:::env
        click EM1 "https://github.com/transformerlab/transformerlab-api/blob/main/install.sh"
        EM2["requirements.in"]:::env
        click EM2 "https://github.com/transformerlab/transformerlab-api/blob/main/requirements.in"
        EM3["requirements-gpu.in"]:::env
        click EM3 "https://github.com/transformerlab/transformerlab-api/blob/main/requirements-gpu.in"
        EM4["requirements-uv.txt"]:::env
        click EM4 "https://github.com/transformerlab/transformerlab-api/blob/main/requirements-uv.txt"
        EM5["requirements-no-gpu-uv.txt"]:::env
        click EM5 "https://github.com/transformerlab/transformerlab-api/blob/main/requirements-no-gpu-uv.txt"
        EM6["ruff.toml"]:::env
        click EM6 "https://github.com/transformerlab/transformerlab-api/blob/main/ruff.toml"
    end

    %% Connections
    CL -->|"sends-request"| AP

    AP -->|"routes"| RE
    AP -->|"loads"| P1
    AP -->|"loads"| P2
    AP <-->|"read/write"| DS

    CICD -->|"triggers"| DC_CPU
    CICD -->|"triggers"| DC_GPU

    AZ <-->|"manages"| AP
    AWS <-->|"manages"| AP

    EM1 --- EM2 --- EM3 --- EM4 --- EM5 --- EM6
    EM1 --- AP
    EM1 --- RE
    EM1 --- DS

    %% Styles
    classDef api fill:#f9c,stroke:#333,stroke-width:2px;
    classDef service fill:#ccf,stroke:#333,stroke-width:2px;
    classDef db fill:#cfc,stroke:#333,stroke-width:2px;
    classDef plugin fill:#fcf,stroke:#333,stroke-width:2px;
    classDef docker fill:#9cf,stroke:#333,stroke-width:2px;
    classDef cloud fill:#ff9,stroke:#333,stroke-width:2px;
    classDef cicd fill:#cff,stroke:#333,stroke-width:2px;
    classDef env fill:#fcc,stroke:#333,stroke-width:2px;
    classDef client fill:#eef,stroke:#333,stroke-width:2px;
```
