# sponteen/high_quality_image_captioner architecture

> Architecture diagram of the GitHub repository sponteen/high_quality_image_captioner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sponteen/high_quality_image_captioner
- Repository: https://github.com/sponteen/high_quality_image_captioner

## Overview

##### Project type and purpose
- Type: A single-purpose microservice (no frontend) exposed via a REST API and CLI.
- Purpose: Accept images, run them through the BLIP large caption model (via Hugging Face), and return text captions.

##### Main architectural pattern
- Clean/hexagonal architecture with “Ports & Adapters.”
- Top-level adapters:
– API adapter (FastAPI in src/api)
– CLI adapter (in src/cli)
- Core domain: caption‐generation orchestration (in src/app.py, src/helpers, src/models)
- External service port: Hugging Face model hub (blip-image-captioning-large)

1. Clients
– HTTP client (e.g., curl, Postman, external service)
– CLI user (via bin/debug.sh or src/cli/main.py)
– Jupyter Notebook (for ad-hoc testing in jupyter_notebooks/)
2. API Layer
– FastAPI server (entrypoint is src/app.py)
– Endpoint definitions (src/api/ports/caption_images.py)
3. CLI Layer
– CLI entrypoint (src/cli/main.py)
4. Application Core (Business Logic)
– CaptionManager / orchestrator (in src/app.py)
– Helper functions (src/helpers/loaders.py for loading images/model, src/helpers/generators.py for tokenization or preprocessing)
– Domain models (src/models/images.py defines input/output models)
5. External Dependencies
– Hugging Face Transformers library (loaded via pyproject.toml)
– BLIP large model artifact (downloaded from huggingface.co)
6. Infrastructure & Deployment
– Docker container (Dockerfile + docker-compose.yml)
– Deployment scripts (deploy/entrypoint.sh, SSL certs in deploy/ssl)
– Makefile targets (make docker-run)
– bin scripts for local dev (bin/debug.sh, bin/pack_workspace.sh)

##### Relationships and data flow
- Client → API Layer: HTTP POST /caption-images with image payload
- API Layer → Application Core: Calls CaptionManager.generate(image)
- Application Core → External Model Hub:
– On first call, Loader fetches BLIP large model via HF API
– Model runs inference using Transformers’ pipeline
- Application Core → API Layer: Returns generated caption text
- API Layer → Client: HTTP JSON response
- CLI path: CLI adapter reads local image → calls same CaptionManager → prints caption
- Jupyter path: Notebook uses helpers or direct API calls for experimentation

##### Technologies & frameworks
- Python 3.x (poetry/pyproject.toml)
- FastAPI (REST API)
- Uvicorn / ASGI server
- Hugging Face Transformers (BLIP)
- Docker & docker-compose
- Makefile for orchestration
- Jupyter for demos

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients
    subgraph "Clients" 
        direction TB
        HTTPClient["HTTP Client\n(e.g., curl, Postman)"]:::adpt
        Jupyter["Jupyter Notebook"]:::adpt
        CLIUser["CLI User"]:::adpt
    end

    click Jupyter "https://github.com/sponteen/high_quality_image_captioner/blob/main/jupyter_notebooks/send_image_to_caption.ipynb"
    click CLIUser "https://github.com/sponteen/high_quality_image_captioner/blob/main/bin/debug.sh"

    %% Docker Container Boundary
    subgraph "Docker Container" 
        direction TB
        %% API Layer
        subgraph "API Layer" 
            direction TB
            API["FastAPI Server\nsrc/app.py"]:::adpt
            CaptionPort["API Port\nsrc/api/ports/caption_images.py"]:::adpt
        end

        %% CLI Layer
        subgraph "CLI Layer"
            direction TB
            CLIAdapter["CLI Adapter\nsrc/cli/main.py"]:::adpt
        end

        %% Application Core
        subgraph "Application Core"
            direction TB
            Orchestrator["CaptionManager\n(src/app.py)"]:::core
            Loaders["Image Loader\nsrc/helpers/loaders.py"]:::core
            Generators["Caption Helpers\nsrc/helpers/generators.py"]:::core
            Models["Domain Models\nsrc/models/images.py"]:::core
            Constants1["Image List Constants\nsrc/constants/hiq_images.py"]:::core
            Constants2["Endpoint URLs\nsrc/constants/hiq_images_endpoint.py"]:::core
        end

        Settings["pyproject.toml\n(project settings)"]:::ext

    end

    %% External Dependencies
    HF["Hugging Face Model Hub"]:::ext
    click Settings "https://github.com/sponteen/high_quality_image_captioner/blob/main/pyproject.toml"

    %% Infrastructure & Deployment
    subgraph "Infrastructure & Deployment"
        direction TB
        DockerfileNode["Dockerfile"]:::infra
        Compose["docker-compose.yml"]:::infra
        Entrypoint["deploy/entrypoint.sh"]:::infra
        SSL["SSL Certs\ndeploy/ssl/"]:::infra
        MakefileNode["Makefile"]:::infra
        Pack["bin/pack_workspace.sh"]:::infra
    end

    click DockerfileNode "https://github.com/sponteen/high_quality_image_captioner/tree/main/Dockerfile"
    click Compose "https://github.com/sponteen/high_quality_image_captioner/blob/main/docker-compose.yml"
    click Entrypoint "https://github.com/sponteen/high_quality_image_captioner/blob/main/deploy/entrypoint.sh"
    click SSL "https://github.com/sponteen/high_quality_image_captioner/tree/main/deploy/ssl/"
    click MakefileNode "https://github.com/sponteen/high_quality_image_captioner/tree/main/Makefile"
    click Pack "https://github.com/sponteen/high_quality_image_captioner/blob/main/bin/pack_workspace.sh"

    %% Data Flows
    HTTPClient -->|"HTTP POST image"| API
    API -->|"calls generate()"| Orchestrator
    CaptionPort -->|"routes request"| Orchestrator
    CLIAdapter -->|"invoke generate()"| Orchestrator
    Jupyter -->|"calls helpers or API"| Orchestrator

    Orchestrator -->|"load_model()"| Loaders
    Orchestrator -->|"preprocess() / tokenize"| Generators
    Orchestrator -->|"use schemas"| Models
    Orchestrator -->|"return caption_text"| API
    Orchestrator -->|"print caption"| CLIAdapter

    Loaders -->|"fetch model"| HF
    HF -->|"model artifact"| Loaders
    Orchestrator -->|"infer()"| HF

    API -->|"HTTP JSON response"| HTTPClient
    CLIAdapter -->|"output text"| CLIUser
    Jupyter -->|"display caption"| Jupyter

    %% Click Events for API and Core
    click API "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/app.py"
    click CaptionPort "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/api/ports/caption_images.py"
    click CLIAdapter "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/cli/main.py"
    click Orchestrator "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/app.py"
    click Loaders "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/helpers/loaders.py"
    click Generators "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/helpers/generators.py"
    click Models "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/models/images.py"
    click Constants1 "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/constants/hiq_images.py"
    click Constants2 "https://github.com/sponteen/high_quality_image_captioner/blob/main/src/constants/hiq_images_endpoint.py"

    %% Styles
    classDef adpt fill:#d0e7ff,stroke:#0184c6,color:#000
    classDef core fill:#d4f5d4,stroke:#28a028,color:#000
    classDef ext fill:#e0e0e0,stroke:#888,color:#000,stroke-dasharray: 5 5
    classDef infra fill:#ffe0b2,stroke:#f57c00,color:#000
```
