How sponteen/high_quality_image_captioner works
Project type and purpose
• Type: A single-purpose microservice (no frontend) exposed via a REST API and CLI.
• Purpose: Accept images, run them through the BLIP large caption model (via Hugging Face), and return text captions.
Main architectural pattern
• Clean/hexagonal architecture with “Ports & Adapters.”
• Top-level adapters:
– API adapter (FastAPI in src/api)
– CLI adapter (in src/cli)
• Core domain: caption‐generation orchestration (in src/app.py, src/helpers, src/models)
• External service port: Hugging Face model hub (blip-image-captioning-large)
1. Clients
– HTTP client (e.g., curl, Postman, external service)
– CLI user (via bin/debug.sh or src/cli/main.py)
– Jupyter Notebook (for ad-hoc testing in jupyter_notebooks/)
2. API Layer
– FastAPI server (entrypoint is src/app.py)
– Endpoint definitions (src/api/ports/caption_images.py)
3. CLI Layer
– CLI entrypoint (src/cli/main.py)
4. Application Core (Business Logic)
– CaptionManager / orchestrator (in src/app.py)
– Helper functions (src/helpers/loaders.py for loading images/model, src/helpers/generators.py for tokenization or preprocessing)
– Domain models (src/models/images.py defines input/output models)
5. External Dependencies
– Hugging Face Transformers library (loaded via pyproject.toml)
– BLIP large model artifact (downloaded from huggingface.co)
6. Infrastructure & Deployment
– Docker container (Dockerfile + docker-compose.yml)
– Deployment scripts (deploy/entrypoint.sh, SSL certs in deploy/ssl)
– Makefile targets (make docker-run)
– bin scripts for local dev (bin/debug.sh, bin/pack_workspace.sh)
Relationships and data flow
• Client → API Layer: HTTP POST /caption-images with image payload
• API Layer → Application Core: Calls CaptionManager.generate(image)
• Application Core → External Model Hub:
– On first call, Loader fetches BLIP large model via HF API
– Model runs inference using Transformers’ pipeline
• Application Core → API Layer: Returns generated caption text
• API Layer → Client: HTTP JSON response
• CLI path: CLI adapter reads local image → calls same CaptionManager → prints caption
• Jupyter path: Notebook uses helpers or direct API calls for experimentation
Technologies & frameworks
• Python 3.x (poetry/pyproject.toml)
• FastAPI (REST API)
• Uvicorn / ASGI server
• Hugging Face Transformers (BLIP)
• Docker & docker-compose
• Makefile for orchestration
• Jupyter for demos