# luca-medeiros/lang-segment-anything architecture

> Architecture diagram of the GitHub repository luca-medeiros/lang-segment-anything, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luca-medeiros/lang-segment-anything
- Repository: https://github.com/luca-medeiros/lang-segment-anything

## Overview

##### Project type and purpose
- It can be used as a local Gradio web app (app.py), as a containerized API, or imported as a library in other Python code.

##### File structure
- Top level
– app.py: entry point for Gradio UI
– Dockerfile / .dockerignore: containerize the app with GPU support
– pyproject.toml, requirements.txt: dependency management
– assets/: example inputs and outputs
- lang_sam/ (inference core)
– lang_sam.py: LangSAM class (public API)
– server.py: LitServe/FastAPI wrapper for HTTP endpoints
– models/: wrappers for GroundingDINO (gdino.py), SAM 2.1 (sam.py), plus utils
– utils.py: shared helper functions
- CI & GitHub configuration (.github/, pre-commit, workflows)

##### Extract insights from the README
- Supports batch inference, zero-shot detection, customizable prompts.
- Deployment options: run locally (`python app.py` → Gradio at port 8000), Docker (`docker run --gpus all -p 8000:8000`), or serve via Lightning AI LitServe (server.py).

##### Main components and their interactions
a. Client/UI layer
- Gradio Frontend (app.py)
- External clients using Python library (LangSAM.predict) or HTTP API
b. Service/API layer
- server.py exposes REST endpoints (via LitServe or FastAPI)
c. Inference core
- LangSAM class (lang_sam/lang_sam.py) orchestrates model calls
- Model wrappers in lang_sam/models:
- GroundingDINO detection → text→bbox
- SAM segmentation → mask generation
- Shared utilities (lang_sam/utils.py, models/utils.py)
d. Data / Assets
- Input images (from client uploads or library calls)
e. Infrastructure
- Docker container for GPU inference
- CI pipelines (pre-commit, GitHub Actions)
- Optional LitServe / Lightning AI hosting

##### Architectural patterns and design principles
- Separation of concerns: UI vs. API vs. inference logic
- Adapter pattern: model wrappers abstract GroundingDINO and SAM APIs
- Dependency injection: LangSAM takes models as parameters for easy extension
- Modular packaging: clear boundary between public API (lang_sam/lang_sam.py) and internal modules

- Arrowheads showing request/response flow
- Notes on technologies: PyTorch/TorchVision, Docker, LitServe
- Deployment boundary (container vs. local library)

##### Final advice
- Be exhaustive: break the system into UI, API, orchestration, model adapters, and infra.
- Don’t overthink—more components with clear interactions is better than a vague monolith.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client/UI Layer
    ClientBrowser["User Browser\n(Gradio UI)"]:::ui
    ExternalClient["External Client\n(LangSAM.predict)"]:::ui

    %% UI Component
    GradioUI["Gradio UI\n(app.py)"]:::ui

    %% API Layer
    HTTPAPI["HTTP API\n(lang_sam/server.py)"]:::api

    %% Inference Core
    Orchestrator["LangSAM Orchestrator\n(lang_sam/lang_sam.py)"]:::core
    SharedUtils["Shared Utilities\n(lang_sam/utils.py)"]:::core

    %% Model Adapters
    subgraph "Model Adapters"
        direction TB
        GDINO["GroundingDINO Wrapper\n(lang_sam/models/gdino.py)"]:::model
        SAM["SAM 2.1 Wrapper\n(lang_sam/models/sam.py)"]:::model
        ModelUtils["Model Utilities\n(lang_sam/models/utils.py)"]:::model
    end

    %% Data / Storage
    Assets["Assets Storage\nassets/"]:::storage

    %% Infrastructure & CI
    subgraph "Infrastructure & CI"
        direction TB
        DockerC["Docker Container\n(Dockerfile, .dockerignore)"]:::infra
        DepMgmt["Dependency Management\n(pyproject.toml, requirements.txt)"]:::infra
        PreCommit["CI Pre-commit\n(.pre-commit-config.yaml)"]:::infra
        GHWorkflows["GitHub Actions\n(.github/workflows/)"]:::infra
        IssueTemplates["Issue Templates\n(.github/ISSUE_TEMPLATE/)"]:::infra
        PRTemplate["PR Templates\n(.github/pull_request_template.md)"]:::infra
    end

    %% Data Flows
    ClientBrowser -->|"uploads image + prompt"| GradioUI
    ExternalClient -->|"library call\npredict(image,text)"| Orchestrator
    ExternalClient -->|"HTTP request\nimage + prompt"| HTTPAPI

    GradioUI -->|"calls LangSAM.predict"| Orchestrator
    HTTPAPI -->|"calls LangSAM.predict"| Orchestrator

    Orchestrator -->|"use utils"| SharedUtils
    Orchestrator -->|"request bbox"| GDINO
    GDINO -->|"return bounding boxes"| Orchestrator
    Orchestrator -->|"request masks"| SAM
    SAM -->|"return masks"| Orchestrator
    Orchestrator -->|"use model utils"| ModelUtils

    Orchestrator -->|"returns results"| GradioUI
    Orchestrator -->|"returns results"| HTTPAPI
    Orchestrator -->|"returns results"| ExternalClient

    Orchestrator -->|"saves outputs"| Assets

    %% Click Events
    click GradioUI "https://github.com/luca-medeiros/lang-segment-anything/blob/main/app.py"
    click DockerC "https://github.com/luca-medeiros/lang-segment-anything/tree/main/Dockerfile"
    click DockerC "https://github.com/luca-medeiros/lang-segment-anything/blob/main/.dockerignore"
    click DepMgmt "https://github.com/luca-medeiros/lang-segment-anything/blob/main/pyproject.toml"
    click DepMgmt "https://github.com/luca-medeiros/lang-segment-anything/blob/main/requirements.txt"
    click Assets "https://github.com/luca-medeiros/lang-segment-anything/tree/main/assets/"
    click Orchestrator "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/lang_sam.py"
    click HTTPAPI "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/server.py"
    click GDINO "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/models/gdino.py"
    click SAM "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/models/sam.py"
    click ModelUtils "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/models/utils.py"
    click SharedUtils "https://github.com/luca-medeiros/lang-segment-anything/blob/main/lang_sam/utils.py"
    click PreCommit "https://github.com/luca-medeiros/lang-segment-anything/blob/main/.pre-commit-config.yaml"
    click GHWorkflows "https://github.com/luca-medeiros/lang-segment-anything/tree/main/.github/workflows/"
    click IssueTemplates "https://github.com/luca-medeiros/lang-segment-anything/tree/main/.github/ISSUE_TEMPLATE/"
    click PRTemplate "https://github.com/luca-medeiros/lang-segment-anything/blob/main/.github/pull_request_template.md"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef api fill:#90EE90,stroke:#333,stroke-width:1px
    classDef core fill:#FFA500,stroke:#333,stroke-width:1px
    classDef model fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef storage fill:#FFFF99,stroke:#333,stroke-width:1px
    classDef infra fill:#E6E6FA,stroke:#333,stroke-width:1px
```
