# artlensteam/backendhackudc25 architecture

> Architecture diagram of the GitHub repository artlensteam/backendhackudc25, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/artlensteam/backendhackudc25
- Repository: https://github.com/artlensteam/backendhackudc25

## Overview

##### Project Overview
- ArtLens is an interactive museum guide that enriches the visitor experience by recognizing artworks (using a CLIP model), generating rich, contextualized descriptions (using a modified LLama 3.2 model), searching similar artworks via embedding retrieval with Milvus, and converting text descriptions into audio using gTTS.
- The project was developed as part of HackUDC 2025 and is structured into separate repositories for the frontend and backend, with the current repository containing the backend service and dataset.

##### Key Components
a. Frontend (Mobile Application):
- Although hosted in a separate repository, represent it as the user’s mobile app interface.
- It handles camera capture for artwork recognition and displays personalized guides/audio to the user.
b. Backend API:
- Represented by the “app” directory. It includes:
- Main application entry point (app/main.py)
- REST routers (app/routers/audio.py and app/routers/search.py) for handling audio generation and search requests.
- Business logic and service layers (app/services) that encapsulate configuration for Milvus, search services, and style configuration.
- Logger and general app configuration (app/config.py, app/core/logger.py).
- Models representing data (app/models/models.py).
- Unit tests (app/tests/) to show there are testing measures in place.
c. AI and Processing Services:
- Artwork Recognition: Using the CLIP model (and possibly other AI components) to analyze imagery.
- Description Generation: Powered by the LLama 3.2 model with modifications.
- Text-to-Speech (TTS): Using gTTS to produce audio guides.
d. External Dependencies & Infrastructure:
- Milvus: A vector database for embedding retrieval; this is run as an independent service (started with docker-compose).
- Ollama: Mentioned in the Docker Compose setup for services.
- Dataset: The “dataset” folder holds art images along with JSON metadata, which is used by the system to provide context and reference data for artworks.
- Docker Services: The docker-compose.yaml file indicates external containerized services are used to support Milvus and potentially Ollama.

##### Relationships and Data Flow
- User Interaction: The mobile frontend sends requests (e.g., for artwork recognition or to retrieve an audio guide) to the backend API.
- Routing and Service Invocation:
- The REST API endpoints (audio, search routers) in the backend receive and process user requests.
- For artwork recognition, the backend integrates with the CLIP model (this component is implicit but central to the artwork identification workflow).
- Once recognized, the backend communicates with the LLama-based description generator to produce enriched text.
- The generated text is then passed to the TTS service (gTTS) for audio conversion.
- Embedding Retrieval:
- When personalized recommendations or similar artwork search is needed, the backend uses Milvus (running in its container via Docker Compose) for fast embedding lookups.
- Data Store:
- The dataset directory serves as the source of artwork images and their metadata, which might be ingested or referenced by the backend service.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clear division between front-end (presentation layer) and backend business logic/services.
- Modular Structure: The “app” directory is organized into core, models, routers, and services, which supports a microservices-inspired modular design (even if running within a single backend project).
- Containerization: External services like Milvus and Ollama are containerized, highlighting the emphasis on isolation and scalability.
- Externalization of AI/ML Components: The use of specialized AI models (CLIP, LLama, and gTTS) indicates a design focused on leveraging state-of-the-art models as separate components or services.

##### Final Recommendations
- That every component is drawn with clarity – avoid overlaps and ambiguities.
- External interactions with Docker Compose-managed services.
- Keep in mind that while the backend repository is the focus here, mention that the frontend is an integral component (with reference to its separate repository) to capture the full stack picture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Mobile Frontend"
        UI["Mobile App Interface"]:::frontend
    end

    subgraph "Backend API"
        MAIN["Main Application (main.py)"]:::backend
        CFG["Configuration & Logger (config.py, logger)"]:::backend
        RT_AUD["Audio Router (audio.py)"]:::backend
        RT_SEARCH["Search Router (search.py)"]:::backend
        subgraph "Services Layer"
            MIL_CONF["Milvus Config (milvus_config.py)"]:::services
            SEARCH_SVC["Search Service (search_service.py)"]:::services
            STYLE_CFG["Style Config (style_config.py)"]:::services
        end
        MODELS["Data Models (models.py)"]:::backend
        TESTS["Unit Tests (tests/)"]:::backend
    end

    subgraph "AI & Processing Services"
        CLIP["Artwork Recognition (CLIP)"]:::ai
        LLAMA["Description Generation (LLama 3.2)"]:::ai
        TTS["Text-To-Speech (gTTS)"]:::ai
    end

    subgraph "External Services & Containers"
        MILVUS["Milvus Vector DB"]:::external
        OLLAMA["Ollama Service"]:::external
    end

    subgraph "Dataset Repository"
        DATA["Artwork Images & Metadata (dataset)"]:::dataset
    end

    %% Data Flow
    UI -->|"sendsRequest"| RT_AUD
    UI -->|"sendsRequest"| RT_SEARCH

    MAIN --> CFG
    MAIN --> RT_AUD
    MAIN --> RT_SEARCH
    MAIN --> MIL_CONF
    MAIN --> MODELS
    MAIN --> TESTS
    MAIN -->|"ingestsData"| DATA

    RT_AUD -->|"invokes"| TTS
    RT_AUD -->|"invokes"| LLAMA
    RT_SEARCH -->|"invokes"| SEARCH_SVC
    SEARCH_SVC -->|"queries"| MILVUS

    MIL_CONF ---|"configures"| MILVUS
    STYLE_CFG ---|"definesStyle"| LLAMA

    CLIP ---|"recognizesArt"| LLAMA
    LLAMA ---|"generatesDesc"| TTS

    RT_SEARCH ---|"utilizes"| OLLAMA

    %% Click Events
    click MAIN "https://github.com/artlensteam/backendhackudc25/blob/main/app/main.py"
    click CFG "https://github.com/artlensteam/backendhackudc25/blob/main/app/config.py"
    click CFG "https://github.com/artlensteam/backendhackudc25/blob/main/app/core/logger.py"
    click RT_AUD "https://github.com/artlensteam/backendhackudc25/blob/main/app/routers/audio.py"
    click RT_SEARCH "https://github.com/artlensteam/backendhackudc25/blob/main/app/routers/search.py"
    click MIL_CONF "https://github.com/artlensteam/backendhackudc25/blob/main/app/services/milvus_config.py"
    click SEARCH_SVC "https://github.com/artlensteam/backendhackudc25/blob/main/app/services/search_service.py"
    click STYLE_CFG "https://github.com/artlensteam/backendhackudc25/blob/main/app/services/style_config.py"
    click MODELS "https://github.com/artlensteam/backendhackudc25/blob/main/app/models/models.py"
    click TESTS "https://github.com/artlensteam/backendhackudc25/tree/main/app/tests/"
    click DATA "https://github.com/artlensteam/backendhackudc25/tree/main/dataset/"

    %% Styles
    classDef frontend fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef backend fill:#ABEBC6,stroke:#239B56,stroke-width:2px;
    classDef services fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef ai fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef external fill:#D7BDE2,stroke:#76448A,stroke-width:2px;
    classDef dataset fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
```
