# danielrapi/-mit_global_hackathon-_beaverfeaver architecture

> Architecture diagram of the GitHub repository danielrapi/-mit_global_hackathon-_beaverfeaver, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/danielrapi/-mit_global_hackathon-_beaverfeaver
- Repository: https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver

## Overview

##### Project Type and Purpose
- Type: Full-stack web application with embedded ML/AI pipeline.

##### File Structure
- Top-level directories:
- _frontend: Next.js/React TypeScript UI
- backend: Python FastAPI server + ML pipeline
- __testing__: unit/integration tests for CV and image utilities
- backend/patchcore, backend/utils, backend/interaction: ML model, helper utilities, and storage of inference outputs
- inference_outputs, uploads, results: on-disk storage of images, JSON summaries, pickles
- Key config files: Procfile (deployment), requirements.txt (Python deps), package.json/tsconfig.json/pnpm-lock (frontend deps).

A. User & Client
- Web browser (runs Next.js app)
B. Frontend Layer (_frontend)
- UI components: upload form, map (Leaflet), image modal, sliders
- API routes (/api/embeddings, any custom routes)
- Vector search & data utilities
C. API Gateway / Edge (Next.js serverless functions)
- /api/embeddings → forwards to backend
D. Backend API (FastAPI in backend/app.py)
- Endpoints: image upload, inference trigger, result retrieval
E. Processing Pipeline (backend/process_pipeline.py)
- Orchestrates steps: EXIF extraction, anomaly detection, LLM analysis, embedding generation
F. Anomaly Detection Engine (backend/patchcore)
- Patchcore model (Anomalib implementation)
- Weights stored under backend/patchcore/drone/v25/...
G. EXIF & Inference Utilities (backend/utils)
- exif_utils.py → GPS extraction or placeholder generation
- inference_utils.py → segmentation mask & heatmap creation
- llm_utils.py → calls to OpenAI GPT-4o-mini
H. Vector Embedding & Search
- Creates embeddings of LLM annotations
- Serves them via vector-search.ts (frontend) and corresponding backend functions (if any)
I. Storage
- File storage (local directories or S3): uploads/, inference_outputs/, results/
- Pickles, JSON summaries, CSV of predictions
J. External Service
- OpenAI API (for LLM analysis)
K. Database (optional/placeholder)

##### Relationships & Data Flow
1. User uploads image in browser → Next.js UI
2. Frontend calls Next.js API route → forwards POST to FastAPI upload endpoint
3. FastAPI writes raw image to uploads/
4. FastAPI invokes Processing Pipeline
a. EXIF extraction → get coordinates
b. Anomaly detection (Patchcore) → produce masks, heatmap
c. LLM analysis (via OpenAI) → textual interpretation
d. Embedding generation → vector representation
e. Write outputs to inference_outputs/ (images, JSON, pickles), results/
5. FastAPI returns summary JSON + URLs to frontend
6. Frontend fetches JSON, overlays heatmap and markers on Leaflet map
7. Frontend may query embeddings via vector-search for related anomalies

##### Architectural Patterns & Design Principles
- Layered architecture: UI → API → Orchestration → ML engines → Storage
- Separation of concerns:
- Frontend purely UI & API client
- Backend API only handles HTTP & orchestration
- Pipeline modules each focus on one domain (EXIF, ML, LLM, embedding)
- Modular ML: Patchcore and LLM as interchangeable services
- File-based storage for reproducibility; could evolve into object storage

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User & Client
    subgraph "User & Client"
        U["Web Browser"]:::frontend
    end

    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        UI["Next.js UI"]:::frontend
        API_GW["API Gateway<br/>(/api/embeddings)"]:::api
        DS["Data & Vector Search Utilities"]:::frontend
        OAI_F["OpenAI Client"]:::frontend
    end

    %% Backend API & Orchestration
    subgraph "Backend Layer"
        direction TB
        FASTAPI["FastAPI Server"]:::api
        PP["Processing Pipeline"]:::api

        subgraph "Pipeline Modules"
            direction TB
            EXIF["EXIF Utilities"]:::ml
            PATCHCORE["Anomaly Detection Engine<br/>(Patchcore)"]:::ml
            LLM_B["LLM Utilities"]:::ml
            EMB["Embedding Generation"]:::ml
        end
    end

    %% External Service
    subgraph "External Services"
        OAI["OpenAI API"]:::external
    end

    %% Storage
    subgraph "Storage"
        direction TB
        UPLOADS["Upload Storage"]:::storage
        INF_OUT["Inference Outputs"]:::storage
        RESULTS["Results Storage"]:::storage
    end

    %% Relationships
    U -->|"Upload Image"| UI
    UI -->|"POST /api/embeddings"| API_GW
    API_GW -->|"Forward Request"| FASTAPI
    FASTAPI -->|"Save Image"| UPLOADS
    FASTAPI -->|"Trigger Pipeline"| PP

    PP -->|"1. EXIF Extraction"| EXIF
    PP -->|"2. Anomaly Detection"| PATCHCORE
    PP -->|"3. LLM Analysis"| LLM_B
    PP -->|"4. Embedding Generation"| EMB

    EXIF -->|Writes| INF_OUT
    PATCHCORE -->|Writes| INF_OUT
    LLM_B -->|Writes| INF_OUT
    EMB -->|Writes| RESULTS

    LLM_B -->|"Call"| OAI

    FASTAPI -->|"Return JSON & URLs"| UI
    UI -->|"Overlay Data"| DS

    %% Click Events
    click UI "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/tree/main/_frontend/app/components"
    click API_GW "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/_frontend/app/api/embeddings/route.ts"
    click OAI_F "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/_frontend/app/lib/openai.ts"
    click DS "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/_frontend/app/data/vector-search.ts"
    click DS "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/_frontend/app/data/vector-utils.ts"
    click FASTAPI "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/backend/app.py"
    click PP "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/backend/process_pipeline.py"
    click PATCHCORE "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/tree/main/backend/patchcore"
    click EXIF "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/backend/utils/exif_utils.py"
    click LLM_B "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/backend/utils/llm_utils.py"
    click INF_OUT "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/tree/main/inference_outputs/"
    click UPLOADS "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/tree/main/uploads/"
    click RESULTS "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/tree/main/results/"
    click PATCHCORE "https://github.com/danielrapi/-mit_global_hackathon-_beaverfeaver/blob/main/backend/patchcore/drone/v25/weights/lightning/model.ckpt"

    %% Styles
    classDef frontend fill:#add8e6,stroke:#333,stroke-width:1px
    classDef api fill:#90ee90,stroke:#333,stroke-width:1px
    classDef ml fill:#ffa500,stroke:#333,stroke-width:1px
    classDef external fill:#dda0dd,stroke:#333,stroke-width:1px
    classDef storage fill:#d3d3d3,stroke:#333,stroke-width:1px
```
