# aimlenthusiasts/thermofusion architecture

> Architecture diagram of the GitHub repository aimlenthusiasts/thermofusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aimlenthusiasts/thermofusion
- Repository: https://github.com/aimlenthusiasts/thermofusion

## Overview

##### Project type and purpose
- ThermoFusion is a full-stack, AI-driven web application for super-resolving thermal infrared imagery using high-resolution optical data.
- It combines a PyTorch-based fusion network, a FastAPI inference/preprocessing API, and a React/Vite dashboard for interactive visualization and parameter tuning.

##### File structure
Top level:
– backend/
- main.py, ai/ (model.pth, training.py), requirements.txt, Dockerfile, apt.txt
- Implements FastAPI endpoints, model loading, preprocessing, metrics, and training scripts
– frontend/
- React+TypeScript app bootstrapped by Vite
- src/ (components/, contexts/, hooks/, lib/, pages/)
- UI library under components/ui, Leaflet/Mapbox integration, file upload, charts, parameter sliders
- Configuration: package.json, tsconfig, tailwind.config, .env.example
Patterns & configs:
– Separation of concerns: “backend” vs. “frontend”
– AI submodule in backend/ai
– No centralized DB—uses filesystem or object storage for inputs/outputs
– Dockerfile for containerized deployment, apt.txt for system deps

##### Extract architecture insights from README
- Three core stages: Dataset preprocessing & alignment, Model training (FusionSR), Inference API + metrics
- Key features: multi-branch encoder–decoder, physics-informed losses, tile-based GPU acceleration
- API endpoints: `/api/v1/preprocess`, `/api/v1/infer`, `/api/v1/metrics`, `/api/v1/status`
- Frontend features: map/tile preview, fusion parameter sliders, performance charts, GeoTIFF download

1) Data Sources: Optical HR images, Thermal LR images (inputs)
2) Preprocessing Service (FastAPI module)
– Radiometric calibration
– Geometric co-registration (rasterio/pyproj + SIFT/RANSAC + CNN refinement)
3) Fusion Model Service
– PyTorch FusionSR network (OpticalEncoder, ThermalEncoder, CrossAttentionFusion, Upsampler, Refiner)
– Model repository (`model.pth`)
4) Metrics Service
– PSNR, SSIM, RMSE calculators
5) Frontend Dashboard (React/Vite)
– File upload UI → calls preprocessing/infer endpoints
– Map/Tile viewer (Leaflet/Mapbox GL)
– Parameter sliders & real-time metric plots
6) Storage (Local FS or Cloud Object Storage)
– Raw inputs, intermediate tiles, output heatmaps, logs
7) Optional Deployment Layer
– Docker container, GPU node, Kubernetes/Cloud functions (for scalability)

- User’s browser → (HTTPS) → Frontend App
- Frontend → (REST) → FastAPI Preprocess endpoint → data → Storage
- Frontend → Fusion endpoint → returns HR thermal map → Storage & Frontend
- Frontend → Metrics endpoint → returns PSNR/SSIM/RMSE → Frontend plots
- Training pipeline (offline): Dataset → training.py → updates model.pth → Storage

c. Architectural patterns & principles:
- Layered architecture: UI layer, API layer, AI/model layer, storage layer
- Modular services: Preprocessing, Inference, Metrics, Training
- Separation of concerns: Clear boundary between data-prep, model logic, and UI
- Stateless API: Each request is self-contained; storage holds state
- Containerization: Dockerfile + system dependencies (apt.txt) for reproducibility

d. Key technologies & frameworks (annotate near each component):
- Frontend: React, TypeScript, Vite, TailwindCSS, Mapbox/Leaflet
- Backend API: FastAPI, Uvicorn, Python 3.10+, pip, Docker
- AI/ML: PyTorch, numpy, rasterio, pyproj, OpenCV (SIFT/ORB), CUDA
- Storage: Local filesystem or cloud buckets (S3/GCS)
- Deployment: Docker, optional Kubernetes or Vercel (for frontend)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User and UI Layer
    User["User's Browser"]:::ui
    Frontend["Frontend Dashboard"]:::ui
    User -->|"HTTPS"| Frontend

    %% API Layer
    subgraph "API Layer"
        Preprocessing["Preprocessing Service<br/>(FastAPI)<br/><i>Endpoints: /api/v1/preprocess</i>"]:::api
        FusionEndpoint["Fusion Inference Service<br/>(FastAPI)<br/><i>Endpoints: /api/v1/infer</i>"]:::api
        Metrics["Metrics Service<br/>(FastAPI)<br/><i>Endpoints: /api/v1/metrics</i>"]:::api
    end

    Frontend -->|"REST"| Preprocessing
    Frontend -->|"REST"| FusionEndpoint
    Frontend -->|"REST"| Metrics

    %% AI/Model Layer
    subgraph "AI/Model Layer"
        FusionModel["PyTorch FusionSR Network<br/>(OpticalEncoder, ThermalEncoder,<br/>CrossAttentionFusion, Upsampler,<br/>Refiner)"]:::model
        ModelStore[(Model Repository<br/>model.pth)]:::model
    end

    Preprocessing -->|"file I/O"| Storage
    Preprocessing -->|"passes tiles"| FusionModel
    FusionEndpoint -->|"loads model"| ModelStore
    FusionEndpoint -->|"calls infer"| FusionModel
    Metrics -->|"calls metrics"| FusionModel

    %% Offline Training
    Training["Offline Training Pipeline"]:::model
    Training -->|"writes model"| ModelStore

    %% Storage Layer
    Storage[(Storage<br/>Local FS / S3/GCS)]:::storage
    FusionEndpoint -->|"writes output"| Storage
    Preprocessing -->|"writes intermediate"| Storage

    %% Deployment Layer
    subgraph "Deployment / Infrastructure"
        GPU["GPU Node / Cluster"]:::infra
        Container["Containerization"]:::infra
    end

    FusionModel --> GPU
    Preprocessing --> Container
    FusionEndpoint --> Container
    Metrics --> Container
    Training --> GPU

    %% Click Events
    click Preprocessing "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/main.py"
    click FusionEndpoint "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/main.py"
    click Metrics "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/main.py"
    click FusionModel "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/ai/training.py"
    click ModelStore "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/ai/model.pth"
    click Container "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/requirements.txt"
    click Container "https://github.com/aimlenthusiasts/thermofusion/blob/main/backend/apt.txt"
    click Container "https://github.com/aimlenthusiasts/thermofusion/tree/main/backend/Dockerfile"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/App.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/main.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/pages/Upload.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/pages/TiffViewer.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/pages/Dashboard.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/components/ThermalVisualizationCard.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/components/ui/slider.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/components/ui/chart.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/lib/api.ts"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/package.json"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/vite.config.ts"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/.env.example"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/contexts/AuthContext.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/components/ProtectedRoute.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/hooks/use-mobile.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/hooks/use-toast.tsx"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/src/lib/utils.ts"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/public/model.glb"
    click Frontend "https://github.com/aimlenthusiasts/thermofusion/blob/main/frontend/index.html"

    %% Styles
    classDef ui fill:#bbdefb,stroke:#1e88e5,color:#0d47a1;
    classDef api fill:#c8e6c9,stroke:#2e7d32,color:#1b5e20;
    classDef model fill:#ffe0b2,stroke:#ef6c00,color:#e65100;
    classDef storage fill:#e0e0e0,stroke:#616161,color:#212121;
    classDef infra fill:#e1bee7,stroke:#8e24aa,color:#4a148c;
```
