How aimlenthusiasts/thermofusion works
Project type and purpose
• ThermoFusion is a full-stack, AI-driven web application for super-resolving thermal infrared imagery using high-resolution optical data.
• It combines a PyTorch-based fusion network, a FastAPI inference/preprocessing API, and a React/Vite dashboard for interactive visualization and parameter tuning.
File structure
Top level:
– backend/
• main.py, ai/ (model.pth, training.py), requirements.txt, Dockerfile, apt.txt
• Implements FastAPI endpoints, model loading, preprocessing, metrics, and training scripts
– frontend/
• React+TypeScript app bootstrapped by Vite
• src/ (components/, contexts/, hooks/, lib/, pages/)
• UI library under components/ui, Leaflet/Mapbox integration, file upload, charts, parameter sliders
• Configuration: package.json, tsconfig, tailwind.config, .env.example
Patterns & configs:
– Separation of concerns: “backend” vs. “frontend”
– AI submodule in backend/ai
– No centralized DB—uses filesystem or object storage for inputs/outputs
– Dockerfile for containerized deployment, apt.txt for system deps
Extract architecture insights from README
• Three core stages: Dataset preprocessing & alignment, Model training (FusionSR), Inference API + metrics
• Key features: multi-branch encoder–decoder, physics-informed losses, tile-based GPU acceleration
• API endpoints: /api/v1/preprocess, /api/v1/infer, /api/v1/metrics, /api/v1/status
• Frontend features: map/tile preview, fusion parameter sliders, performance charts, GeoTIFF download
1) Data Sources: Optical HR images, Thermal LR images (inputs)
2) Preprocessing Service (FastAPI module)
– Radiometric calibration
– Geometric co-registration (rasterio/pyproj + SIFT/RANSAC + CNN refinement)
3) Fusion Model Service
– PyTorch FusionSR network (OpticalEncoder, ThermalEncoder, CrossAttentionFusion, Upsampler, Refiner)
– Model repository (model.pth)
4) Metrics Service
– PSNR, SSIM, RMSE calculators
5) Frontend Dashboard (React/Vite)
– File upload UI → calls preprocessing/infer endpoints
– Map/Tile viewer (Leaflet/Mapbox GL)
– Parameter sliders & real-time metric plots
6) Storage (Local FS or Cloud Object Storage)
– Raw inputs, intermediate tiles, output heatmaps, logs
7) Optional Deployment Layer
– Docker container, GPU node, Kubernetes/Cloud functions (for scalability)
• User’s browser → (HTTPS) → Frontend App
• Frontend → (REST) → FastAPI Preprocess endpoint → data → Storage
• Frontend → Fusion endpoint → returns HR thermal map → Storage & Frontend
• Frontend → Metrics endpoint → returns PSNR/SSIM/RMSE → Frontend plots
• Training pipeline (offline): Dataset → training.py → updates model.pth → Storage
c. Architectural patterns & principles:
• Layered architecture: UI layer, API layer, AI/model layer, storage layer
• Modular services: Preprocessing, Inference, Metrics, Training
• Separation of concerns: Clear boundary between data-prep, model logic, and UI
• Stateless API: Each request is self-contained; storage holds state
• Containerization: Dockerfile + system dependencies (apt.txt) for reproducibility
d. Key technologies & frameworks (annotate near each component):
• Frontend: React, TypeScript, Vite, TailwindCSS, Mapbox/Leaflet
• Backend API: FastAPI, Uvicorn, Python 3.10+, pip, Docker
• AI/ML: PyTorch, numpy, rasterio, pyproj, OpenCV (SIFT/ORB), CUDA
• Storage: Local filesystem or cloud buckets (S3/GCS)
• Deployment: Docker, optional Kubernetes or Vercel (for frontend)