# zkawfanx/atlantis architecture

> Architecture diagram of the GitHub repository zkawfanx/atlantis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zkawfanx/atlantis
- Repository: https://github.com/zkawfanx/atlantis

## Overview

##### Project Type and Purpose
- This is a research‐oriented pipeline for underwater depth estimation and stylization, combining depth prediction (MiDaS), image captioning (LAVIS/BLIP2), and ControlNet training/inference on Stable Diffusion.
- It implements the CVPR2024 “Atlantis” framework: prepare {Underwater, Depth, Text} triplets → train a Depth2Underwater ControlNet → generate novel underwater scenes from terrestrial depths.

##### File Structure Analysis
- Top‐level:
- `.gitmodules`: submodules for MiDaS and Diffusers
- `src`: core scripts and vendored libraries
- Under `src`:
- `MiDaS/`: Depth estimation codebase (submodule)
- `diffusers/`: Hugging Face Diffusers (submodule)
- `blip2_caption.py`: text caption generation using LAVIS/BLIP2
- `triplets.py`: script to load and format {underwater, depth, text} dataset
- `inference.py`: uses trained ControlNet + Stable Diffusion to generate images
- No monolithic “training script” here; ControlNet training is done via the Diffusers examples folder referenced in README.

##### Key Architectural Insights from README
- Three sequential stages:
A. Triplet preparation (Depth via MiDaS + Text via BLIP2)
B. ControlNet training (Diffusers + accelerate + stable-diffusion-v1-5)
C. Inference/generation (trained ControlNet + input terrestrial depths)
- Heavy reliance on external frameworks: PyTorch, accelerate, cv2.

##### Main Components
- Data Preparation
- Input: Raw underwater images, terrestrial images
- Subcomponents: MiDaS depth estimator, BLIP2 captioner, triplet formatter
- Model Training
- Input: Prepared triplets
- Subcomponents: Hugging Face Diffusers ControlNet training script, Stable Diffusion v1.5 backbone, accelerate scheduler
- Output: Depth2Underwater ControlNet weights
- Inference
- Input: Terrestrial depth maps + ControlNet weights
- Subcomponents: inference.py, ControlNet + SD1.5 pipeline
- Output: Generated underwater images (“Atlantis”)

##### Component Interactions and Data Flow
- Raw Underwater → MiDaS → 8-bit Depth maps
- Raw Underwater → BLIP2 → Text descriptions
- Depth + Underwater + Text → triplets.py → Dataset structure
- Dataset → Diffusers/train_controlnet.py → ControlNet model
- Terrestrial Depths + ControlNet → inference.py → Stylized Underwater Outputs

##### Architectural Patterns and Principles
- Pipeline pattern: clear separation of data prep, training, inference
- Modular integration of third-party tools as submodules (MiDaS, Diffusers)
- Batch processing and GPU acceleration via accelerate and PyTorch

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Preparation Layer
    subgraph "Data Preparation"
        direction TB
        RawUW(("Raw Underwater Images")):::data
        Terrestrial(("Terrestrial Images")):::data
        MiDaS["MiDaS Depth Estimator"]:::compute
        BLIP2["BLIP2 Caption Generator"]:::compute
        Triplets["Triplets Formatter"]:::script
        TripletDS[(Prepared Triplets Dataset)]:::data
    end

    %% Training Layer
    subgraph "Model Training"
        direction TB
        TrainModule["Stable Diffusion v1.5 + ControlNet Training"]:::compute
        Weights[(Depth2Underwater ControlNet Weights)]:::data
    end

    %% Inference Layer
    subgraph "Inference"
        direction TB
        InferenceModule["ControlNet Inference Module"]:::script
        Generated((Generated Underwater Images)):::data
    end

    %% Data Flows
    RawUW -->|".png"| MiDaS
    RawUW -->|".png"| BLIP2
    RawUW -->|".png"| Triplets
    MiDaS -->|".npy"| Triplets
    BLIP2 -->|"JSONL"| Triplets
    Triplets -->|"Triplets"| TripletDS

    TripletDS -->|"Dataset"| TrainModule
    TrainModule -->|".ckpt"| Weights

    Terrestrial -->|"Depth Map"| InferenceModule
    Weights -->|"ControlNet Weights"| InferenceModule
    InferenceModule -->|"Underwater PNG"| Generated

    %% Static Assets
    StaticAssets(("Static Assets")):::data

    %% Click Events
    click MiDaS "https://github.com/zkawfanx/atlantis/tree/main/src/MiDaS/"
    click BLIP2 "https://github.com/zkawfanx/atlantis/blob/main/src/blip2_caption.py"
    click Triplets "https://github.com/zkawfanx/atlantis/blob/main/src/triplets.py"
    click TrainModule "https://github.com/zkawfanx/atlantis/tree/main/src/diffusers/"
    click InferenceModule "https://github.com/zkawfanx/atlantis/blob/main/src/inference.py"
    click StaticAssets "https://github.com/zkawfanx/atlantis/tree/main/assets/"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1B4F72,color:#1B2631,stroke-width:1px
    classDef compute fill:#A9DFBF,stroke:#145A32,color:#0B3D02,stroke-width:1px
    classDef script fill:#F9E79F,stroke:#B9770E,color:#784212,stroke-width:1px
```
