# yujiaoshi/sat2strpanoramasynthesis architecture

> Architecture diagram of the GitHub repository yujiaoshi/sat2strpanoramasynthesis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yujiaoshi/sat2strpanoramasynthesis
- Repository: https://github.com/yujiaoshi/sat2strpanoramasynthesis

## Overview

##### Project Overview
- This project is a research codebase implementing Geometry-Guided Street-View Panorama Synthesis from satellite imagery. It leverages a differentiable geometry-guided projection (S2SP module) together with a generator network and a VGG-based perceptual loss.
- The core idea is to explicitly model geometric transformations between the satellite view and the intended street-view panorama, enhancing the synthesis process compared to standard generative approaches.

##### Identifying Main Components
- VGG Module:
- Contains files like vgg.py and perceptual_loss.py (and its corresponding test file).
- Implements the VGG-based perceptual loss which is used during training to enforce image realism.

- Geometry Module:
- Files include Geometry.py, projector.py, test_geometry.py, and utils.py.
- Implements the geometric transformations (the S2SP module) that project satellite images into ground-level layout.

- Load Data Module:
- Contains several scripts in load_data for handling different datasets (e.g., CVACT, CVUSA, unaligned or half-size versions).
- This module is responsible for preprocessing and feeding data into the training/testing pipeline.

- Evaluation Metrics:
- The evaluation_metrics directory contains scripts to evaluate model outputs (for example, metrics_tf.py and metrics_tf_cvact.py), along with a shell script to run evaluations.

- Training/Execution Scripts:
- Under script3, several training related components exist (main.py, baseline scripts, model definitions, and shell scripts for running experiments).
- This is where the end-to-end pipeline is orchestrated, invoking data loading, geometric projection, and synthesis generation (using the learned generator network) during training and testing.

##### Relationships and Data Flow
- Data Input: The load_data module ingests satellite images from the CVUSA and CVACT datasets.
- Geometric Projection: The Geometry module processes the data to compute height maps and project the satellite imagery into approximate ground-view representations.
- Image Synthesis: The output of the geometric projection feeds into the synthesis generator (encoded in the script3 models), which then produces the street-view panorama.
- Perceptual Supervision: The VGG module applies a perceptual loss by comparing synthesized images against real street-view panoramas, ensuring high-level feature consistency.
- Evaluation: Once synthesis is complete, the evaluation_metrics module computes quantitative metrics (via TensorFlow-based functions) to help assess performance.
- Training Workflow: The main training loop (in script3/main.py) orchestrates these components, running them in sequence for both training and testing modes as specified in the README.

##### Architectural Patterns and Design Principles
- Modular Design: Each component (data loading, geometric transformation, synthesis network, perceptual loss, evaluation) is encapsulated in its own directory/module, making the architecture modular.
- End-to-End Differentiability: The design emphasizes differentiability so that the S2SP module and generator network are trained together.
- Separation of Concerns: Distinct responsibilities—data ingestion, transformation, image generation, and evaluation—are clearly separated which helps in isolating issues and further extension.

## Diagram (Mermaid)

```mermaid
flowchart TD
    TL["Training Loop (script3/main.py)"]:::synthesis
    DL["Data Loader - CVUSA/CVACT"]:::dataLoader
    GM["Geometry Projection (S2SP)"]:::geometry
    GN["Generator (Street-View Synthesis)"]:::synthesis
    VG["VGG-Based Perceptual Loss"]:::loss
    EM["Evaluation Metrics"]:::evaluation

    TL -->|"start"| DL
    DL -->|"preprocess"| GM
    GM -->|"project"| GN
    GN -->|"evaluate"| EM
    GN -->|"perceptual"| VG
    VG -->|"feedback"| TL

    click TL "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/script3/main.py"
    click DL "https://github.com/yujiaoshi/sat2strpanoramasynthesis/tree/main/load_data/"
    click GM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/geometry/Geometry.py"
    click GM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/geometry/projector.py"
    click GM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/geometry/utils.py"
    click GN "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/script3/model.py"
    click VG "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/VGG/vgg.py"
    click VG "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/VGG/perceptual_loss.py"
    click EM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/evaluation_metrics/metrics_tf.py"
    click EM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/evaluation_metrics/metrics_tf_cvact.py"
    click EM "https://github.com/yujiaoshi/sat2strpanoramasynthesis/blob/main/evaluation_metrics/run.sh"

    classDef dataLoader fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef geometry fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef synthesis fill:#ffeeba,stroke:#856404,stroke-width:2px;
    classDef loss fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef evaluation fill:#f8d7da,stroke:#721c24,stroke-width:2px;
```
