# lukashaas/pigeon architecture

> Architecture diagram of the GitHub repository lukashaas/pigeon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lukashaas/pigeon
- Repository: https://github.com/lukashaas/pigeon

## Overview

##### Main Components

a) Data Processing Pipeline:
- Input Layer: Raw images
- Preprocessing Module (from /preprocessing directory)
- Dataset preprocessing
- Geo augmentation
- Embedding generation

b) Core Model Architecture:
- CLIP Embedder (models/clip_embedder.py)
- Super Guessr Model (models/super_guessr.py)
- Proto Refiner (models/proto_refiner.py)
- Custom Layers
- HEDGE layer
- Positional Encoder

c) Dataset Management:
- Multiple dataset types:
- Pretraining datasets
- Finetuning datasets
- Benchmark datasets
- Geocell System (from dataset_creation/geocell)

d) Training System:
- Training loops
- Evaluation metrics
- Model saving/loading

##### Specific Design Elements

Use these visual elements:
- Rounded rectangles: Processing modules
- Hexagons: Model components

- Blue: Data processing components
- Green: Model components
- Yellow: Training components
- Gray: Storage/Data
- Red: Evaluation components

##### Important Details

- Configuration system (config.py) influence on all components
- Multiple dataset paths for different purposes
- Model component interactions
- Training feedback loops
- Evaluation metrics flow

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Data Layer"
        RawImages[("Raw Images")]:::data
        StreetView[("Street View Images")]:::data
        BenchmarkData[("Benchmark Data")]:::data
        click RawImages "https://github.com/LukasHaas/PIGEON/tree/main/data/images_mp_16/"
        click StreetView "https://github.com/LukasHaas/PIGEON/tree/main/data/streetview_cropped/"
        click BenchmarkData "https://github.com/LukasHaas/PIGEON/tree/main/data/benchmarks/"
    end

    subgraph "Processing Layer"
        PreProcess["Dataset Preprocessing"]:::processing
        GeoAug["Geo Augmentation"]:::processing
        EmbedGen["Embedding Generation"]:::processing
        click PreProcess "https://github.com/LukasHaas/PIGEON/blob/main/preprocessing/dataset_preprocessing.py"
        click GeoAug "https://github.com/LukasHaas/PIGEON/blob/main/preprocessing/geo_augmentor.py"
        click EmbedGen "https://github.com/LukasHaas/PIGEON/blob/main/preprocessing/embed.py"
    end

    subgraph "Dataset Management"
        GeocellSys["Geocell System"]:::data
        PretrainDS["Pretraining Dataset"]:::data
        FinetuneDS["Finetuning Dataset"]:::data
        BenchmarkDS["Benchmark Dataset"]:::data
        click GeocellSys "https://github.com/LukasHaas/PIGEON/tree/main/dataset_creation/geocell/"
        click PretrainDS "https://github.com/LukasHaas/PIGEON/blob/main/dataset_creation/pretrain/pretrain_dataset.py"
        click FinetuneDS "https://github.com/LukasHaas/PIGEON/blob/main/dataset_creation/finetune/finetune_dataset.py"
        click BenchmarkDS "https://github.com/LukasHaas/PIGEON/blob/main/dataset_creation/benchmark/benchmark_dataset.py"
    end

    subgraph "Model Architecture"
        CLIPEmbed{{CLIP Embedder}}:::model
        SuperGuessr{{Super Guessr}}:::model
        ProtoRef{{Proto Refiner}}:::model
        HEDGE["HEDGE Layer"]:::model
        PosEnc["Positional Encoder"]:::model
        click CLIPEmbed "https://github.com/LukasHaas/PIGEON/blob/main/models/clip_embedder.py"
        click SuperGuessr "https://github.com/LukasHaas/PIGEON/blob/main/models/super_guessr.py"
        click ProtoRef "https://github.com/LukasHaas/PIGEON/blob/main/models/proto_refiner.py"
        click HEDGE "https://github.com/LukasHaas/PIGEON/blob/main/models/layers/hedge.py"
        click PosEnc "https://github.com/LukasHaas/PIGEON/blob/main/models/layers/positional_encoder.py"
    end

    subgraph "Training System"
        TrainLoop["Training Loops"]:::training
        TrainModes["Training Modes"]:::training
        ModelStorage[("Model Storage")]:::storage
        click TrainLoop "https://github.com/LukasHaas/PIGEON/blob/main/training/train_eval_loop.py"
        click TrainModes "https://github.com/LukasHaas/PIGEON/blob/main/training/train_modes.py"
        click ModelStorage "https://github.com/LukasHaas/PIGEON/tree/main/saved_models/"
    end

    subgraph "Evaluation System"
        EvalMetrics["Evaluation Metrics"]:::eval
        MainEval["Main Evaluation"]:::eval
        BenchEval["Benchmark Evaluation"]:::eval
        click EvalMetrics "https://github.com/LukasHaas/PIGEON/blob/main/evaluation/metrics.py"
        click MainEval "https://github.com/LukasHaas/PIGEON/blob/main/evaluation/evaluate.py"
        click BenchEval "https://github.com/LukasHaas/PIGEON/blob/main/dataset_creation/benchmark/eval_dataset.p"
    end

    Config["System Configuration"]:::config
    click Config "https://github.com/LukasHaas/PIGEON/blob/main/config.py"

    %% Relationships
    RawImages & StreetView & BenchmarkData --> PreProcess
    PreProcess --> GeoAug
    GeoAug --> EmbedGen
    EmbedGen --> GeocellSys
    
    GeocellSys --> PretrainDS & FinetuneDS & BenchmarkDS
    
    PretrainDS & FinetuneDS & BenchmarkDS --> TrainLoop
    
    TrainLoop --> CLIPEmbed
    CLIPEmbed --> SuperGuessr
    SuperGuessr --> ProtoRef
    HEDGE & PosEnc --> SuperGuessr
    
    TrainLoop --> ModelStorage
    TrainModes --> TrainLoop
    
    ProtoRef --> EvalMetrics
    EvalMetrics --> MainEval
    MainEval --> BenchEval
    
    Config -..-> PreProcess & GeocellSys & TrainLoop & ModelStorage

    %% Styles
    classDef data fill:#gray,stroke:#333,stroke-width:2px
    classDef processing fill:#3498db,stroke:#333,stroke-width:2px
    classDef model fill:#2ecc71,stroke:#333,stroke-width:2px
    classDef training fill:#f1c40f,stroke:#333,stroke-width:2px
    classDef eval fill:#e74c3c,stroke:#333,stroke-width:2px
    classDef storage fill:#95a5a6,stroke:#333,stroke-width:2px
    classDef config fill:#9b59b6,stroke:#333,stroke-width:2px
```
