# perform-lab-rit/domain-adaptation-eye-tracking architecture

> Architecture diagram of the GitHub repository perform-lab-rit/domain-adaptation-eye-tracking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/perform-lab-rit/domain-adaptation-eye-tracking
- Repository: https://github.com/perform-lab-rit/domain-adaptation-eye-tracking

## Overview

##### Project type and purpose
- This is a research-oriented Python codebase implementing a deep domain adaptation pipeline for eye-tracking segmentation.
- Core goal: improve performance on real eye-image datasets by generating and filtering synthetic images via Structure-Retaining CycleGANs (SRCGAN), pruning with a Siamese network, then training segmentation models (RITNet, DANN).

##### File structure
- Top-level scripts & orchestration (“extractor.py”, “srcgan_train.py”, “siamese_train.py”, “reconstruct_filter_stats.py”, “train_* .py”)
- “rc/” folder holds Bash wrappers to run full experiments on a Slurm cluster
- “vietlib/” Python package:
– data (extractors, transforms, constants)
– metrics (FID score, inception)
– model (CGAN, CNN, inceptionv4, siamese, mask_cgan, layers, loss, noise, segment)
– train (trainer utilities, cgan pipeline, siamese pipeline)
– utils (logging, image I/O, progress bar, weight init, data structures)
- docs/figs contains architecture and dataset-example images

##### Key insights from README
- Overall pipeline:
1) Extract raw eye datasets (OpenEDS, RIT-Eyes) into HDF5
2) Train multiple CycleGAN variants → generate synthetic eye images
3) Filter/prune synthetic images via Siamese network to maximize domain overlap
4) Assemble final datasets (orig_cgan.h5, cycled_synth.h5, cycled_synth_filter.h5)
5) Train segmentation models (RITNet, DANN) on real, synthetic, and mixed datasets
6) Evaluate performance (mIoU, domain-overlap metrics, FID score)

##### Main components
A. Data Layer
– Raw datasets: OpenEDS real HDF5, RIT-Eyes synthetic HDF5
B. Data Preprocessing & Extraction
– extractor.py module → HDF5 files
– transforms and data pipelines in vietlib/data
C. Generation Layer
– CycleGAN training (srcgan_train.py + vietlib/model/cgan.py)
– 64 parallel jobs via rc/srcgan_train.sh on Slurm
D. Filtering Layer
– Siamese network training (siamese_train.py + vietlib/model/siamese.py)
– Filter synthetic images (reconstruct_filter_stats.py)
E. Dataset Assembly
– make_desired_cycled.sh to pick best hyperparameters → outputs final HDF5 datasets
F. Model Training Layer
– RITNet training (train_ritnet_only.py, train_ritnet_mix.py) using vietlib/model/segment.py
– DANN training (train_dann_only.py, train_dann_mix.py) using vietlib/model/cnn.py + adversarial loss
– Scripts orchestrated via rc/train_model_population_and_stats_*.sh
G. Evaluation & Metrics
– FID score in vietlib/metrics/fid
– mIoU and other statistics computed in vietlib/utils/experiment.py and trainer modules
H. Orchestration & Environment
– Bash scripts in rc/ for Slurm cluster jobs (A100 GPUs, multi-day runs)
– install.sh and conda environment

- Raw Data → [extractor.py] → Preprocessed HDF5
- Preprocessed HDF5 → [CycleGAN Training] → Synthetic HDF5
- Synthetic HDF5 + Siamese weights → [Filtering Module] → Filtered Synthetic HDF5
- Real HDF5 + (Synthetic or Filtered) HDF5 → [Segmentation Model Training (RITNet/DANN)] → Trained Models
- Trained Models → [Evaluation] → Metrics, Figures, Logs

##### Architectural patterns & design principles
- Modular Python package (“vietlib”) supporting inversion of control via command‐line scripts
- Batch pipeline: data extraction → data generation → filtering → model training → evaluation
- Reproducible experiment orchestration via RC scripts (parameter sweeps, Slurm integration)
- HDF5 as intermediate data exchange format

##### Technologies, frameworks, libraries
- Python 3, H5py, NumPy, PyTorch (in vietlib/model)
- Slurm (cluster scheduler), Bash for orchestration
- Anaconda for environment management
- FID/Inception for generative quality metrics

##### Detail
- HDF5 filename (open_eds_real.h5, rit_eyes.h5, cycled_synth.h5, etc.)
- Hyperparameter sweep loops in the CycleGAN stage

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RawReal[(OpenEDS Real HDF5)]:::data
        RawSynth[(RIT-Eyes Synthetic HDF5)]:::data
        Extractor["extractor.py"]:::data
        Preproc[(Preprocessed HDF5)]:::data
        RawReal -->|extract| Extractor -->|outputs| Preproc
        RawSynth -->|extract| Extractor
        subgraph "vietlib/data"
            direction TB
            VL_Extractor["extractor.py"]:::data
            VL_Transforms["transforms.py"]:::data
            VL_Const["const.py"]:::data
        end
        Preproc --> VL_Extractor
    end

    %% Generation Layer
    subgraph "Generation Layer"
        direction TB
        RC_CycleGAN["rc/srcgan_train.sh"]:::orchestration
        SG_Script["srcgan_train.py"]:::data
        CGAN_Model["cgan.py"]:::util
        SynthData[(cycled_synth.h5)]:::data
        RC_CycleGAN --> SG_Script --> CGAN_Model --> SynthData
    end

    %% Filtering Layer
    subgraph "Filtering Layer"
        direction TB
        RC_FilterWrap["rc/reconstruct_filter_stats.sh"]:::orchestration
        SiameseScript["siamese_train.py"]:::data
        SiameseModel["siamese.py"]:::util
        FilterStats["reconstruct_filter_stats.py"]:::data
        FilteredData[(cycled_synth_filter.h5)]:::data
        RC_FilterWrap --> SiameseScript --> SiameseModel --> FilterStats --> FilteredData
    end

    %% Dataset Assembly
    subgraph "Dataset Assembly"
        direction TB
        RC_Assembly["rc/make_desired_cycled.sh"]:::orchestration
        FinalData1[(orig_cgan.h5)]:::data
        FinalData2[(cycled_synth.h5)]:::data
        FinalData3[(cycled_synth_filter.h5)]:::data
        RC_Assembly --> FinalData1
        RC_Assembly --> FinalData2
        RC_Assembly --> FinalData3
    end

    %% Model Training Layer
    subgraph "Model Training"
        direction TB
        subgraph RITNet_Training
            direction TB
            RC_RIT["rc/ritnet_raw_train.sh"]:::orchestration
            TrainRITOnly["train_ritnet_only.py"]:::training
            TrainRITMix["train_ritnet_mix.py"]:::training
            RITSegment["segment.py"]:::util
            RC_RIT --> TrainRITOnly --> RITSegment
            RC_RIT --> TrainRITMix --> RITSegment
        end
        subgraph DANN_Training
            direction TB
            RC_DANN1["rc/train_model_population_and_stats_single_dataset.sh"]:::orchestration
            RC_DANN2["rc/train_model_population_and_stats_mix_dataset.sh"]:::orchestration
            TrainDANNOnly["train_dann_only.py"]:::training
            TrainDANNMix["train_dann_mix.py"]:::training
            DANNModel["cnn.py"]:::util
            LossAdv["loss.py"]:::util
            RC_DANN1 --> TrainDANNOnly --> DANNModel & LossAdv
            RC_DANN2 --> TrainDANNMix --> DANNModel & LossAdv
        end
    end

    %% Evaluation & Metrics
    subgraph "Evaluation & Metrics"
        direction TB
        FIDScore["fid_score.py"]:::filtering
        Inception["inception.py"]:::filtering
        ExperimentUtils["experiment.py"]:::filtering
        SynthModels["Trained Models"]:::training
        SynthModels --> FIDScore
        SynthModels --> Inception
        SynthModels --> ExperimentUtils
    end

    %% Orchestration & Environment
    subgraph "Cluster Orchestration & Setup"
        direction TB
        RC_Folder["rc/"]:::orchestration
        Install["install.sh"]:::orchestration
    end

    %% Utilities Library
    subgraph "Core Utilities"
        direction TB
        Logger["logger.py"]:::util
        ImageIO["image.py"]:::util
        ProgressBar["progress_bar.py"]:::util
        WeightsInit["weights_init.py"]:::util
        NNUtils["nn.py"]:::util
        SumTree["sum_tree.py"]:::util
    end

    %% Connections
    Preproc -->|train| SG_Script
    SynthData -->|filter| SiameseScript
    FilteredData -->|train| TrainRITOnly
    FilteredData -->|train| TrainDANNOnly
    FinalData1 & FinalData2 & FinalData3 -->|train| TrainRITMix
    FinalData1 & FinalData2 & FinalData3 -->|train| TrainDANNMix
    SynthModels -->|evaluate| FIDScore

    %% Click events
    click Extractor "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/extractor.py"
    click VL_Extractor "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/data/extractor.py"
    click VL_Transforms "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/data/transforms.py"
    click VL_Const "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/data/const.py"
    click SG_Script "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/srcgan_train.py"
    click CGAN_Model "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/model/cgan.py"
    click RC_CycleGAN "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/srcgan_train.sh"
    click SiameseScript "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/siamese_train.py"
    click SiameseModel "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/model/siamese.py"
    click FilterStats "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/reconstruct_filter_stats.py"
    click RC_FilterWrap "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/reconstruct_filter_stats.sh"
    click RC_Assembly "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/make_desired_cycled.sh"
    click TrainRITOnly "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/train_ritnet_only.py"
    click TrainRITMix "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/train_ritnet_mix.py"
    click RITSegment "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/model/segment.py"
    click RC_RIT "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/ritnet_raw_train.sh"
    click TrainDANNOnly "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/train_dann_only.py"
    click TrainDANNMix "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/train_dann_mix.py"
    click DANNModel "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/model/cnn.py"
    click LossAdv "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/model/loss.py"
    click RC_DANN1 "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/train_model_population_and_stats_single_dataset.sh"
    click RC_DANN2 "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/rc/train_model_population_and_stats_mix_dataset.sh"
    click FIDScore "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/metrics/fid/fid_score.py"
    click Inception "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/metrics/fid/inception.py"
    click ExperimentUtils "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/experiment.py"
    click RC_Folder "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/tree/main/rc/"
    click Install "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/install.sh"
    click Logger "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/logger.py"
    click ImageIO "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/image.py"
    click ProgressBar "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/progress_bar.py"
    click WeightsInit "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/weights_init.py"
    click NNUtils "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/nn.py"
    click SumTree "https://github.com/perform-lab-rit/domain-adaptation-eye-tracking/blob/main/vietlib/utils/data_structure/sum_tree.py"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef training fill:#d4edda,stroke:#333,stroke-width:1px
    classDef filtering fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef orchestration fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef util fill:#f8d7da,stroke:#333,stroke-width:1px
```
