How perform-lab-rit/domain-adaptation-eye-tracking works
Project type and purpose
• This is a research-oriented Python codebase implementing a deep domain adaptation pipeline for eye-tracking segmentation.
• Core goal: improve performance on real eye-image datasets by generating and filtering synthetic images via Structure-Retaining CycleGANs (SRCGAN), pruning with a Siamese network, then training segmentation models (RITNet, DANN).
File structure
• Top-level scripts & orchestration (“extractor.py”, “srcgan_train.py”, “siamese_train.py”, “reconstruct_filter_stats.py”, “train_* .py”)
• “rc/” folder holds Bash wrappers to run full experiments on a Slurm cluster
• “vietlib/” Python package:
– data (extractors, transforms, constants)
– metrics (FID score, inception)
– model (CGAN, CNN, inceptionv4, siamese, mask_cgan, layers, loss, noise, segment)
– train (trainer utilities, cgan pipeline, siamese pipeline)
– utils (logging, image I/O, progress bar, weight init, data structures)
• docs/figs contains architecture and dataset-example images
Key insights from README
• Overall pipeline:
1) Extract raw eye datasets (OpenEDS, RIT-Eyes) into HDF5
2) Train multiple CycleGAN variants → generate synthetic eye images
3) Filter/prune synthetic images via Siamese network to maximize domain overlap
4) Assemble final datasets (orig_cgan.h5, cycled_synth.h5, cycled_synth_filter.h5)
5) Train segmentation models (RITNet, DANN) on real, synthetic, and mixed datasets
6) Evaluate performance (mIoU, domain-overlap metrics, FID score)
Main components
A. Data Layer
– Raw datasets: OpenEDS real HDF5, RIT-Eyes synthetic HDF5
B. Data Preprocessing & Extraction
– extractor.py module → HDF5 files
– transforms and data pipelines in vietlib/data
C. Generation Layer
– CycleGAN training (srcgan_train.py + vietlib/model/cgan.py)
– 64 parallel jobs via rc/srcgan_train.sh on Slurm
D. Filtering Layer
– Siamese network training (siamese_train.py + vietlib/model/siamese.py)
– Filter synthetic images (reconstruct_filter_stats.py)
E. Dataset Assembly
– make_desired_cycled.sh to pick best hyperparameters → outputs final HDF5 datasets
F. Model Training Layer
– RITNet training (train_ritnet_only.py, train_ritnet_mix.py) using vietlib/model/segment.py
– DANN training (train_dann_only.py, train_dann_mix.py) using vietlib/model/cnn.py + adversarial loss
– Scripts orchestrated via rc/train_model_population_and_stats_*.sh
G. Evaluation & Metrics
– FID score in vietlib/metrics/fid
– mIoU and other statistics computed in vietlib/utils/experiment.py and trainer modules
H. Orchestration & Environment
– Bash scripts in rc/ for Slurm cluster jobs (A100 GPUs, multi-day runs)
– install.sh and conda environment
• Raw Data → [extractor.py] → Preprocessed HDF5
• Preprocessed HDF5 → [CycleGAN Training] → Synthetic HDF5
• Synthetic HDF5 + Siamese weights → [Filtering Module] → Filtered Synthetic HDF5
• Real HDF5 + (Synthetic or Filtered) HDF5 → [Segmentation Model Training (RITNet/DANN)] → Trained Models
• Trained Models → [Evaluation] → Metrics, Figures, Logs
Architectural patterns & design principles
• Modular Python package (“vietlib”) supporting inversion of control via command‐line scripts
• Batch pipeline: data extraction → data generation → filtering → model training → evaluation
• Reproducible experiment orchestration via RC scripts (parameter sweeps, Slurm integration)
• HDF5 as intermediate data exchange format
Technologies, frameworks, libraries
• Python 3, H5py, NumPy, PyTorch (in vietlib/model)
• Slurm (cluster scheduler), Bash for orchestration
• Anaconda for environment management
• FID/Inception for generative quality metrics
Detail
• HDF5 filename (open_eds_real.h5, rit_eyes.h5, cycled_synth.h5, etc.)
• Hyperparameter sweep loops in the CycleGAN stage