How abhiram0102/rapid-twin works
RAPiD-Twin is an experimental research prototype for combining retinal imagery, physiological time series, and longitudinal health-state modelling—not a verified clinical system. Its intended workflow runs retinal images through segmentation and graph/image encoders, combines those representations with encoded physiology, then models temporal state and experimental disease outputs; biomarker and reliability concepts feed a proposed API. The README documents this architecture but cautions that end-to-end patient alignment is unestablished. The only sampled source is the wearable collector, so other stage relationships reflect documented intent, not inspected runtime calls.
Main entry points: wearables.py Wearable collectors · 06_phase5_fusion_transformer.ipynb Multimodal fusion · 02_phase1_segmentation_finetune.ipynb Vessel segmentation · 10_phase9_reliability.ipynb Reliability concepts
Components 17
Retinal processing
- Vessel segmentation
notebooks/02_phase1_segmentation_finetune.ipynb - Vessel graph extraction
notebooks/03_phase2_graph_gnn.ipynb - Graph encoder
notebooks/03_phase2_graph_gnn.ipynb - Retinal encoder
notebooks/04_phase3_retinal_encoder.ipynb
Physiology inputs
- Wearable collectors
src/wearables.py - Physiological series
- Physiology encoder
notebooks/05_phase4_physiology_encoder.ipynb
Twin modelling
- Multimodal fusion
notebooks/06_phase5_fusion_transformer.ipynb - Temporal state model
notebooks/07_phase6_temporal_twin.ipynb
Outputs and API
- Disease heads
notebooks/08_phase7_disease_heads.ipynb - Biomarker extraction
notebooks/09_phase8_biomarkers.ipynb - Reliability concepts
notebooks/10_phase9_reliability.ipynb - Proposed API
notebooks/11_phase10_api.ipynb
Other components
- Fundus image
- Apple HealthKit
- Google Fit
- Fitbit API
Connections 18
- Fundus image → Vessel segmentation: segments
README.md - Vessel segmentation → Vessel graph extraction: extracts graph
README.md - Vessel graph extraction → Graph encoder: encodes graph
README.md - Fundus image → Retinal encoder: encodes image
README.md - Apple HealthKit → Wearable collectors: collects data
src/wearables.py - Google Fit → Wearable collectors: collects data
src/wearables.py - Fitbit API → Wearable collectors: collects data
src/wearables.py - Wearable collectors → Physiological series: returns series
src/wearables.py - Physiological series → Physiology encoder: encodes series
README.md - Graph encoder → Multimodal fusion: fuses representation
README.md - Retinal encoder → Multimodal fusion: fuses representation
README.md - Physiology encoder → Multimodal fusion: fuses representation
README.md - Multimodal fusion → Temporal state model: models state
README.md - Temporal state model → Disease heads: feeds heads
README.md - Vessel segmentation → Biomarker extraction: extracts biomarkers
README.md - Disease heads → Reliability concepts: informs concepts
README.md - Biomarker extraction → Reliability concepts: informs concepts
README.md - Reliability concepts → Proposed API: feeds proposal
README.md