How rederoth/aiconic_scandy works
AICONic ScanDy simulates gaze exploration and object segmentation as a coupled, uncertainty-aware process over dynamic video. A researcher configures an experiment; the runner processes frames and optical flow, obtains segmentation measurements, updates a particle-filter segmentation, and advances the scanpath decision state to produce foveations and per-frame evidence. The README documents bottom-up appearance/motion cues and semantic or foveated-object prompts. Evaluation and visualization are separate downstream capabilities. Some subsystem wiring is documented rather than visible in the sampled excerpts, so uncertain internal links are omitted.
Main entry points: scanpath_producer.py Scanpath producer · run_experiment.py Experiment runner · ObjectSegmentationFilter.py Segmentation filter · particle_filter.py Particle filter
Components 21
Experiment runtime
- Experiment runner
aiconic_scandy/src/run_experiment.py - Frame and flow input
aiconic_scandy/src/run_experiment.py - Foveation results
aiconic_scandy/src/run_experiment.py
Object segmentation
- Segmentation filter
aiconic_scandy/src/ObjectSegmentationFilter.py - Particle filter
segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py - Measurement model
segmentation_particle_filter_leightweight/src/segmentation_particle_filter/measurement_model.py - Forward model
segmentation_particle_filter_leightweight/src/segmentation_particle_filter/forward_model.py - Segmentation particles
segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle.py - Image segmenters
segmentation_particle_filter_leightweight/src/segmentation_particle_filter/segmenters.py - Segmentation cues
aiconic_scandy/src/ObjectSegmentationFilter.py
Gaze decisions
- Scanpath producer
aiconic_scandy/src/scanpath_producer.py - Gaze filter
aiconic_scandy/src/GazeFilter.py - Task relevance
aiconic_scandy/src/TaskRelevantFilter.py - Shared model state
aiconic_scandy/src/scanpath_producer.py
Evaluation and analysis
- Scanpath evaluation
aiconic_scandy/src/scanpath_eval.ipynb - Ground-truth evaluation
aiconic_scandy/src/gt_scanpath_eval.py - Evidence visualization
aiconic_scandy/src/data_processing_hq.py - Summary plots
aiconic_scandy/src/summary_plots.py
Other components
- Researcher
- Video inputs
- Experiment results
result_data
Connections 24
- Researcher → Experiment runner: configures
- Video inputs → Frame and flow input: provides frames
- Experiment runner → Frame and flow input: processes sequence
- Experiment runner → Segmentation filter: updates segmentation
- Experiment runner → Scanpath producer: advances decisions
- Frame and flow input → Segmentation filter: supplies frame and flow
- Segmentation filter → Particle filter: updates particles
- Particle filter → Measurement model: weights and resamples
- Particle filter → Forward model: propagates particles
- Particle filter → Image segmenters: obtains segmentation
- Forward model → Segmentation particles: moves particles
- Measurement model → Segmentation particles: updates weights
- Segmentation cues → Segmentation filter: provides measurements
- Segmentation filter → Scanpath producer: provides object state
- Scanpath producer → Gaze filter: updates gaze state
- Scanpath producer → Task relevance: updates task state
- Scanpath producer → Shared model state: maintains state
- Scanpath producer → Foveation results: produces foveations
- Experiment runner → Experiment results: saves results
- Foveation results → Scanpath evaluation: evaluated by
- Foveation results → Ground-truth evaluation: compared with
- Experiment results → Evidence visualization: visualized from
- Experiment results → Summary plots: summarized by
- Scanpath evaluation → Summary plots: summarized by