# rederoth/aiconic_scandy architecture

> Architecture diagram of the GitHub repository rederoth/aiconic_scandy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rederoth/aiconic_scandy
- Repository: https://github.com/rederoth/aiconic_scandy
- Video tour (about a minute): https://gitdiagram.com/rederoth/aiconic_scandy/video

## Overview

AICONic ScanDy simulates gaze exploration and object segmentation as a coupled, uncertainty-aware process over dynamic video. A researcher configures an experiment; the runner processes frames and optical flow, obtains segmentation measurements, updates a particle-filter segmentation, and advances the scanpath decision state to produce foveations and per-frame evidence. The README documents bottom-up appearance/motion cues and semantic or foveated-object prompts. Evaluation and visualization are separate downstream capabilities. Some subsystem wiring is documented rather than visible in the sampled excerpts, so uncertain internal links are omitted.

Main entry points: [`aiconic_scandy/src/scanpath_producer.py`](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_producer.py) (Scanpath producer), [`aiconic_scandy/src/run_experiment.py`](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py) (Experiment runner), [`aiconic_scandy/src/ObjectSegmentationFilter.py`](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/ObjectSegmentationFilter.py) (Segmentation filter), [`segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py`](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py) (Particle filter)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiment["Experiment runtime"]
  node_runner["Experiment runner<br/>[run_experiment.py]"]
  node_inputproc["Frame and flow input<br/>[run_experiment.py]"]
  node_foveations["Foveation results<br/>[run_experiment.py]"]
end

subgraph group_segmentation["Object segmentation"]
  node_segfilter["Segmentation filter"]
  node_particlefilter["Particle filter<br/>[particle_filter.py]"]
  node_measurement["Measurement model"]
  node_forward["Forward model<br/>[forward_model.py]"]
  node_particles["Segmentation particles<br/>[particle.py]"]
  node_segmenters["Image segmenters<br/>[segmenters.py]"]
  node_segcues["Segmentation cues"]
end

subgraph group_decision["Gaze decisions"]
  node_scanproducer["Scanpath producer"]
  node_gaze["Gaze filter<br/>[GazeFilter.py]"]
  node_task["Task relevance"]
  node_state["Shared model state"]
end

subgraph group_analysis["Evaluation and analysis"]
  node_evaluation["Scanpath evaluation"]
  node_gt_evaluation["Ground-truth evaluation"]
  node_visualization["Evidence visualization"]
  node_summaries["Summary plots<br/>[summary_plots.py]"]
end

node_researcher(("Researcher"))
node_video["Video inputs"]
node_results["Experiment results"]

node_researcher -->|"configures"| node_runner
node_video -->|"provides frames"| node_inputproc
node_runner -->|"processes sequence"| node_inputproc
node_runner -->|"updates segmentation"| node_segfilter
node_runner -->|"advances decisions"| node_scanproducer
node_inputproc -->|"supplies frame and flow"| node_segfilter
node_segfilter -->|"updates particles"| node_particlefilter
node_particlefilter -->|"weights and resamples"| node_measurement
node_particlefilter -->|"propagates particles"| node_forward
node_particlefilter -->|"obtains segmentation"| node_segmenters
node_forward -->|"moves particles"| node_particles
node_measurement -->|"updates weights"| node_particles
node_segcues -->|"provides measurements"| node_segfilter
node_segfilter -->|"provides object state"| node_scanproducer
node_scanproducer -->|"updates gaze state"| node_gaze
node_scanproducer -->|"updates task state"| node_task
node_scanproducer -->|"maintains state"| node_state
node_scanproducer -->|"produces foveations"| node_foveations
node_runner -->|"saves results"| node_results
node_foveations -->|"evaluated by"| node_evaluation
node_foveations -->|"compared with"| node_gt_evaluation
node_results -->|"visualized from"| node_visualization
node_results -->|"summarized by"| node_summaries
node_evaluation -.->|"summarized by"| node_summaries

click node_runner "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py"
click node_inputproc "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py"
click node_segfilter "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/ObjectSegmentationFilter.py"
click node_particlefilter "https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py"
click node_measurement "https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/measurement_model.py"
click node_forward "https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/forward_model.py"
click node_particles "https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle.py"
click node_segmenters "https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/segmenters.py"
click node_segcues "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/ObjectSegmentationFilter.py"
click node_scanproducer "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_producer.py"
click node_gaze "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/GazeFilter.py"
click node_task "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/TaskRelevantFilter.py"
click node_state "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_producer.py"
click node_foveations "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py"
click node_evaluation "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_eval.ipynb"
click node_gt_evaluation "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/gt_scanpath_eval.py"
click node_visualization "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/data_processing_hq.py"
click node_summaries "https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/summary_plots.py"
click node_results "https://github.com/rederoth/aiconic_scandy/tree/main/result_data"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_inputproc,node_foveations toneBlue
class node_segfilter,node_particlefilter,node_measurement,node_forward,node_particles,node_segmenters,node_segcues toneAmber
class node_scanproducer,node_gaze,node_task,node_state toneMint
class node_evaluation,node_gt_evaluation,node_visualization,node_summaries toneRose
class node_researcher,node_video toneIndigo
class node_results toneTeal
```

## Components

### Experiment runtime

- [Experiment runner](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py): `aiconic_scandy/src/run_experiment.py`
- [Frame and flow input](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py): `aiconic_scandy/src/run_experiment.py`
- [Foveation results](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/run_experiment.py): `aiconic_scandy/src/run_experiment.py`

### Object segmentation

- [Segmentation filter](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/ObjectSegmentationFilter.py): `aiconic_scandy/src/ObjectSegmentationFilter.py`
- [Particle filter](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py): `segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle_filter.py`
- [Measurement model](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/measurement_model.py): `segmentation_particle_filter_leightweight/src/segmentation_particle_filter/measurement_model.py`
- [Forward model](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/forward_model.py): `segmentation_particle_filter_leightweight/src/segmentation_particle_filter/forward_model.py`
- [Segmentation particles](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle.py): `segmentation_particle_filter_leightweight/src/segmentation_particle_filter/particle.py`
- [Image segmenters](https://github.com/rederoth/aiconic_scandy/blob/main/segmentation_particle_filter_leightweight/src/segmentation_particle_filter/segmenters.py): `segmentation_particle_filter_leightweight/src/segmentation_particle_filter/segmenters.py`
- [Segmentation cues](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/ObjectSegmentationFilter.py): `aiconic_scandy/src/ObjectSegmentationFilter.py`

### Gaze decisions

- [Scanpath producer](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_producer.py): `aiconic_scandy/src/scanpath_producer.py`
- [Gaze filter](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/GazeFilter.py): `aiconic_scandy/src/GazeFilter.py`
- [Task relevance](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/TaskRelevantFilter.py): `aiconic_scandy/src/TaskRelevantFilter.py`
- [Shared model state](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_producer.py): `aiconic_scandy/src/scanpath_producer.py`

### Evaluation and analysis

- [Scanpath evaluation](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/scanpath_eval.ipynb): `aiconic_scandy/src/scanpath_eval.ipynb`
- [Ground-truth evaluation](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/gt_scanpath_eval.py): `aiconic_scandy/src/gt_scanpath_eval.py`
- [Evidence visualization](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/data_processing_hq.py): `aiconic_scandy/src/data_processing_hq.py`
- [Summary plots](https://github.com/rederoth/aiconic_scandy/blob/main/aiconic_scandy/src/summary_plots.py): `aiconic_scandy/src/summary_plots.py`

### Other components

- **Researcher**
- **Video inputs**
- [Experiment results](https://github.com/rederoth/aiconic_scandy/tree/main/result_data): `result_data`

## Connections

- Researcher → Experiment runner: configures
- Video inputs → Frame and flow input: provides frames
- Experiment runner → Frame and flow input: processes sequence
- Experiment runner → Segmentation filter: updates segmentation
- Experiment runner → Scanpath producer: advances decisions
- Frame and flow input → Segmentation filter: supplies frame and flow
- Segmentation filter → Particle filter: updates particles
- Particle filter → Measurement model: weights and resamples
- Particle filter → Forward model: propagates particles
- Particle filter → Image segmenters: obtains segmentation
- Forward model → Segmentation particles: moves particles
- Measurement model → Segmentation particles: updates weights
- Segmentation cues → Segmentation filter: provides measurements
- Segmentation filter → Scanpath producer: provides object state
- Scanpath producer → Gaze filter: updates gaze state
- Scanpath producer → Task relevance: updates task state
- Scanpath producer → Shared model state: maintains state
- Scanpath producer → Foveation results: produces foveations
- Experiment runner → Experiment results: saves results
- Foveation results → Scanpath evaluation: evaluated by
- Foveation results → Ground-truth evaluation: compared with
- Experiment results → Evidence visualization: visualized from
- Experiment results → Summary plots: summarized by
- Scanpath evaluation → Summary plots: summarized by