# siyuvision/ia4u-deep-dive architecture

> Architecture diagram of the GitHub repository siyuvision/ia4u-deep-dive, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/siyuvision/ia4u-deep-dive
- Repository: https://github.com/siyuvision/ia4u-deep-dive

## Overview

This repository packages three reproducible computer-vision experiments: whole-slide pathology detection comparison, multimodal renal-tubule box evaluation, and nnFoundation CT tumor segmentation. The user-facing pattern is to run an experiment, obtain model outputs, evaluate them, and export or report results. The CT pilot’s stage order is explicit in its pipeline runner; several other subsystem connections are documented only at the README or directory level. Edges involving unsampled behavior are omitted or marked unverified rather than inferred from filenames.

Main entry points: [`projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/run_pipeline.py) (Pilot orchestrator), [`projects/nnfoundation-tumseg-ct/repo/fourshot_config.py`](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/fourshot_config.py) (Four-shot protocol), [`projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py) (Run evaluation), [`projects/nnfoundation-tumseg-ct/repo/project_env.py`](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/project_env.py) (Project environment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pathology["Pathology detection"]
  node_wsi_run["WSI inference"]
  node_classpose_run["ClassPose inference"]
  node_patch_run["Patch inference<br/>[run_patch.py]"]
  node_wsi_prepare["WSI preparation<br/>[prepare_wsi.py]"]
  node_point_eval["Point evaluation<br/>[evaluate_points.py]"]
  node_qupath_path["QuPath export<br/>[export_qupath.py]"]
end

subgraph group_bbox["Renal box benchmark"]
  node_bbox_runner["Model calls<br/>[run_models.py]"]
  node_api_client["API client<br/>[openrouter.py]"]
  node_raw_runs["Call records<br/>[run_models.py]"]
  node_bbox_evaluate["Run evaluation<br/>[evaluate_runs.py]"]
  node_parsing["Response parsing<br/>[parsing.py]"]
  node_box_geometry["Box geometry<br/>[boxes.py]"]
  node_scoring["Box scoring<br/>[evaluation.py]"]
  node_efficiency["Cost and speed<br/>[efficiency.py]"]
  node_render_boxes["Result rendering<br/>[render.py]"]
  node_qupath_bbox["QuPath export<br/>[export_qupath.py]"]
end

subgraph group_ct["CT segmentation"]
  node_ct_pipeline["Pilot orchestrator<br/>[run_pipeline.py]"]
  node_data_plan["Data planning<br/>[plan_data.py]"]
  node_ct_train["Model training<br/>[train.py]"]
  node_ct_predict["CT prediction<br/>[predict.py]"]
  node_ct_eval["Segmentation evaluation<br/>[evaluate.py]"]
  node_fourshot_config["Four-shot protocol<br/>[fourshot_config.py]"]
  node_fourshot_prep["Four-shot preparation"]
  node_fourshot_run["Four-shot run<br/>[run_fourshot.py]"]
  node_fourshot_eval["Four-shot evaluation"]
  node_spatial_guard["Spatial prior guard<br/>[spatial_guard.py]"]
  node_assets["Scientific assets<br/>[render_assets.py]"]
  node_fourshot_report["Four-shot report<br/>[report_fourshot.py]"]
  node_ct_env["Project environment<br/>[project_env.py]"]
end

node_user(("Researcher"))
node_model_api{{"OpenRouter API"}}

node_user -->|"runs inference"| node_wsi_run
node_user -->|"runs inference"| node_classpose_run
node_user -->|"runs benchmark"| node_bbox_runner
node_user -->|"runs experiment"| node_ct_pipeline
node_wsi_run -->|"exports points"| node_qupath_path
node_classpose_run -->|"exports points"| node_qupath_path
node_patch_run -->|"exports points"| node_qupath_path
node_bbox_runner -->|"calls client"| node_api_client
node_api_client -->|"sends requests"| node_model_api
node_bbox_runner -.->|"records calls"| node_raw_runs
node_bbox_evaluate -->|"uses geometry"| node_box_geometry
node_bbox_evaluate -->|"scores answers"| node_scoring
node_bbox_evaluate -->|"summarizes efficiency"| node_efficiency
node_bbox_evaluate -->|"renders results"| node_render_boxes
node_scoring -->|"parses text"| node_parsing
node_scoring -->|"converts and scores"| node_box_geometry
node_efficiency -->|"reads latency"| node_api_client
node_render_boxes -->|"uses boxes"| node_box_geometry
node_user -->|"exports results"| node_qupath_bbox
node_ct_pipeline -->|"runs stage"| node_data_plan
node_ct_pipeline -->|"runs stage"| node_ct_train
node_ct_pipeline -->|"runs stage"| node_ct_predict
node_ct_pipeline -->|"runs stage"| node_ct_eval
node_fourshot_config -->|"uses settings"| node_ct_env
node_fourshot_prep -->|"uses protocol"| node_fourshot_config
node_fourshot_prep -->|"uses prior guard"| node_spatial_guard
node_fourshot_run -->|"uses protocol"| node_fourshot_config
node_fourshot_eval -->|"uses protocol"| node_fourshot_config
node_fourshot_eval -->|"reuses evaluation"| node_ct_eval
node_fourshot_eval -->|"uses prior guard"| node_spatial_guard
node_fourshot_report -->|"uses protocol"| node_fourshot_config
node_fourshot_report -->|"uses assets"| node_assets
node_user -->|"prepares cohort"| node_fourshot_prep
node_user -->|"runs retest"| node_fourshot_run
node_user -->|"reviews evidence"| node_fourshot_report
node_ct_train -->|"configures runtime"| node_ct_env
node_ct_predict -->|"configures runtime"| node_ct_env
node_ct_eval -->|"uses paths"| node_ct_env

click node_wsi_run "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_kongnet_wsi_windows.py"
click node_classpose_run "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_classpose_wsi.py"
click node_patch_run "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_patch.py"
click node_wsi_prepare "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/prepare_wsi.py"
click node_point_eval "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/evaluate_points.py"
click node_qupath_path "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/export_qupath.py"
click node_bbox_runner "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py"
click node_api_client "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/openrouter.py"
click node_raw_runs "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py"
click node_bbox_evaluate "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py"
click node_parsing "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/parsing.py"
click node_box_geometry "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/boxes.py"
click node_scoring "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/evaluation.py"
click node_efficiency "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/efficiency.py"
click node_render_boxes "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/render.py"
click node_qupath_bbox "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/export_qupath.py"
click node_ct_pipeline "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/run_pipeline.py"
click node_data_plan "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/plan_data.py"
click node_ct_train "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/train.py"
click node_ct_predict "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/predict.py"
click node_ct_eval "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/evaluate.py"
click node_fourshot_config "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/fourshot_config.py"
click node_fourshot_prep "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/prepare_fourshot.py"
click node_fourshot_run "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/run_fourshot.py"
click node_fourshot_eval "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py"
click node_spatial_guard "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/spatial_guard.py"
click node_assets "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/render_assets.py"
click node_fourshot_report "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/report_fourshot.py"
click node_ct_env "https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/project_env.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_wsi_run,node_classpose_run,node_patch_run,node_wsi_prepare,node_point_eval,node_qupath_path toneBlue
class node_bbox_runner,node_api_client,node_raw_runs,node_bbox_evaluate,node_parsing,node_box_geometry,node_scoring,node_efficiency,node_render_boxes,node_qupath_bbox toneAmber
class node_ct_pipeline,node_data_plan,node_ct_train,node_ct_predict,node_ct_eval,node_fourshot_config,node_fourshot_prep,node_fourshot_run,node_fourshot_eval,node_spatial_guard,node_assets,node_fourshot_report,node_ct_env,node_model_api toneMint
class node_user toneIndigo
```

## Components

### Pathology detection

- [WSI inference](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_kongnet_wsi_windows.py): `projects/kongnet-vs-classpose/scripts/run_kongnet_wsi_windows.py`
- [ClassPose inference](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_classpose_wsi.py): `projects/kongnet-vs-classpose/scripts/run_classpose_wsi.py`
- [Patch inference](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/run_patch.py): `projects/kongnet-vs-classpose/scripts/run_patch.py`
- [WSI preparation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/prepare_wsi.py): `projects/kongnet-vs-classpose/scripts/prepare_wsi.py`
- [Point evaluation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/evaluate_points.py): `projects/kongnet-vs-classpose/scripts/evaluate_points.py`
- [QuPath export](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/kongnet-vs-classpose/scripts/export_qupath.py): `projects/kongnet-vs-classpose/scripts/export_qupath.py`

### Renal box benchmark

- [Model calls](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py): `projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py`
- [API client](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/openrouter.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/openrouter.py`
- [Call records](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py): `projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py`
- [Run evaluation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py): `projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`
- [Response parsing](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/parsing.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/parsing.py`
- [Box geometry](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/boxes.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/boxes.py`
- [Box scoring](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/evaluation.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/evaluation.py`
- [Cost and speed](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/efficiency.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/efficiency.py`
- [Result rendering](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/render.py): `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/render.py`
- [QuPath export](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/mllm-renal-tubule-bbox/repo/scripts/export_qupath.py): `projects/mllm-renal-tubule-bbox/repo/scripts/export_qupath.py`

### CT segmentation

- [Pilot orchestrator](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/run_pipeline.py): `projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`
- [Data planning](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/plan_data.py): `projects/nnfoundation-tumseg-ct/repo/plan_data.py`
- [Model training](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/train.py): `projects/nnfoundation-tumseg-ct/repo/train.py`
- [CT prediction](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/predict.py): `projects/nnfoundation-tumseg-ct/repo/predict.py`
- [Segmentation evaluation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/evaluate.py): `projects/nnfoundation-tumseg-ct/repo/evaluate.py`
- [Four-shot protocol](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/fourshot_config.py): `projects/nnfoundation-tumseg-ct/repo/fourshot_config.py`
- [Four-shot preparation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/prepare_fourshot.py): `projects/nnfoundation-tumseg-ct/repo/prepare_fourshot.py`
- [Four-shot run](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/run_fourshot.py): `projects/nnfoundation-tumseg-ct/repo/run_fourshot.py`
- [Four-shot evaluation](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py): `projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py`
- [Spatial prior guard](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/spatial_guard.py): `projects/nnfoundation-tumseg-ct/repo/spatial_guard.py`
- [Scientific assets](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/render_assets.py): `projects/nnfoundation-tumseg-ct/repo/render_assets.py`
- [Four-shot report](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/report_fourshot.py): `projects/nnfoundation-tumseg-ct/repo/report_fourshot.py`
- [Project environment](https://github.com/siyuvision/ia4u-deep-dive/blob/main/projects/nnfoundation-tumseg-ct/repo/project_env.py): `projects/nnfoundation-tumseg-ct/repo/project_env.py`

### Other components

- **Researcher**
- **OpenRouter API**

## Connections

- Researcher → WSI inference: runs inference
- Researcher → ClassPose inference: runs inference
- Researcher → Model calls: runs benchmark
- Researcher → Pilot orchestrator: runs experiment
- WSI inference → QuPath export: exports points (evidence: `projects/kongnet-vs-classpose/scripts/run_kongnet_wsi_windows.py`)
- ClassPose inference → QuPath export: exports points (evidence: `projects/kongnet-vs-classpose/scripts/run_classpose_wsi.py`)
- Patch inference → QuPath export: exports points (evidence: `projects/kongnet-vs-classpose/scripts/run_patch.py`)
- Model calls → API client: calls client (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py`)
- API client → OpenRouter API: sends requests (evidence: `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/openrouter.py`)
- Model calls → Call records: records calls (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/run_models.py`)
- Run evaluation → Box geometry: uses geometry (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`)
- Run evaluation → Box scoring: scores answers (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`)
- Run evaluation → Cost and speed: summarizes efficiency (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`)
- Run evaluation → Result rendering: renders results (evidence: `projects/mllm-renal-tubule-bbox/repo/scripts/evaluate_runs.py`)
- Box scoring → Response parsing: parses text (evidence: `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/evaluation.py`)
- Box scoring → Box geometry: converts and scores (evidence: `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/evaluation.py`)
- Cost and speed → API client: reads latency (evidence: `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/efficiency.py`)
- Result rendering → Box geometry: uses boxes (evidence: `projects/mllm-renal-tubule-bbox/repo/src/mllm_bbox/render.py`)
- Researcher → QuPath export: exports results
- Pilot orchestrator → Data planning: runs stage (evidence: `projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`)
- Pilot orchestrator → Model training: runs stage (evidence: `projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`)
- Pilot orchestrator → CT prediction: runs stage (evidence: `projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`)
- Pilot orchestrator → Segmentation evaluation: runs stage (evidence: `projects/nnfoundation-tumseg-ct/repo/run_pipeline.py`)
- Four-shot protocol → Project environment: uses settings (evidence: `projects/nnfoundation-tumseg-ct/repo/fourshot_config.py`)
- Four-shot preparation → Four-shot protocol: uses protocol (evidence: `projects/nnfoundation-tumseg-ct/repo/prepare_fourshot.py`)
- Four-shot preparation → Spatial prior guard: uses prior guard (evidence: `projects/nnfoundation-tumseg-ct/repo/prepare_fourshot.py`)
- Four-shot run → Four-shot protocol: uses protocol (evidence: `projects/nnfoundation-tumseg-ct/repo/run_fourshot.py`)
- Four-shot evaluation → Four-shot protocol: uses protocol (evidence: `projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py`)
- Four-shot evaluation → Segmentation evaluation: reuses evaluation (evidence: `projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py`)
- Four-shot evaluation → Spatial prior guard: uses prior guard (evidence: `projects/nnfoundation-tumseg-ct/repo/evaluate_fourshot.py`)
- Four-shot report → Four-shot protocol: uses protocol (evidence: `projects/nnfoundation-tumseg-ct/repo/report_fourshot.py`)
- Four-shot report → Scientific assets: uses assets (evidence: `projects/nnfoundation-tumseg-ct/repo/report_fourshot.py`)
- Researcher → Four-shot preparation: prepares cohort
- Researcher → Four-shot run: runs retest
- Researcher → Four-shot report: reviews evidence
- Model training → Project environment: configures runtime (evidence: `projects/nnfoundation-tumseg-ct/repo/train.py`)
- CT prediction → Project environment: configures runtime (evidence: `projects/nnfoundation-tumseg-ct/repo/predict.py`)
- Segmentation evaluation → Project environment: uses paths (evidence: `projects/nnfoundation-tumseg-ct/repo/evaluate.py`)
