# wenchuan-zhang/patho-r1 architecture

> Architecture diagram of the GitHub repository wenchuan-zhang/patho-r1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wenchuan-zhang/patho-r1
- Repository: https://github.com/wenchuan-zhang/patho-r1

## Overview

Patho-R1 is presented as a pathology vision-language model and reasoning system. The checked-in runtime evidence chiefly covers document-to-figure dataset extraction and Patho-CLIP dataset preparation/evaluation; the README also documents model inference, but the model implementation is not present in the supplied tree. Edges are limited to relationships shown in excerpts or explicitly documented; unsampled areas are not treated as inspected.

Main entry points: [`data/ImageTextExtraction/LMBased/main.py`](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/main.py) (Extraction workflow), [`data/ImageTextExtraction/LMBased/layout_async.py`](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/layout_async.py) (Layout and caption analysis), [`clip/val/zero_shot_evaluation.py`](https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/zero_shot_evaluation.py) (Zero-shot evaluation), [`data/ImageTextExtraction/LMBased/inline_async.py`](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/inline_async.py) (Inline reference extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_extraction["Figure Extraction"]
  node_extractor["Extraction workflow<br/>[main.py]"]
  node_render["PDF rendering<br/>[pdf2img.py]"]
  node_layout["Layout and caption analysis<br/>[layout_async.py]"]
  node_crop["Figure cropping<br/>[crop.py]"]
  node_prompts["Analysis prompts<br/>[prompts.py]"]
  node_page_tools["Page context helpers<br/>[tools.py]"]
  node_inline["Inline reference extraction<br/>[inline_async.py]"]
  node_merge["Result merging<br/>[alignment.py]"]
end

subgraph group_dataset["Dataset Preparation"]
  node_shards["WebDataset shards"]
end

subgraph group_clip["Patho-CLIP Evaluation"]
  node_clip_model{{"Patho-CLIP model"}}
  node_zero_shot["Zero-shot evaluation"]
  node_retrieval["Retrieval evaluation"]
  node_few_shot["Few-shot evaluation"]
  node_metrics["Evaluation metrics"]
end

subgraph group_reasoning["Pathology Reasoning"]
  node_r1_inference["Patho-R1 inference<br/>[README.md]"]
  node_reasoning_output["Reasoned answer"]
end

node_operator(("Dataset operator"))
node_pdf["PDF document"]
node_qwen{{"Qwen-compatible API"}}
node_tabular["Image-caption table"]
node_pathologist(("Pathology user"))

node_operator -->|"runs"| node_extractor
node_pdf -->|"provides input"| node_extractor
node_extractor -->|"runs analysis"| node_layout
node_layout -->|"requests analysis"| node_qwen
node_layout -->|"uses prompts"| node_prompts
node_layout -->|"uses helpers"| node_page_tools
node_extractor -->|"runs extraction"| node_inline
node_inline -->|"sends requests"| node_layout
node_inline -->|"uses prompts"| node_prompts
node_extractor -->|"merges results"| node_merge
node_operator -->|"creates shards"| node_shards
node_tabular -->|"provides captions"| node_shards
node_shards -.->|"supplies dataset"| node_zero_shot
node_pathologist -->|"asks question"| node_r1_inference
node_r1_inference -->|"generates answer"| node_reasoning_output
node_pathologist -->|"evaluates model"| node_zero_shot
node_pathologist -->|"evaluates retrieval"| node_retrieval
node_pathologist -.->|"evaluates few-shot"| node_few_shot
node_zero_shot -->|"encodes images"| node_clip_model
node_retrieval -->|"encodes pairs"| node_clip_model
node_zero_shot -->|"computes accuracy"| node_metrics
node_retrieval -->|"computes recall"| node_metrics

click node_extractor "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/main.py"
click node_render "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/pdf2img.py"
click node_layout "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/layout_async.py"
click node_crop "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/crop.py"
click node_prompts "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/prompts.py"
click node_page_tools "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/tools.py"
click node_inline "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/inline_async.py"
click node_merge "https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/alignment.py"
click node_shards "https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/dataset_process/quit_to_web_dataset.py"
click node_zero_shot "https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/zero_shot_evaluation.py"
click node_retrieval "https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/retrieval_evaluation.py"
click node_few_shot "https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/few_shot_evaluation.py"
click node_r1_inference "https://github.com/wenchuan-zhang/patho-r1/blob/main/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extractor,node_render,node_layout,node_crop,node_prompts,node_page_tools,node_inline,node_merge,node_pathologist toneBlue
class node_shards toneAmber
class node_clip_model,node_zero_shot,node_retrieval,node_few_shot,node_metrics,node_qwen toneMint
class node_r1_inference,node_reasoning_output toneRose
class node_operator,node_pdf,node_tabular toneIndigo
```

## Components

### Figure Extraction

- [Extraction workflow](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/main.py): `data/ImageTextExtraction/LMBased/main.py`
- [PDF rendering](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/pdf2img.py): `data/ImageTextExtraction/LMBased/pdf2img.py`
- [Layout and caption analysis](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/layout_async.py): `data/ImageTextExtraction/LMBased/layout_async.py`
- [Figure cropping](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/crop.py): `data/ImageTextExtraction/LMBased/crop.py`
- [Analysis prompts](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/prompts.py): `data/ImageTextExtraction/LMBased/prompts.py`
- [Page context helpers](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/tools.py): `data/ImageTextExtraction/LMBased/tools.py`
- [Inline reference extraction](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/inline_async.py): `data/ImageTextExtraction/LMBased/inline_async.py`
- [Result merging](https://github.com/wenchuan-zhang/patho-r1/blob/main/data/ImageTextExtraction/LMBased/alignment.py): `data/ImageTextExtraction/LMBased/alignment.py`

### Dataset Preparation

- [WebDataset shards](https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/dataset_process/quit_to_web_dataset.py): `clip/dataset_process/quit_to_web_dataset.py`

### Patho-CLIP Evaluation

- **Patho-CLIP model**
- [Zero-shot evaluation](https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/zero_shot_evaluation.py): `clip/val/zero_shot_evaluation.py`
- [Retrieval evaluation](https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/retrieval_evaluation.py): `clip/val/retrieval_evaluation.py`
- [Few-shot evaluation](https://github.com/wenchuan-zhang/patho-r1/blob/main/clip/val/few_shot_evaluation.py): `clip/val/few_shot_evaluation.py`
- **Evaluation metrics**

### Pathology Reasoning

- [Patho-R1 inference](https://github.com/wenchuan-zhang/patho-r1/blob/main/README.md): `README.md`
- **Reasoned answer**

### Other components

- **Dataset operator**
- **PDF document**
- **Qwen-compatible API**
- **Image-caption table**
- **Pathology user**

## Connections

- Dataset operator → Extraction workflow: runs (evidence: `data/ImageTextExtraction/LMBased/main.py`)
- PDF document → Extraction workflow: provides input (evidence: `data/ImageTextExtraction/LMBased/main.py`)
- Extraction workflow → Layout and caption analysis: runs analysis (evidence: `data/ImageTextExtraction/LMBased/main.py`)
- Layout and caption analysis → Qwen-compatible API: requests analysis (evidence: `data/ImageTextExtraction/LMBased/layout_async.py`)
- Layout and caption analysis → Analysis prompts: uses prompts (evidence: `data/ImageTextExtraction/LMBased/layout_async.py`)
- Layout and caption analysis → Page context helpers: uses helpers (evidence: `data/ImageTextExtraction/LMBased/layout_async.py`)
- Extraction workflow → Inline reference extraction: runs extraction (evidence: `data/ImageTextExtraction/LMBased/main.py`)
- Inline reference extraction → Layout and caption analysis: sends requests (evidence: `data/ImageTextExtraction/LMBased/inline_async.py`)
- Inline reference extraction → Analysis prompts: uses prompts (evidence: `data/ImageTextExtraction/LMBased/inline_async.py`)
- Extraction workflow → Result merging: merges results (evidence: `data/ImageTextExtraction/LMBased/main.py`)
- Dataset operator → WebDataset shards: creates shards (evidence: `clip/dataset_process/quit_to_web_dataset.py`)
- Image-caption table → WebDataset shards: provides captions (evidence: `clip/dataset_process/quit_to_web_dataset.py`)
- WebDataset shards → Zero-shot evaluation: supplies dataset
- Pathology user → Patho-R1 inference: asks question (evidence: `README.md`)
- Patho-R1 inference → Reasoned answer: generates answer (evidence: `README.md`)
- Pathology user → Zero-shot evaluation: evaluates model (evidence: `clip/val/zero_shot_evaluation.py`)
- Pathology user → Retrieval evaluation: evaluates retrieval (evidence: `clip/val/retrieval_evaluation.py`)
- Pathology user → Few-shot evaluation: evaluates few-shot
- Zero-shot evaluation → Patho-CLIP model: encodes images (evidence: `clip/val/zero_shot_evaluation.py`)
- Retrieval evaluation → Patho-CLIP model: encodes pairs (evidence: `clip/val/retrieval_evaluation.py`)
- Zero-shot evaluation → Evaluation metrics: computes accuracy (evidence: `clip/val/zero_shot_evaluation.py`)
- Retrieval evaluation → Evaluation metrics: computes recall (evidence: `clip/val/retrieval_evaluation.py`)