How wenchuan-zhang/patho-r1 works
Patho-R1 is presented as a pathology vision-language model and reasoning system. The checked-in runtime evidence chiefly covers document-to-figure dataset extraction and Patho-CLIP dataset preparation/evaluation; the README also documents model inference, but the model implementation is not present in the supplied tree. Edges are limited to relationships shown in excerpts or explicitly documented; unsampled areas are not treated as inspected.
Main entry points: main.py Extraction workflow · layout_async.py Layout and caption analysis · zero_shot_evaluation.py Zero-shot evaluation · inline_async.py Inline reference extraction
Components 21
Figure Extraction
- Extraction workflow
data/ImageTextExtraction/LMBased/main.py - PDF rendering
data/ImageTextExtraction/LMBased/pdf2img.py - Layout and caption analysis
data/ImageTextExtraction/LMBased/layout_async.py - Figure cropping
data/ImageTextExtraction/LMBased/crop.py - Analysis prompts
data/ImageTextExtraction/LMBased/prompts.py - Page context helpers
data/ImageTextExtraction/LMBased/tools.py - Inline reference extraction
data/ImageTextExtraction/LMBased/inline_async.py - Result merging
data/ImageTextExtraction/LMBased/alignment.py
Dataset Preparation
- WebDataset shards
clip/dataset_process/quit_to_web_dataset.py
Patho-CLIP Evaluation
- Patho-CLIP model
- Zero-shot evaluation
clip/val/zero_shot_evaluation.py - Retrieval evaluation
clip/val/retrieval_evaluation.py - Few-shot evaluation
clip/val/few_shot_evaluation.py - Evaluation metrics
Pathology Reasoning
- Patho-R1 inference
README.md - Reasoned answer
Other components
- Dataset operator
- PDF document
- Qwen-compatible API
- Image-caption table
- Pathology user
Connections 22
- Dataset operator → Extraction workflow: runs
data/ImageTextExtraction/LMBased/main.py - PDF document → Extraction workflow: provides input
data/ImageTextExtraction/LMBased/main.py - Extraction workflow → Layout and caption analysis: runs analysis
data/ImageTextExtraction/LMBased/main.py - Layout and caption analysis → Qwen-compatible API: requests analysis
data/ImageTextExtraction/LMBased/layout_async.py - Layout and caption analysis → Analysis prompts: uses prompts
data/ImageTextExtraction/LMBased/layout_async.py - Layout and caption analysis → Page context helpers: uses helpers
data/ImageTextExtraction/LMBased/layout_async.py - Extraction workflow → Inline reference extraction: runs extraction
data/ImageTextExtraction/LMBased/main.py - Inline reference extraction → Layout and caption analysis: sends requests
data/ImageTextExtraction/LMBased/inline_async.py - Inline reference extraction → Analysis prompts: uses prompts
data/ImageTextExtraction/LMBased/inline_async.py - Extraction workflow → Result merging: merges results
data/ImageTextExtraction/LMBased/main.py - Dataset operator → WebDataset shards: creates shards
clip/dataset_process/quit_to_web_dataset.py - Image-caption table → WebDataset shards: provides captions
clip/dataset_process/quit_to_web_dataset.py - WebDataset shards → Zero-shot evaluation: supplies dataset
- Pathology user → Patho-R1 inference: asks question
README.md - Patho-R1 inference → Reasoned answer: generates answer
README.md - Pathology user → Zero-shot evaluation: evaluates model
clip/val/zero_shot_evaluation.py - Pathology user → Retrieval evaluation: evaluates retrieval
clip/val/retrieval_evaluation.py - Pathology user → Few-shot evaluation: evaluates few-shot
- Zero-shot evaluation → Patho-CLIP model: encodes images
clip/val/zero_shot_evaluation.py - Retrieval evaluation → Patho-CLIP model: encodes pairs
clip/val/retrieval_evaluation.py - Zero-shot evaluation → Evaluation metrics: computes accuracy
clip/val/zero_shot_evaluation.py - Retrieval evaluation → Evaluation metrics: computes recall
clip/val/retrieval_evaluation.py