# jorwnpay/petrg-3d architecture

> Architecture diagram of the GitHub repository jorwnpay/petrg-3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jorwnpay/petrg-3d
- Repository: https://github.com/jorwnpay/petrg-3d

## Overview

PETRG-3D trains and runs a multimodal model that turns paired 3D PET/CT scans into structured Chinese findings. Training pairs processed volumes with report text and optimizes the model; inference produces predictions in a resumable CSV. Evaluation can clean those reports, score text similarity, or extract anatomical labels through an OpenAI-compatible LLM and calculate clinical efficacy scores. README documentation establishes the evaluation stages; the inference implementation was not sampled, so its internal call wiring is omitted.

Main entry points: [`src/train.py`](https://github.com/jorwnpay/petrg-3d/blob/main/src/train.py) (Training entrypoint), [`src/test.py`](https://github.com/jorwnpay/petrg-3d/blob/main/src/test.py) (Inference entrypoint), [`src/Model/my_embedding_layer.py`](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/my_embedding_layer.py) (Multimodal embeddings), [`src/Model/PETRG_3D.py`](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/PETRG_3D.py) (PETRG3D)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_train_dataset["Training dataset"]
  node_test_dataset["Inference dataset"]
  node_train_args["Training arguments<br/>[default.py]"]
  node_test_args["Inference arguments<br/>[default.py]"]
end

subgraph group_model["PETRG-3D model"]
  node_model["PETRG3D<br/>[PETRG_3D.py]"]
  node_embeddings["Multimodal embeddings"]
  node_vision["3D ViT backbone<br/>[vit_3d.py]"]
  node_position["Position encoding"]
  node_helpers["Model helpers<br/>[helpers.py]"]
end

subgraph group_runtime["Training and inference"]
  node_trainer["Training entrypoint<br/>[train.py]"]
  node_inference["Inference entrypoint<br/>[test.py]"]
end

subgraph group_evaluation["Report evaluation"]
  node_clean["Report cleaning"]
  node_nlg["Chinese NLG metrics"]
  node_bleu["Local BLEU<br/>[nmt_bleu.py]"]
  node_label_extract["PET/CT label extraction"]
  node_ces["Clinical efficacy scores<br/>[cal_petrg_ces.py]"]
end

node_researcher(("Researcher"))
node_volumes_reports["PET/CT and reports"]
node_llm["Chat LLM"]
node_checkpoint["Visual checkpoint"]
node_predictions["Prediction CSV"]
node_clean_reports["Clean report JSONs"]
node_api["LLM API"]
node_labels["Predicted labels"]
node_ground_truth["Ground-truth labels"]
node_metrics["Metric CSVs"]

node_researcher -->|"configures and launches"| node_trainer
node_researcher -->|"configures and launches"| node_inference
node_volumes_reports -->|"loads and preprocesses"| node_train_dataset
node_volumes_reports -->|"loads for inference"| node_test_dataset
node_trainer -->|"parses arguments"| node_train_args
node_trainer -->|"constructs"| node_train_dataset
node_trainer -->|"constructs and trains"| node_model
node_trainer -.->|"loads optional weights"| node_checkpoint
node_model -->|"builds and calls"| node_embeddings
node_model -->|"wraps with LoRA and calls"| node_llm
node_embeddings -->|"encodes volumes"| node_vision
node_embeddings -.->|"loads visual weights"| node_checkpoint
node_vision -->|"uses encodings"| node_position
node_embeddings -->|"uses components"| node_helpers
node_researcher -->|"provides inference inputs"| node_predictions
node_inference -->|"parses arguments"| node_test_args
node_inference -->|"constructs"| node_test_dataset
node_inference -->|"generates reports"| node_model
node_inference -->|"writes predictions"| node_predictions
node_predictions -->|"reads"| node_clean
node_clean -->|"writes cleaned reports"| node_clean_reports
node_predictions -->|"supplies references and predictions"| node_nlg
node_nlg -->|"computes BLEU"| node_bleu
node_clean_reports -->|"supplies findings"| node_label_extract
node_label_extract -->|"requests structured labels"| node_api
node_label_extract -->|"writes label JSONs"| node_labels
node_labels -->|"supplies predictions"| node_ces
node_ground_truth -->|"supplies reference labels"| node_ces
node_ces -->|"writes scores"| node_metrics
node_nlg -->|"reports text metrics"| node_metrics

click node_train_dataset "https://github.com/jorwnpay/petrg-3d/blob/main/src/Dataset/petct_dataset_train.py"
click node_test_dataset "https://github.com/jorwnpay/petrg-3d/blob/main/src/Dataset/petct_dataset_test.py"
click node_train_args "https://github.com/jorwnpay/petrg-3d/blob/main/src/args/train/default.py"
click node_test_args "https://github.com/jorwnpay/petrg-3d/blob/main/src/args/test/default.py"
click node_trainer "https://github.com/jorwnpay/petrg-3d/blob/main/src/train.py"
click node_inference "https://github.com/jorwnpay/petrg-3d/blob/main/src/test.py"
click node_model "https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/PETRG_3D.py"
click node_embeddings "https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/my_embedding_layer.py"
click node_vision "https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/vit_3d.py"
click node_position "https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/position_encoding.py"
click node_helpers "https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/helpers.py"
click node_clean "https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/clean_csv_report.py"
click node_nlg "https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/CN_nlg_evaluation.py"
click node_bleu "https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/local_metrics/bleu/nmt_bleu.py"
click node_label_extract "https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/extract_pet_ct_labels.py"
click node_ces "https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/cal_petrg_ces.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_dataset,node_test_dataset,node_train_args,node_test_args toneBlue
class node_model,node_embeddings,node_vision,node_position,node_helpers toneAmber
class node_trainer,node_inference,node_api toneMint
class node_clean,node_nlg,node_bleu,node_label_extract,node_ces toneRose
class node_researcher,node_volumes_reports,node_llm,node_checkpoint,node_predictions,node_clean_reports,node_labels,node_ground_truth,node_metrics toneIndigo
```

## Components

### Data preparation

- [Training dataset](https://github.com/jorwnpay/petrg-3d/blob/main/src/Dataset/petct_dataset_train.py): `src/Dataset/petct_dataset_train.py`
- [Inference dataset](https://github.com/jorwnpay/petrg-3d/blob/main/src/Dataset/petct_dataset_test.py): `src/Dataset/petct_dataset_test.py`
- [Training arguments](https://github.com/jorwnpay/petrg-3d/blob/main/src/args/train/default.py): `src/args/train/default.py`
- [Inference arguments](https://github.com/jorwnpay/petrg-3d/blob/main/src/args/test/default.py): `src/args/test/default.py`

### PETRG-3D model

- [PETRG3D](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/PETRG_3D.py): `src/Model/PETRG_3D.py`
- [Multimodal embeddings](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/my_embedding_layer.py): `src/Model/my_embedding_layer.py`
- [3D ViT backbone](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/vit_3d.py): `src/Model/vit_3d.py`
- [Position encoding](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/position_encoding.py): `src/Model/position_encoding.py`
- [Model helpers](https://github.com/jorwnpay/petrg-3d/blob/main/src/Model/helpers.py): `src/Model/helpers.py`

### Training and inference

- [Training entrypoint](https://github.com/jorwnpay/petrg-3d/blob/main/src/train.py): `src/train.py`
- [Inference entrypoint](https://github.com/jorwnpay/petrg-3d/blob/main/src/test.py): `src/test.py`

### Report evaluation

- [Report cleaning](https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/clean_csv_report.py): `evaluation/clean_csv_report.py`
- [Chinese NLG metrics](https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/CN_nlg_evaluation.py): `evaluation/CN_nlg_evaluation.py`
- [Local BLEU](https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/local_metrics/bleu/nmt_bleu.py): `evaluation/local_metrics/bleu/nmt_bleu.py`
- [PET/CT label extraction](https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/extract_pet_ct_labels.py): `evaluation/extract_pet_ct_labels.py`
- [Clinical efficacy scores](https://github.com/jorwnpay/petrg-3d/blob/main/evaluation/cal_petrg_ces.py): `evaluation/cal_petrg_ces.py`

### Other components

- **Researcher**
- **PET/CT and reports**
- **Chat LLM**
- **Visual checkpoint**
- **Prediction CSV**
- **Clean report JSONs**
- **LLM API**
- **Predicted labels**
- **Ground-truth labels**
- **Metric CSVs**

## Connections

- Researcher → Training entrypoint: configures and launches
- Researcher → Inference entrypoint: configures and launches
- PET/CT and reports → Training dataset: loads and preprocesses
- PET/CT and reports → Inference dataset: loads for inference
- Training entrypoint → Training arguments: parses arguments
- Training entrypoint → Training dataset: constructs
- Training entrypoint → PETRG3D: constructs and trains
- Training entrypoint → Visual checkpoint: loads optional weights
- PETRG3D → Multimodal embeddings: builds and calls
- PETRG3D → Chat LLM: wraps with LoRA and calls
- Multimodal embeddings → 3D ViT backbone: encodes volumes
- Multimodal embeddings → Visual checkpoint: loads visual weights
- 3D ViT backbone → Position encoding: uses encodings
- Multimodal embeddings → Model helpers: uses components
- Researcher → Prediction CSV: provides inference inputs
- Inference entrypoint → Inference arguments: parses arguments
- Inference entrypoint → Inference dataset: constructs
- Inference entrypoint → PETRG3D: generates reports
- Inference entrypoint → Prediction CSV: writes predictions
- Prediction CSV → Report cleaning: reads
- Report cleaning → Clean report JSONs: writes cleaned reports
- Prediction CSV → Chinese NLG metrics: supplies references and predictions
- Chinese NLG metrics → Local BLEU: computes BLEU
- Clean report JSONs → PET/CT label extraction: supplies findings
- PET/CT label extraction → LLM API: requests structured labels
- PET/CT label extraction → Predicted labels: writes label JSONs
- Predicted labels → Clinical efficacy scores: supplies predictions
- Ground-truth labels → Clinical efficacy scores: supplies reference labels
- Clinical efficacy scores → Metric CSVs: writes scores
- Chinese NLG metrics → Metric CSVs: reports text metrics