# oceanflowlab/quatrope architecture

> Architecture diagram of the GitHub repository oceanflowlab/quatrope, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oceanflowlab/quatrope
- Repository: https://github.com/oceanflowlab/quatrope

## Overview

QuatRoPE is a PyTorch 3D scene-language model for object grounding and spatial reasoning. Its main workflow prepares scene annotations and features, loads paired scene/question data, combines object and visual features with 3D positions, and generates answers through a LLaMA-based model. Training and evaluation cover grounding and language benchmarks; the README also describes ground-truth and Mask3D segmentation stages and an ASR benchmark. The graph reflects documented capabilities, but several links among unsampled model and preprocessing modules are intentionally omitted.

Main entry points: [`tasks/train.py`](https://github.com/oceanflowlab/quatrope/blob/main/tasks/train.py) (Train And Infer), [`models/graph3dllm.py`](https://github.com/oceanflowlab/quatrope/blob/main/models/graph3dllm.py) (Scene-Language Model), [`preprocess/prepare_mask3d_data.py`](https://github.com/oceanflowlab/quatrope/blob/main/preprocess/prepare_mask3d_data.py) (Scene Preparation), [`dataset/base_dataset.py`](https://github.com/oceanflowlab/quatrope/blob/main/dataset/base_dataset.py) (Scene Dataset)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Scene Data"]
  node_rawdata["Scene Annotations"]
  node_preprocess["Scene Preparation"]
  node_annoprep["Task Annotations"]
  node_dataset["Scene Dataset<br/>[base_dataset.py]"]
  node_loaders["Data Loaders<br/>[dataloader.py]"]
end

subgraph group_model["Spatial Model"]
  node_graphmodel["Scene-Language Model<br/>[graph3dllm.py]"]
  node_position["3D Position Encoding"]
  node_llama["LLaMA Backbone<br/>[modeling_llama.py]"]
  node_moe["MoE LoRA<br/>[layer.py]"]
  node_prompts["Task Prompts<br/>[prompts.py]"]
end

subgraph group_runtime["Training Runtime"]
  node_training["Train And Infer<br/>[train.py]"]
  node_generation["Generation Support<br/>[utils.py]"]
  node_checkpoint["Model Checkpoint"]
end

subgraph group_evaluation["Evaluation"]
  node_benchmarks["Benchmark Inputs<br/>[gen_asr.py]"]
  node_metrics["Task Metrics<br/>[eval.py]"]
  node_results["Evaluation Results"]
end

node_engineer(("Researcher"))
node_scenes["3D Scene Files"]
node_llmservice[("Pretrained LLaMA")]

node_engineer -->|"starts run"| node_training
node_rawdata -->|"provides scenes"| node_preprocess
node_scenes -->|"provides geometry"| node_preprocess
node_preprocess -->|"prepares features"| node_dataset
node_rawdata -->|"provides labels"| node_annoprep
node_annoprep -->|"prepares annotations"| node_dataset
node_training -->|"sets up"| node_loaders
node_loaders -->|"loads batches"| node_dataset
node_training -->|"trains or evaluates"| node_graphmodel
node_graphmodel -->|"encodes positions"| node_position
node_graphmodel -->|"uses backbone"| node_llama
node_llmservice -->|"initializes weights"| node_llama
node_graphmodel -.->|"uses adapters"| node_moe
node_prompts -->|"guides tasks"| node_graphmodel
node_graphmodel -->|"generates answers"| node_generation
node_training -->|"scores predictions"| node_metrics
node_benchmarks -.->|"supplies ASR data"| node_training
node_training -->|"saves weights"| node_checkpoint
node_training -->|"reports metrics"| node_results
node_metrics -->|"computes scores"| node_results
node_engineer -->|"reviews"| node_results

click node_preprocess "https://github.com/oceanflowlab/quatrope/blob/main/preprocess/prepare_mask3d_data.py"
click node_annoprep "https://github.com/oceanflowlab/quatrope/blob/main/preprocess/prepare_objalign_annos.py"
click node_dataset "https://github.com/oceanflowlab/quatrope/blob/main/dataset/base_dataset.py"
click node_loaders "https://github.com/oceanflowlab/quatrope/blob/main/dataset/dataloader.py"
click node_graphmodel "https://github.com/oceanflowlab/quatrope/blob/main/models/graph3dllm.py"
click node_position "https://github.com/oceanflowlab/quatrope/blob/main/models/position_embedding.py"
click node_llama "https://github.com/oceanflowlab/quatrope/blob/main/models/modeling_llama.py"
click node_moe "https://github.com/oceanflowlab/quatrope/blob/main/models/moe/layer.py"
click node_prompts "https://github.com/oceanflowlab/quatrope/blob/main/prompts/prompts.py"
click node_training "https://github.com/oceanflowlab/quatrope/blob/main/tasks/train.py"
click node_generation "https://github.com/oceanflowlab/quatrope/blob/main/generation/utils.py"
click node_benchmarks "https://github.com/oceanflowlab/quatrope/blob/main/ASR/gen_asr.py"
click node_metrics "https://github.com/oceanflowlab/quatrope/blob/main/utils/eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_rawdata,node_preprocess,node_annoprep,node_dataset,node_loaders toneBlue
class node_graphmodel,node_position,node_llama,node_moe,node_prompts,node_llmservice toneAmber
class node_training,node_generation,node_checkpoint toneMint
class node_benchmarks,node_metrics,node_results toneRose
class node_engineer,node_scenes toneIndigo
```

## Components

### Scene Data

- **Scene Annotations**
- [Scene Preparation](https://github.com/oceanflowlab/quatrope/blob/main/preprocess/prepare_mask3d_data.py): `preprocess/prepare_mask3d_data.py`
- [Task Annotations](https://github.com/oceanflowlab/quatrope/blob/main/preprocess/prepare_objalign_annos.py): `preprocess/prepare_objalign_annos.py`
- [Scene Dataset](https://github.com/oceanflowlab/quatrope/blob/main/dataset/base_dataset.py): `dataset/base_dataset.py`
- [Data Loaders](https://github.com/oceanflowlab/quatrope/blob/main/dataset/dataloader.py): `dataset/dataloader.py`

### Spatial Model

- [Scene-Language Model](https://github.com/oceanflowlab/quatrope/blob/main/models/graph3dllm.py): `models/graph3dllm.py`
- [3D Position Encoding](https://github.com/oceanflowlab/quatrope/blob/main/models/position_embedding.py): `models/position_embedding.py`
- [LLaMA Backbone](https://github.com/oceanflowlab/quatrope/blob/main/models/modeling_llama.py): `models/modeling_llama.py`
- [MoE LoRA](https://github.com/oceanflowlab/quatrope/blob/main/models/moe/layer.py): `models/moe/layer.py`
- [Task Prompts](https://github.com/oceanflowlab/quatrope/blob/main/prompts/prompts.py): `prompts/prompts.py`

### Training Runtime

- [Train And Infer](https://github.com/oceanflowlab/quatrope/blob/main/tasks/train.py): `tasks/train.py`
- [Generation Support](https://github.com/oceanflowlab/quatrope/blob/main/generation/utils.py): `generation/utils.py`
- **Model Checkpoint**

### Evaluation

- [Benchmark Inputs](https://github.com/oceanflowlab/quatrope/blob/main/ASR/gen_asr.py): `ASR/gen_asr.py`
- [Task Metrics](https://github.com/oceanflowlab/quatrope/blob/main/utils/eval.py): `utils/eval.py`
- **Evaluation Results**

### Other components

- **Researcher**
- **3D Scene Files**
- **Pretrained LLaMA**

## Connections

- Researcher → Train And Infer: starts run
- Scene Annotations → Scene Preparation: provides scenes
- 3D Scene Files → Scene Preparation: provides geometry
- Scene Preparation → Scene Dataset: prepares features
- Scene Annotations → Task Annotations: provides labels
- Task Annotations → Scene Dataset: prepares annotations
- Train And Infer → Data Loaders: sets up
- Data Loaders → Scene Dataset: loads batches
- Train And Infer → Scene-Language Model: trains or evaluates
- Scene-Language Model → 3D Position Encoding: encodes positions
- Scene-Language Model → LLaMA Backbone: uses backbone
- Pretrained LLaMA → LLaMA Backbone: initializes weights
- Scene-Language Model → MoE LoRA: uses adapters
- Task Prompts → Scene-Language Model: guides tasks
- Scene-Language Model → Generation Support: generates answers
- Train And Infer → Task Metrics: scores predictions
- Benchmark Inputs → Train And Infer: supplies ASR data
- Train And Infer → Model Checkpoint: saves weights
- Train And Infer → Evaluation Results: reports metrics
- Task Metrics → Evaluation Results: computes scores
- Researcher → Evaluation Results: reviews