# oneoflzx/dtir architecture

> Architecture diagram of the GitHub repository oneoflzx/dtir, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oneoflzx/dtir
- Repository: https://github.com/oneoflzx/dtir

## Overview

RT2I is a reasoning-based text-to-image retrieval system: users provide query metadata and images, generate structured digital-twin representations, then retrieve and rank matching images using configurable LLMs. The graph includes the documented object detection, segmentation, depth, and captioning capabilities. Some internal calls are only indicated by the source index or README, so those links are marked as unverified rather than asserted as confirmed calls.

Main entry points: [`digital_twins_pipeline/image_to_digital_pipeline.py`](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/image_to_digital_pipeline.py) (Twin pipeline), [`retrieval_pipeline/llm_retrieval.py`](https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py) (Retrieval runner), [`digital_twins_pipeline/digital_twins_generation.py`](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/digital_twins_generation.py) (Generation runner), [`models/llm_api.py`](https://github.com/oneoflzx/dtir/blob/main/models/llm_api.py) (LLM API wrapper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Twin generation"]
  node_generation_entry["Generation runner"]
  node_twin_pipeline["Twin pipeline"]
  node_validator["Twin validator<br/>[check_dt_valid.py]"]
end

subgraph group_models["Vision models"]
  node_owl["OWL-ViT detection<br/>[owl_vit.py]"]
  node_sam["SAM segmentation<br/>[sam.py]"]
  node_depth["Depth estimation<br/>[depth_any_thing.py]"]
  node_qwen["Qwen-VL semantics<br/>[qwenvl.py]"]
  node_internvl["InternVL captions<br/>[internvl.py]"]
end

subgraph group_retrieval["Retrieval and ranking"]
  node_retrieval_entry["Retrieval runner<br/>[llm_retrieval.py]"]
  node_twin_loader["Twin loader"]
  node_coarse_score["Relevance scoring<br/>[llm_retrieval.py]"]
  node_object_retrieval["Object retrieval<br/>[llm_retrieval.py]"]
  node_ranker["Comparative ranking<br/>[llm_retrieval.py]"]
  node_llm_api["LLM API wrapper<br/>[llm_api.py]"]
end

subgraph group_shared["Shared data utilities"]
  node_utils["Image and mask utilities<br/>[utils.py]"]
  node_logging["Logging<br/>[log.py]"]
end

node_user(("Researcher"))
node_query_images["Queries and images<br/>[query_info.json]"]
node_twin_files["Digital-twin files"]
node_llm_service{{"LLM provider"}}
node_results["Ranked results"]

node_user -->|"provides queries"| node_query_images
node_query_images -->|"supplies inputs"| node_generation_entry
node_generation_entry -->|"runs generation"| node_twin_pipeline
node_generation_entry -->|"checks output"| node_validator
node_validator -->|"returns invalid images"| node_generation_entry
node_twin_pipeline -.->|"uses detection"| node_owl
node_twin_pipeline -.->|"uses masks"| node_sam
node_twin_pipeline -.->|"estimates depth"| node_depth
node_twin_pipeline -.->|"uses vision language"| node_qwen
node_twin_pipeline -->|"extracts captions"| node_internvl
node_twin_pipeline -->|"writes representations"| node_twin_files
node_user -->|"submits query"| node_retrieval_entry
node_retrieval_entry -.->|"loads representations"| node_twin_loader
node_twin_loader -.->|"reads representations"| node_twin_files
node_retrieval_entry -->|"scores images"| node_coarse_score
node_retrieval_entry -.->|"retrieves objects"| node_object_retrieval
node_retrieval_entry -->|"ranks candidates"| node_ranker
node_coarse_score -->|"requests scoring"| node_llm_api
node_ranker -->|"requests ranking"| node_llm_api
node_object_retrieval -.->|"requests extraction"| node_llm_api
node_llm_api -->|"sends prompts"| node_llm_service
node_retrieval_entry -->|"writes results"| node_results
node_twin_loader -->|"decodes masks"| node_utils
node_twin_pipeline -.->|"uses helpers"| node_utils
node_generation_entry -->|"logs progress"| node_logging

click node_query_images "https://github.com/oneoflzx/dtir/blob/main/data/RT2I/query_info.json"
click node_generation_entry "https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/digital_twins_generation.py"
click node_twin_pipeline "https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/image_to_digital_pipeline.py"
click node_validator "https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/check_dt_valid.py"
click node_owl "https://github.com/oneoflzx/dtir/blob/main/models/owl_vit.py"
click node_sam "https://github.com/oneoflzx/dtir/blob/main/models/sam.py"
click node_depth "https://github.com/oneoflzx/dtir/blob/main/models/depth_any_thing.py"
click node_qwen "https://github.com/oneoflzx/dtir/blob/main/models/qwenvl.py"
click node_internvl "https://github.com/oneoflzx/dtir/blob/main/models/internvl.py"
click node_retrieval_entry "https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py"
click node_twin_loader "https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/digital_twins_loader.py"
click node_coarse_score "https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py"
click node_object_retrieval "https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py"
click node_ranker "https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py"
click node_llm_api "https://github.com/oneoflzx/dtir/blob/main/models/llm_api.py"
click node_utils "https://github.com/oneoflzx/dtir/blob/main/utils/utils.py"
click node_logging "https://github.com/oneoflzx/dtir/blob/main/utils/log.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_generation_entry,node_twin_pipeline,node_validator toneBlue
class node_owl,node_sam,node_depth,node_qwen,node_internvl toneAmber
class node_retrieval_entry,node_twin_loader,node_coarse_score,node_object_retrieval,node_ranker,node_llm_api toneMint
class node_utils,node_logging toneRose
class node_user,node_twin_files,node_llm_service,node_results toneIndigo
class node_query_images toneTeal
```

## Components

### Twin generation

- [Generation runner](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/digital_twins_generation.py): `digital_twins_pipeline/digital_twins_generation.py`
- [Twin pipeline](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/image_to_digital_pipeline.py): `digital_twins_pipeline/image_to_digital_pipeline.py`
- [Twin validator](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/check_dt_valid.py): `digital_twins_pipeline/check_dt_valid.py`

### Vision models

- [OWL-ViT detection](https://github.com/oneoflzx/dtir/blob/main/models/owl_vit.py): `models/owl_vit.py`
- [SAM segmentation](https://github.com/oneoflzx/dtir/blob/main/models/sam.py): `models/sam.py`
- [Depth estimation](https://github.com/oneoflzx/dtir/blob/main/models/depth_any_thing.py): `models/depth_any_thing.py`
- [Qwen-VL semantics](https://github.com/oneoflzx/dtir/blob/main/models/qwenvl.py): `models/qwenvl.py`
- [InternVL captions](https://github.com/oneoflzx/dtir/blob/main/models/internvl.py): `models/internvl.py`

### Retrieval and ranking

- [Retrieval runner](https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py): `retrieval_pipeline/llm_retrieval.py`
- [Twin loader](https://github.com/oneoflzx/dtir/blob/main/digital_twins_pipeline/digital_twins_loader.py): `digital_twins_pipeline/digital_twins_loader.py`
- [Relevance scoring](https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py): `retrieval_pipeline/llm_retrieval.py`
- [Object retrieval](https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py): `retrieval_pipeline/llm_retrieval.py`
- [Comparative ranking](https://github.com/oneoflzx/dtir/blob/main/retrieval_pipeline/llm_retrieval.py): `retrieval_pipeline/llm_retrieval.py`
- [LLM API wrapper](https://github.com/oneoflzx/dtir/blob/main/models/llm_api.py): `models/llm_api.py`

### Shared data utilities

- [Image and mask utilities](https://github.com/oneoflzx/dtir/blob/main/utils/utils.py): `utils/utils.py`
- [Logging](https://github.com/oneoflzx/dtir/blob/main/utils/log.py): `utils/log.py`

### Other components

- **Researcher**
- [Queries and images](https://github.com/oneoflzx/dtir/blob/main/data/RT2I/query_info.json): `data/RT2I/query_info.json`
- **Digital-twin files**
- **LLM provider**
- **Ranked results**

## Connections

- Researcher → Queries and images: provides queries (evidence: `README.md`)
- Queries and images → Generation runner: supplies inputs (evidence: `digital_twins_pipeline/digital_twins_generation.py`)
- Generation runner → Twin pipeline: runs generation (evidence: `digital_twins_pipeline/digital_twins_generation.py`)
- Generation runner → Twin validator: checks output (evidence: `digital_twins_pipeline/digital_twins_generation.py`)
- Twin validator → Generation runner: returns invalid images (evidence: `digital_twins_pipeline/digital_twins_generation.py`)
- Twin pipeline → OWL-ViT detection: uses detection (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Twin pipeline → SAM segmentation: uses masks (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Twin pipeline → Depth estimation: estimates depth (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Twin pipeline → Qwen-VL semantics: uses vision language (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Twin pipeline → InternVL captions: extracts captions (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Twin pipeline → Digital-twin files: writes representations (evidence: `README.md`)
- Researcher → Retrieval runner: submits query (evidence: `README.md`)
- Retrieval runner → Twin loader: loads representations (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Twin loader → Digital-twin files: reads representations
- Retrieval runner → Relevance scoring: scores images (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Retrieval runner → Object retrieval: retrieves objects (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Retrieval runner → Comparative ranking: ranks candidates (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Relevance scoring → LLM API wrapper: requests scoring (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Comparative ranking → LLM API wrapper: requests ranking (evidence: `retrieval_pipeline/llm_retrieval.py`)
- Object retrieval → LLM API wrapper: requests extraction (evidence: `retrieval_pipeline/llm_retrieval.py`)
- LLM API wrapper → LLM provider: sends prompts (evidence: `models/llm_api.py`)
- Retrieval runner → Ranked results: writes results (evidence: `README.md`)
- Twin loader → Image and mask utilities: decodes masks (evidence: `digital_twins_pipeline/digital_twins_loader.py`)
- Twin pipeline → Image and mask utilities: uses helpers (evidence: `digital_twins_pipeline/image_to_digital_pipeline.py`)
- Generation runner → Logging: logs progress (evidence: `digital_twins_pipeline/digital_twins_generation.py`)
