How oneoflzx/dtir works
RT2I is a reasoning-based text-to-image retrieval system: users provide query metadata and images, generate structured digital-twin representations, then retrieve and rank matching images using configurable LLMs. The graph includes the documented object detection, segmentation, depth, and captioning capabilities. Some internal calls are only indicated by the source index or README, so those links are marked as unverified rather than asserted as confirmed calls.
Main entry points: image_to_digital_pipeline.py Twin pipeline · llm_retrieval.py Retrieval runner · digital_twins_generation.py Generation runner · llm_api.py LLM API wrapper
Components 21
Twin generation
- Generation runner
digital_twins_pipeline/digital_twins_generation.py - Twin pipeline
digital_twins_pipeline/image_to_digital_pipeline.py - Twin validator
digital_twins_pipeline/check_dt_valid.py
Vision models
- OWL-ViT detection
models/owl_vit.py - SAM segmentation
models/sam.py - Depth estimation
models/depth_any_thing.py - Qwen-VL semantics
models/qwenvl.py - InternVL captions
models/internvl.py
Retrieval and ranking
- Retrieval runner
retrieval_pipeline/llm_retrieval.py - Twin loader
digital_twins_pipeline/digital_twins_loader.py - Relevance scoring
retrieval_pipeline/llm_retrieval.py - Object retrieval
retrieval_pipeline/llm_retrieval.py - Comparative ranking
retrieval_pipeline/llm_retrieval.py - LLM API wrapper
models/llm_api.py
Shared data utilities
- Image and mask utilities
utils/utils.py - Logging
utils/log.py
Other components
- Researcher
- Queries and images
data/RT2I/query_info.json - Digital-twin files
- LLM provider
- Ranked results
Connections 25
- Researcher → Queries and images: provides queries
README.md - Queries and images → Generation runner: supplies inputs
digital_twins_pipeline/digital_twins_generation.py - Generation runner → Twin pipeline: runs generation
digital_twins_pipeline/digital_twins_generation.py - Generation runner → Twin validator: checks output
digital_twins_pipeline/digital_twins_generation.py - Twin validator → Generation runner: returns invalid images
digital_twins_pipeline/digital_twins_generation.py - Twin pipeline → OWL-ViT detection: uses detection
digital_twins_pipeline/image_to_digital_pipeline.py - Twin pipeline → SAM segmentation: uses masks
digital_twins_pipeline/image_to_digital_pipeline.py - Twin pipeline → Depth estimation: estimates depth
digital_twins_pipeline/image_to_digital_pipeline.py - Twin pipeline → Qwen-VL semantics: uses vision language
digital_twins_pipeline/image_to_digital_pipeline.py - Twin pipeline → InternVL captions: extracts captions
digital_twins_pipeline/image_to_digital_pipeline.py - Twin pipeline → Digital-twin files: writes representations
README.md - Researcher → Retrieval runner: submits query
README.md - Retrieval runner → Twin loader: loads representations
retrieval_pipeline/llm_retrieval.py - Twin loader → Digital-twin files: reads representations
- Retrieval runner → Relevance scoring: scores images
retrieval_pipeline/llm_retrieval.py - Retrieval runner → Object retrieval: retrieves objects
retrieval_pipeline/llm_retrieval.py - Retrieval runner → Comparative ranking: ranks candidates
retrieval_pipeline/llm_retrieval.py - Relevance scoring → LLM API wrapper: requests scoring
retrieval_pipeline/llm_retrieval.py - Comparative ranking → LLM API wrapper: requests ranking
retrieval_pipeline/llm_retrieval.py - Object retrieval → LLM API wrapper: requests extraction
retrieval_pipeline/llm_retrieval.py - LLM API wrapper → LLM provider: sends prompts
models/llm_api.py - Retrieval runner → Ranked results: writes results
README.md - Twin loader → Image and mask utilities: decodes masks
digital_twins_pipeline/digital_twins_loader.py - Twin pipeline → Image and mask utilities: uses helpers
digital_twins_pipeline/image_to_digital_pipeline.py - Generation runner → Logging: logs progress
digital_twins_pipeline/digital_twins_generation.py