How maelic/relateanything works
RelateAnything predicts scored subject–predicate–object relations from an image and supplied boxes or masks, with caller-selected predicate vocabularies and optional spatial/semantic graph decomposition. Its principal runtime path is the Python API into the relation model, followed by score handling and ranked triplets. Separate branches cover detector-assisted deployment, dataset and annotation preparation, model training, and evaluation. The README documents additional deployment targets; only the Python deployment pipeline is represented in detail here because the sampled deployment source is partial.
Main entry points: relsgg.py RelSGG model · api.py Python API · pipeline.py Scene pipeline · train.py Training entry
Components 28
Inference API
- Python API
relsgg/api.py - Predicate embeddings
relsgg/text/student.py - Ranked triplets
relsgg/api.py
Relation model
- RelSGG model
relsgg/model/relsgg.py - Image backbone
relsgg/model/backbone.py - Region pooling
relsgg/model/pooling.py - Pair sampler
relsgg/model/sampler.py - Pair geometry
relsgg/model/geometry.py - Scene transformer
relsgg/model/transformer.py - Vocabulary head
relsgg/model/vocab_head.py - Score contract
relsgg/scoring.py
Data and training
- Source datasets
training - Training packs
relsgg/data - Dataset loader
relsgg/data/dataset.py - Training setup
relsgg/training/setup.py - Training entry
train.py - Training engine
relsgg/training/engine.py - Model checkpoint
relsgg/checkpoint.py - Relation annotation
datagen/sgg_vllm_generate.py
Deployment
- Scene pipeline
deploy/pipeline.py - Object detector
- Exported runtimes
deploy/runtime.py
Evaluation
- Relation evaluator
relsgg/eval/evaluator.py - Evaluation suites
relsgg/eval
Other components
- Image and regions
- Training operator
- Deployment user
- Evaluation operator
Connections 28
- Image and regions → Python API: provides image and regions
- Python API → RelSGG model: runs prediction
- Python API → Predicate embeddings: encodes predicates
- Python API → Score contract: calibrates scores
- RelSGG model → Image backbone: extracts image features
- RelSGG model → Region pooling: pools region features
- RelSGG model → Pair sampler: selects pairs
- RelSGG model → Pair geometry: encodes pair geometry
- RelSGG model → Scene transformer: refines pair context
- RelSGG model → Vocabulary head: scores predicates
- Predicate embeddings → Vocabulary head: supplies embeddings
- Python API → Ranked triplets: returns ranked relations
- Training operator → Training entry: starts training
- Source datasets → Training packs: converted and packed
- Relation annotation → Training packs: generates annotations
- Training packs → Dataset loader: loads samples
- Training entry → Training setup: configures training
- Training entry → Training engine: runs training loop
- Training setup → RelSGG model: builds model
- Training engine → Dataset loader: consumes batches
- Training engine → Model checkpoint: saves weights
- Deployment user → Scene pipeline: submits frames
- Scene pipeline → Object detector: detects regions
- Scene pipeline → Python API: predicts relations
- Scene pipeline → Exported runtimes: uses backend
- Evaluation operator → Relation evaluator: runs evaluation
- Evaluation suites → Relation evaluator: provides benchmarks
- Relation evaluator → RelSGG model: evaluates predictions