How suoych/keds works
KEDs is a research implementation of knowledge-enhanced, dual-stream zero-shot composed image retrieval. A user supplies a reference image and text modification; inference encodes the query, retrieves image- and text-side knowledge, composes candidate features, and returns retrieval results. The repository also supports model training and benchmark evaluation. The excerpts confirm dual-stream model construction, feature indexing, and retrieval utilities; some demo and training call details are unsampled, so uncertain links are omitted.
Main entry points: demo.py Demo inference · eval_retrieval.py Evaluation entry · main.py GPU feature indexes · trainer.py Training loop
Components 17
Composed Retrieval
- Demo inference
src/demo.py - Image and prompt
- Feature retrieval
src/eval_utils.py
Model Training
- Training entry
src/main.py - Training loop
src/trainer.py - Dataset loaders
src/data.py
Knowledge Data
- CC3M feature sets
- Feature sampling
database.py - GPU feature indexes
src/main.py - Caption noun extraction
extract_cc_noun.py
Model Components
- CLIP loading
src/model/clip.py - CLIP encoder
src/model/model.py - Dual-stream modules
src/model/model.py
Benchmark Evaluation
- Evaluation entry
src/eval_retrieval.py - Benchmark evaluators
src/eval_utils.py - CIR benchmarks
src/data.py
Other components
- Retrieval user
Connections 21
- Retrieval user → Demo inference: requests retrieval
- Image and prompt → Demo inference: provides query
- Demo inference → CLIP loading: loads encoder
- CLIP loading → CLIP encoder: constructs
- Demo inference → Dual-stream modules: uses streams
- Demo inference → Feature retrieval: runs retrieval
- Feature retrieval → GPU feature indexes: searches indexes
- GPU feature indexes → CC3M feature sets: indexes features
- Training entry → Dataset loaders: loads batches
- Training entry → Training loop: runs training
- Training entry → Dual-stream modules: trains streams
- Training loop → CLIP encoder: encodes batches
- Training loop → GPU feature indexes: retrieves neighbors
- Feature sampling → CC3M feature sets: copies samples
- Caption noun extraction → CC3M feature sets: reads captions
- Evaluation entry → Dataset loaders: loads data
- Evaluation entry → Benchmark evaluators: runs evaluators
- Benchmark evaluators → CIR benchmarks: evaluates against
- Benchmark evaluators → GPU feature indexes: searches indexes
- Evaluation entry → CLIP loading: loads encoder
- Evaluation entry → Dual-stream modules: loads checkpoints