# suoych/keds architecture

> Architecture diagram of the GitHub repository suoych/keds, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/suoych/keds
- Repository: https://github.com/suoych/keds

## Overview

KEDs is a research implementation of knowledge-enhanced, dual-stream zero-shot composed image retrieval. A user supplies a reference image and text modification; inference encodes the query, retrieves image- and text-side knowledge, composes candidate features, and returns retrieval results. The repository also supports model training and benchmark evaluation. The excerpts confirm dual-stream model construction, feature indexing, and retrieval utilities; some demo and training call details are unsampled, so uncertain links are omitted.

Main entry points: [`src/demo.py`](https://github.com/suoych/keds/blob/main/src/demo.py) (Demo inference), [`src/eval_retrieval.py`](https://github.com/suoych/keds/blob/main/src/eval_retrieval.py) (Evaluation entry), [`src/main.py`](https://github.com/suoych/keds/blob/main/src/main.py) (GPU feature indexes), [`src/trainer.py`](https://github.com/suoych/keds/blob/main/src/trainer.py) (Training loop)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Composed Retrieval"]
  node_demo["Demo inference<br/>[demo.py]"]
  node_query["Image and prompt"]
  node_retrieval["Feature retrieval<br/>[eval_utils.py]"]
end

subgraph group_learning["Model Training"]
  node_training["Training entry<br/>[main.py]"]
  node_trainer["Training loop<br/>[trainer.py]"]
  node_datasets["Dataset loaders<br/>[data.py]"]
end

subgraph group_knowledge["Knowledge Data"]
  node_knowledge_files[("CC3M feature sets")]
  node_sampling["Feature sampling<br/>[database.py]"]
  node_faiss[("GPU feature indexes<br/>[main.py]")]
  node_noun_extract["Caption noun extraction<br/>[extract_cc_noun.py]"]
end

subgraph group_models["Model Components"]
  node_clip_wrapper["CLIP loading<br/>[clip.py]"]
  node_clip_model["CLIP encoder<br/>[model.py]"]
  node_stream_models["Dual-stream modules<br/>[model.py]"]
end

subgraph group_evaluation["Benchmark Evaluation"]
  node_evaluation["Evaluation entry<br/>[eval_retrieval.py]"]
  node_eval_utils["Benchmark evaluators<br/>[eval_utils.py]"]
  node_benchmarks["CIR benchmarks<br/>[data.py]"]
end

node_user(("Retrieval user"))

node_user -->|"requests retrieval"| node_demo
node_query -->|"provides query"| node_demo
node_demo -->|"loads encoder"| node_clip_wrapper
node_clip_wrapper -->|"constructs"| node_clip_model
node_demo -->|"uses streams"| node_stream_models
node_demo -->|"runs retrieval"| node_retrieval
node_retrieval -->|"searches indexes"| node_faiss
node_faiss -->|"indexes features"| node_knowledge_files
node_training -->|"loads batches"| node_datasets
node_training -->|"runs training"| node_trainer
node_training -->|"trains streams"| node_stream_models
node_trainer -->|"encodes batches"| node_clip_model
node_trainer -->|"retrieves neighbors"| node_faiss
node_sampling -->|"copies samples"| node_knowledge_files
node_noun_extract -.->|"reads captions"| node_knowledge_files
node_evaluation -->|"loads data"| node_datasets
node_evaluation -->|"runs evaluators"| node_eval_utils
node_eval_utils -->|"evaluates against"| node_benchmarks
node_eval_utils -->|"searches indexes"| node_faiss
node_evaluation -->|"loads encoder"| node_clip_wrapper
node_evaluation -->|"loads checkpoints"| node_stream_models

click node_demo "https://github.com/suoych/keds/blob/main/src/demo.py"
click node_retrieval "https://github.com/suoych/keds/blob/main/src/eval_utils.py"
click node_training "https://github.com/suoych/keds/blob/main/src/main.py"
click node_trainer "https://github.com/suoych/keds/blob/main/src/trainer.py"
click node_datasets "https://github.com/suoych/keds/blob/main/src/data.py"
click node_sampling "https://github.com/suoych/keds/blob/main/database.py"
click node_faiss "https://github.com/suoych/keds/blob/main/src/main.py"
click node_noun_extract "https://github.com/suoych/keds/blob/main/extract_cc_noun.py"
click node_clip_wrapper "https://github.com/suoych/keds/blob/main/src/model/clip.py"
click node_clip_model "https://github.com/suoych/keds/blob/main/src/model/model.py"
click node_stream_models "https://github.com/suoych/keds/blob/main/src/model/model.py"
click node_evaluation "https://github.com/suoych/keds/blob/main/src/eval_retrieval.py"
click node_eval_utils "https://github.com/suoych/keds/blob/main/src/eval_utils.py"
click node_benchmarks "https://github.com/suoych/keds/blob/main/src/data.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_query,node_retrieval,node_user toneBlue
class node_training,node_trainer,node_datasets toneAmber
class node_knowledge_files,node_sampling,node_faiss,node_noun_extract toneMint
class node_clip_wrapper,node_clip_model,node_stream_models toneRose
class node_evaluation,node_eval_utils,node_benchmarks toneIndigo
```

## Components

### Composed Retrieval

- [Demo inference](https://github.com/suoych/keds/blob/main/src/demo.py): `src/demo.py`
- **Image and prompt**
- [Feature retrieval](https://github.com/suoych/keds/blob/main/src/eval_utils.py): `src/eval_utils.py`

### Model Training

- [Training entry](https://github.com/suoych/keds/blob/main/src/main.py): `src/main.py`
- [Training loop](https://github.com/suoych/keds/blob/main/src/trainer.py): `src/trainer.py`
- [Dataset loaders](https://github.com/suoych/keds/blob/main/src/data.py): `src/data.py`

### Knowledge Data

- **CC3M feature sets**
- [Feature sampling](https://github.com/suoych/keds/blob/main/database.py): `database.py`
- [GPU feature indexes](https://github.com/suoych/keds/blob/main/src/main.py): `src/main.py`
- [Caption noun extraction](https://github.com/suoych/keds/blob/main/extract_cc_noun.py): `extract_cc_noun.py`

### Model Components

- [CLIP loading](https://github.com/suoych/keds/blob/main/src/model/clip.py): `src/model/clip.py`
- [CLIP encoder](https://github.com/suoych/keds/blob/main/src/model/model.py): `src/model/model.py`
- [Dual-stream modules](https://github.com/suoych/keds/blob/main/src/model/model.py): `src/model/model.py`

### Benchmark Evaluation

- [Evaluation entry](https://github.com/suoych/keds/blob/main/src/eval_retrieval.py): `src/eval_retrieval.py`
- [Benchmark evaluators](https://github.com/suoych/keds/blob/main/src/eval_utils.py): `src/eval_utils.py`
- [CIR benchmarks](https://github.com/suoych/keds/blob/main/src/data.py): `src/data.py`

### Other components

- **Retrieval user**

## Connections

- Retrieval user → Demo inference: requests retrieval
- Image and prompt → Demo inference: provides query
- Demo inference → CLIP loading: loads encoder
- CLIP loading → CLIP encoder: constructs
- Demo inference → Dual-stream modules: uses streams
- Demo inference → Feature retrieval: runs retrieval
- Feature retrieval → GPU feature indexes: searches indexes
- GPU feature indexes → CC3M feature sets: indexes features
- Training entry → Dataset loaders: loads batches
- Training entry → Training loop: runs training
- Training entry → Dual-stream modules: trains streams
- Training loop → CLIP encoder: encodes batches
- Training loop → GPU feature indexes: retrieves neighbors
- Feature sampling → CC3M feature sets: copies samples
- Caption noun extraction → CC3M feature sets: reads captions
- Evaluation entry → Dataset loaders: loads data
- Evaluation entry → Benchmark evaluators: runs evaluators
- Benchmark evaluators → CIR benchmarks: evaluates against
- Benchmark evaluators → GPU feature indexes: searches indexes
- Evaluation entry → CLIP loading: loads encoder
- Evaluation entry → Dual-stream modules: loads checkpoints