# yewinhtet74/sdm-retrieval architecture

> Architecture diagram of the GitHub repository yewinhtet74/sdm-retrieval, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yewinhtet74/sdm-retrieval
- Repository: https://github.com/yewinhtet74/sdm-retrieval

## Overview

SDM-Retrieval is a research workflow for comparing training-free compression methods on frozen CLIP image and caption embeddings. Users configure a Flickr30k run, extract or load aligned features, optionally select a fixed training subset to fit a reducer, then evaluate bidirectional retrieval quality and FAISS latency/storage and save results. The graph preserves the documented baseline, PCA, SVD and SDM branches, plus diagnostics. Edges reflect the supplied excerpts, SOURCE INDEX references and README; unsampled data-loading and orchestration details are not inferred beyond those sources.

Main entry points: [`src/evaluate.py`](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/evaluate.py) (Evaluation orchestration), [`src/features.py`](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/features.py) (Feature extraction), [`outputs`](https://github.com/yewinhtet74/sdm-retrieval/tree/main/outputs) (Cached embeddings), [`src/reducers.py`](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/reducers.py) (Reducer methods)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Data and configuration"]
  node_config["Run configuration<br/>[config.py]"]
  node_flickr[("Flickr30k data")]
  node_data["Dataset loading<br/>[data.py]"]
end

subgraph group_embedding["Embedding preparation"]
  node_model["Frozen CLIP<br/>[model.py]"]
  node_features["Feature extraction<br/>[features.py]"]
  node_feature_cache[("Cached embeddings")]
end

subgraph group_compression["Compression methods"]
  node_subset["Training subset<br/>[subset.py]"]
  node_reducers["Reducer methods<br/>[reducers.py]"]
  node_clusters["SDM cluster data"]
end

subgraph group_evaluation["Retrieval evaluation"]
  node_evaluate["Evaluation orchestration<br/>[evaluate.py]"]
  node_metrics["Recall metrics<br/>[metrics.py]"]
  node_retrieval["FAISS benchmarks<br/>[retrieval.py]"]
  node_diagnostic["I2T diagnostics<br/>[diagnose_i2t.py]"]
  node_results[("Evaluation results")]
end

node_user(("Researcher"))

node_user -->|"configures run"| node_config
node_user -->|"provides dataset"| node_flickr
node_config -.->|"configures loading"| node_data
node_flickr -->|"supplies records"| node_data
node_data -->|"builds loaders"| node_features
node_features -->|"loads encoder"| node_model
node_features -->|"writes embeddings"| node_feature_cache
node_feature_cache -->|"supplies training data"| node_subset
node_subset -->|"fits methods"| node_reducers
node_feature_cache -->|"supplies evaluation features"| node_evaluate
node_reducers -->|"transforms embeddings"| node_evaluate
node_evaluate -->|"computes recall"| node_metrics
node_evaluate -->|"benchmarks index"| node_retrieval
node_evaluate -->|"saves result table"| node_results
node_user -->|"requests diagnosis"| node_diagnostic
node_reducers -.->|"uses SDM clusters"| node_clusters

click node_config "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/config.py"
click node_data "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/data.py"
click node_model "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/model.py"
click node_features "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/features.py"
click node_feature_cache "https://github.com/yewinhtet74/sdm-retrieval/tree/main/outputs"
click node_subset "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/subset.py"
click node_reducers "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/reducers.py"
click node_clusters "https://github.com/yewinhtet74/sdm-retrieval/tree/main/clusters"
click node_evaluate "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/evaluate.py"
click node_metrics "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/metrics.py"
click node_retrieval "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/retrieval.py"
click node_diagnostic "https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/diagnose_i2t.py"
click node_results "https://github.com/yewinhtet74/sdm-retrieval/tree/main/outputs/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_flickr,node_data toneBlue
class node_model,node_features,node_feature_cache toneAmber
class node_subset,node_reducers,node_clusters toneMint
class node_evaluate,node_metrics,node_retrieval,node_diagnostic,node_results toneRose
class node_user toneIndigo
```

## Components

### Data and configuration

- [Run configuration](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/config.py): `src/config.py`
- **Flickr30k data**
- [Dataset loading](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/data.py): `src/data.py`

### Embedding preparation

- [Frozen CLIP](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/model.py): `src/model.py`
- [Feature extraction](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/features.py): `src/features.py`
- [Cached embeddings](https://github.com/yewinhtet74/sdm-retrieval/tree/main/outputs): `outputs`

### Compression methods

- [Training subset](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/subset.py): `src/subset.py`
- [Reducer methods](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/reducers.py): `src/reducers.py`
- [SDM cluster data](https://github.com/yewinhtet74/sdm-retrieval/tree/main/clusters): `clusters`

### Retrieval evaluation

- [Evaluation orchestration](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/evaluate.py): `src/evaluate.py`
- [Recall metrics](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/metrics.py): `src/metrics.py`
- [FAISS benchmarks](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/retrieval.py): `src/retrieval.py`
- [I2T diagnostics](https://github.com/yewinhtet74/sdm-retrieval/blob/main/src/diagnose_i2t.py): `src/diagnose_i2t.py`
- [Evaluation results](https://github.com/yewinhtet74/sdm-retrieval/tree/main/outputs/results): `outputs/results`

### Other components

- **Researcher**

## Connections

- Researcher → Run configuration: configures run (evidence: `README.md`)
- Researcher → Flickr30k data: provides dataset (evidence: `README.md`)
- Run configuration → Dataset loading: configures loading
- Flickr30k data → Dataset loading: supplies records (evidence: `README.md`)
- Dataset loading → Feature extraction: builds loaders (evidence: `src/features.py`)
- Feature extraction → Frozen CLIP: loads encoder (evidence: `src/features.py`)
- Feature extraction → Cached embeddings: writes embeddings (evidence: `src/features.py`)
- Cached embeddings → Training subset: supplies training data
- Training subset → Reducer methods: fits methods (evidence: `scripts/03_eval_reducers.py`)
- Cached embeddings → Evaluation orchestration: supplies evaluation features (evidence: `src/evaluate.py`)
- Reducer methods → Evaluation orchestration: transforms embeddings (evidence: `src/evaluate.py`)
- Evaluation orchestration → Recall metrics: computes recall (evidence: `src/evaluate.py`)
- Evaluation orchestration → FAISS benchmarks: benchmarks index (evidence: `src/evaluate.py`)
- Evaluation orchestration → Evaluation results: saves result table (evidence: `src/evaluate.py`)
- Researcher → I2T diagnostics: requests diagnosis
- Reducer methods → SDM cluster data: uses SDM clusters