# sxijyoti/rag-image-restoration architecture

> Architecture diagram of the GitHub repository sxijyoti/rag-image-restoration, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sxijyoti/rag-image-restoration
- Repository: https://github.com/sxijyoti/rag-image-restoration

## Overview

This repository provides retrieval-augmented image restoration: a degraded image is split into patches, encoded with DA-CLIP, matched to clean patches through FAISS, and reconstructed with Gaussian blending, optionally after UNet refinement. It also includes offline index building and refinement training. The README documents the index-building workflow and fusion capability; the sampled inference excerpt does not show those implementations being invoked, so those links are not inferred. Example modules are shown as usage aids, not runtime stages.

Main entry points: [`src/restore_retrieval_only.py`](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/restore_retrieval_only.py) (Restore pipeline), [`src/modules/da_clip_encoder.py`](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/da_clip_encoder.py) (DA-CLIP encoder), [`daclip-faiss-indexer.ipynb`](https://github.com/sxijyoti/rag-image-restoration/blob/main/daclip-faiss-indexer.ipynb) (FAISS index building), [`src/modules/patch_extraction.py`](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/patch_extraction.py) (Patch extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Restoration inference"]
  node_restore["Restore pipeline"]
  node_patching["Patch extraction"]
  node_reconstruction["Gaussian reconstruction"]
  node_patch_examples["Patch examples<br/>[examples.py]"]
end

subgraph group_models["Encoding and fusion"]
  node_encoder["DA-CLIP encoder<br/>[da_clip_encoder.py]"]
  node_fusion["Context fusion<br/>[context_fusion.py]"]
  node_refinement_examples["Encoder examples"]
end

subgraph group_indexing["Index and retrieval"]
  node_index_builder["FAISS index building"]
  node_index[("FAISS patch index")]
  node_retriever["Patch retrieval<br/>[retrieval.py]"]
end

subgraph group_refinement["Refinement training"]
  node_refiner["Refinement UNet"]
end

node_user(("Image restoration user"))
node_clean_dataset[("Clean image dataset")]

node_user -->|"submits image"| node_restore
node_restore -->|"extracts patches"| node_patching
node_restore -->|"encodes patches"| node_encoder
node_restore -->|"searches candidates"| node_retriever
node_retriever -.->|"queries"| node_index
node_retriever -->|"loads patches"| node_clean_dataset
node_index_builder -->|"encodes dataset patches"| node_encoder
node_index_builder -->|"builds index"| node_index
node_index_builder -->|"reads images"| node_clean_dataset
node_restore -->|"optionally refines"| node_refiner
node_restore -->|"blends patches"| node_reconstruction
node_patching -->|"demonstrated by"| node_patch_examples
node_patching -->|"used in example"| node_refinement_examples
node_encoder -->|"used in example"| node_refinement_examples
node_encoder -.->|"provides embeddings"| node_fusion

click node_restore "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/restore_retrieval_only.py"
click node_patching "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/patch_extraction.py"
click node_reconstruction "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/patch_extraction.py"
click node_encoder "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/da_clip_encoder.py"
click node_fusion "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/context_fusion.py"
click node_index_builder "https://github.com/sxijyoti/rag-image-restoration/blob/main/daclip-faiss-indexer.ipynb"
click node_retriever "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/retrieval.py"
click node_refiner "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/train_refinement.py"
click node_refinement_examples "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/encoder_examples.py"
click node_patch_examples "https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/examples.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_restore,node_patching,node_reconstruction,node_patch_examples,node_user toneBlue
class node_encoder,node_fusion,node_refinement_examples,node_clean_dataset toneAmber
class node_index_builder,node_index,node_retriever toneMint
class node_refiner toneRose
```

## Components

### Restoration inference

- [Restore pipeline](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/restore_retrieval_only.py): `src/restore_retrieval_only.py`
- [Patch extraction](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/patch_extraction.py): `src/modules/patch_extraction.py`
- [Gaussian reconstruction](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/patch_extraction.py): `src/modules/patch_extraction.py`
- [Patch examples](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/examples.py): `src/modules/examples.py`

### Encoding and fusion

- [DA-CLIP encoder](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/da_clip_encoder.py): `src/modules/da_clip_encoder.py`
- [Context fusion](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/context_fusion.py): `src/context_fusion.py`
- [Encoder examples](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/modules/encoder_examples.py): `src/modules/encoder_examples.py`

### Index and retrieval

- [FAISS index building](https://github.com/sxijyoti/rag-image-restoration/blob/main/daclip-faiss-indexer.ipynb): `daclip-faiss-indexer.ipynb`
- **FAISS patch index**
- [Patch retrieval](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/retrieval.py): `src/retrieval.py`

### Refinement training

- [Refinement UNet](https://github.com/sxijyoti/rag-image-restoration/blob/main/src/train_refinement.py): `src/train_refinement.py`

### Other components

- **Image restoration user**
- **Clean image dataset**

## Connections

- Image restoration user → Restore pipeline: submits image (evidence: `README.md`)
- Restore pipeline → Patch extraction: extracts patches (evidence: `src/restore_retrieval_only.py`)
- Restore pipeline → DA-CLIP encoder: encodes patches (evidence: `src/restore_retrieval_only.py`)
- Restore pipeline → Patch retrieval: searches candidates (evidence: `src/restore_retrieval_only.py`)
- Patch retrieval → FAISS patch index: queries
- Patch retrieval → Clean image dataset: loads patches (evidence: `src/retrieval.py`)
- FAISS index building → DA-CLIP encoder: encodes dataset patches (evidence: `README.md`)
- FAISS index building → FAISS patch index: builds index (evidence: `README.md`)
- FAISS index building → Clean image dataset: reads images (evidence: `README.md`)
- Restore pipeline → Refinement UNet: optionally refines (evidence: `README.md`)
- Restore pipeline → Gaussian reconstruction: blends patches (evidence: `src/restore_retrieval_only.py`)
- Patch extraction → Patch examples: demonstrated by (evidence: `src/modules/examples.py`)
- Patch extraction → Encoder examples: used in example (evidence: `src/modules/encoder_examples.py`)
- DA-CLIP encoder → Encoder examples: used in example (evidence: `src/modules/encoder_examples.py`)
- DA-CLIP encoder → Context fusion: provides embeddings
