How sxijyoti/rag-image-restoration works
This repository provides retrieval-augmented image restoration: a degraded image is split into patches, encoded with DA-CLIP, matched to clean patches through FAISS, and reconstructed with Gaussian blending, optionally after UNet refinement. It also includes offline index building and refinement training. The README documents the index-building workflow and fusion capability; the sampled inference excerpt does not show those implementations being invoked, so those links are not inferred. Example modules are shown as usage aids, not runtime stages.
Main entry points: restore_retrieval_only.py Restore pipeline · da_clip_encoder.py DA-CLIP encoder · daclip-faiss-indexer.ipynb FAISS index building · patch_extraction.py Patch extraction
Components 13
Restoration inference
- Restore pipeline
src/restore_retrieval_only.py - Patch extraction
src/modules/patch_extraction.py - Gaussian reconstruction
src/modules/patch_extraction.py - Patch examples
src/modules/examples.py
Encoding and fusion
- DA-CLIP encoder
src/modules/da_clip_encoder.py - Context fusion
src/context_fusion.py - Encoder examples
src/modules/encoder_examples.py
Index and retrieval
- FAISS index building
daclip-faiss-indexer.ipynb - FAISS patch index
- Patch retrieval
src/retrieval.py
Refinement training
- Refinement UNet
src/train_refinement.py
Other components
- Image restoration user
- Clean image dataset
Connections 15
- Image restoration user → Restore pipeline: submits image
README.md - Restore pipeline → Patch extraction: extracts patches
src/restore_retrieval_only.py - Restore pipeline → DA-CLIP encoder: encodes patches
src/restore_retrieval_only.py - Restore pipeline → Patch retrieval: searches candidates
src/restore_retrieval_only.py - Patch retrieval → FAISS patch index: queries
- Patch retrieval → Clean image dataset: loads patches
src/retrieval.py - FAISS index building → DA-CLIP encoder: encodes dataset patches
README.md - FAISS index building → FAISS patch index: builds index
README.md - FAISS index building → Clean image dataset: reads images
README.md - Restore pipeline → Refinement UNet: optionally refines
README.md - Restore pipeline → Gaussian reconstruction: blends patches
src/restore_retrieval_only.py - Patch extraction → Patch examples: demonstrated by
src/modules/examples.py - Patch extraction → Encoder examples: used in example
src/modules/encoder_examples.py - DA-CLIP encoder → Encoder examples: used in example
src/modules/encoder_examples.py - DA-CLIP encoder → Context fusion: provides embeddings