How gufran21/local-image-search-system-using-clip works
Project type and purpose
• This is a local image-search application written in Python. It exposes a simple UI (likely web-based or CLI) for users to upload a folder of images and submit text queries. Under the hood it uses OpenAI’s CLIP model to compute image and text embeddings, then ranks images by cosine similarity to the query.
File structure
• Top level:
• app.py – the entry point that implements HTTP endpoints or a CLI loop for folder uploads and text queries.
• models/
– clip_model.py – encapsulates loading the CLIP model and generating embeddings for images or text.
– get_all_images.py – scans a directory of images, reads files, and hands them off to the embedding logic.
• requirements.txt – lists Python packages (e.g., torch, transformers, Flask or Streamlit).
• No separate frontend folder: the UI is embedded in app.py or in a lightweight framework.
README for additional insights
• Tech stack: Python 3.8+, (optionally CUDA), CLIP from Hugging Face or OpenAI.
• Workflow:
a. User uploads a folder → images are encoded to embedding vectors (via clip_model.py).
b. User submits a text prompt → text is encoded to an embedding.
c. Cosine similarity search between the query and each image embedding → top‐k images returned.
Main components
a. User Interface
• Web UI or CLI in app.py where the user triggers “Upload Folder” and “Submit Query.”
b. API / Application Layer
• Ingest Endpoint
• Search Endpoint
c. File System
• Local directory storage for raw image files
d. Image Ingestion Module (models/get_all_images.py)
• Reads files, streams them into the embedding pipeline
e. CLIP Model Module (models/clip_model.py)
• Image Encoder
• Text Encoder
f. Embedding Store
• In-memory list or a lightweight key-value store holding (image_path → embedding_vector)
g. Similarity Search Engine
• Computes cosine similarity, sorts, selects top matches
h. Results Renderer
• Formats matched image thumbnails and metadata back into a UI response
Relationships and data flows
• User → UI → Application Layer
• “Upload Folder” → Image Ingestion Module → CLIP Image Encoder → Embedding Store
• “Submit Query” → CLIP Text Encoder → Similarity Search Engine (reads Embedding Store) → Results Renderer → UI → User
Architectural patterns and design principles
• Layered (Presentation → Application → Domain).
• Single-responsibility modules (one for ingestion, one for encoding, one for search).
• In-process data flow avoids network overhead—suitable for local use.
• Clearly separate “data ingestion” (side-effect: reading files & storing embeddings) from “query processing” (pure compute).