How adarsh-menon/komorebi-semantic-image-search-system works
1. What this system is
- This repository is a Python-based semantic image search service. Its core job is to ingest images, derive semantic representations from them, store those representations in a vector index, and later answer text/image search requests by retrieving similar images.
2. Primary runtime shape
- The main application lives under src/api and is organized like a FastAPI-style service: an app entrypoint, route modules, dependency wiring, request/response schemas, and middleware.
- The service is packaged for local development, containerized deployment, and Kubernetes rollout.
3. Request-facing API layer
- src/api/routes splits the external surface into:
- health for liveness/readiness,
- upload for ingesting images,
- indexer for indexing workflows,
- search for semantic retrieval.
- src/api/middleware adds cross-cutting controls such as authentication and rate limiting, so those concerns are enforced before requests reach the search/indexing logic.
4. Core image-search pipeline
- The main semantic flow appears to be:
1. image upload or ingestion,
2. storage of raw assets,
3. preprocessing,
4. embedding generation,
5. optional caption generation,
6. persistence of vectors and metadata,
7. query-time retrieval and reranking.
- The service layer in src/services is where this pipeline is decomposed into reusable steps rather than being embedded in the API handlers.
5. Model subsystem
- src/models contains wrappers/loaders for the ML backends: CLIP, BLIP, GIT, and sentence-transformer style models.
- model_loader.py suggests a centralized loading/selection boundary so the rest of the system does not depend directly on individual model implementations.
- This subsystem is the main compute-heavy part of the application and is likely the source of both image embeddings and generated captions used to enrich search quality.
6. Search-time intelligence
- query_rewriter.py suggests user queries may be expanded or normalized before retrieval.
- reranker_service.py indicates the system likely performs two-stage search: an initial vector lookup followed by a second-pass relevance refinement.
- caption_service.py implies captions are not only generated for ingestion but may also support ranking or explainability.
7. Vector and metadata storage boundary
- src/vectorstore abstracts similarity search backends:
- qdrant_store.py for a persistent vector database,
- faiss_store.py for local/in-process similarity search,
- metadata_store.py for non-vector attributes tied to image records.
- This is a key architectural boundary: the search layer talks to a vector-store interface, while the implementation can vary between embedded/local and external DB-backed retrieval.
8. Raw file storage boundary
- src/storage provides adapters for local_storage, s3_storage, and gcs_storage.
- This means image binaries are stored separately from embeddings/metadata, and the system can be deployed against local disk or cloud object storage depending on environment.
- utils_image.py likely supports image normalization or file handling around these storage adapters.
9. Background and batch work
- src/tasks separates non-request work from the API:
- index_task.py for bulk or asynchronous indexing,
- cleanup_task.py for retention or housekeeping,
- workers.py for task execution orchestration.
- This suggests the application is not purely synchronous; ingestion/indexing can be offloaded from the request path to improve throughput and reliability.
10. Configuration and observability
- src/core contains shared runtime configuration, constants, exceptions, and logging setup.
- src/monitoring adds metrics, tracing, and Sentry logging, indicating the service is instrumented for production diagnostics rather than being a simple prototype.
- YAML config files under komorebi_semantic_image_search/config and environment-based settings imply the runtime is heavily configuration-driven.
11. Deployment and infrastructure
- infra/Dockerfile and infra/docker-compose.yml support containerized local and production-like environments.
- infra/k8s contains config, secrets, deployment, service, and HPA manifests, so the service is intended to run horizontally in Kubernetes.
- infra/nginx/nginx.conf suggests an edge reverse proxy in front of the app, likely for routing, buffering, or request limits.
- GitHub Actions CI/CD files show automated build/test/deploy paths.
12. Validation, experimentation, and quality
- tests covers API, captioning, embeddings, storage, and vectorstore behavior, reflecting the main subsystem boundaries.
- notebooks indicate offline experimentation and evaluation, including recall-at-k and model comparison, which is important for a semantic search system where retrieval quality is a primary product metric.
13. Likely end-to-end architecture diagram
- A good diagram would show:
- clients calling the API,
- middleware handling auth/rate limiting,
- request handlers invoking services,
- services calling ML models, storage backends, and vector stores,
- tasks handling asynchronous indexing/cleanup,
- monitoring spanning the whole service,
- deployment behind Nginx in containers/Kubernetes with object storage and vector DB as external dependencies.
14. Important architectural takeaway
- The repo is organized around clear separations between API, ML inference, object storage, vector search, and background processing. That separation is the main design feature and should be preserved in any architecture diagram.
Main entry points: image_service.py Image Flow · main.py API App · routes Routes · vectorstore_service.py Vector Service
Components 18
Clients
External request sources and user-facing entry points
- Client · User or application issuing upload and search requests
API Layer
HTTP service entrypoint, routing, schemas, and middleware
- API App · Service entrypoint and application wiring
komorebi_semantic_image_search/src/api/main.py - Middleware · Authentication and rate limiting before request handlers
komorebi_semantic_image_search/src/api/middleware - Routes · Health, upload, indexer, and search endpoints
komorebi_semantic_image_search/src/api/routes - Schemas · API payload definitions for indexing and search
komorebi_semantic_image_search/src/api/schemas
Core Services
Image processing, ML inference, and search orchestration
- Image Flow · Coordinates upload, preprocessing, embedding, captioning, and persistence
komorebi_semantic_image_search/src/services/image_service.py - Preprocess · Transforms raw images into model-ready inputs
komorebi_semantic_image_search/src/services/preprocessing_service.py - Embeddings · Generates semantic vectors from images or text
komorebi_semantic_image_search/src/services/embedding_service.py - Captioning · Produces descriptive captions to enrich retrieval
komorebi_semantic_image_search/src/services/caption_service.py - Query Rewrite · Expands or normalizes search queries before retrieval
komorebi_semantic_image_search/src/services/query_rewriter.py - Reranker · Refines initial similarity results into a final ordering
komorebi_semantic_image_search/src/services/reranker_service.py - Vector Service · Abstracts similarity lookup and metadata access
komorebi_semantic_image_search/src/services/vectorstore_service.py - Model Loader · Centralizes loading of CLIP, BLIP, GIT, and sentence transformers
komorebi_semantic_image_search/src/models/model_loader.py
Data Backends
Object storage and vector/metadata persistence
- Object Storage · Stores raw image binaries on local disk or cloud object storage
komorebi_semantic_image_search/src/storage - Vector Store · Persistent or local vector search implementation with metadata support
komorebi_semantic_image_search/src/vectorstore
Operations
Background work, monitoring, and deployment
- Workers · Asynchronous indexing and cleanup execution
komorebi_semantic_image_search/src/tasks/workers.py - Monitoring · Metrics, tracing, and error reporting
komorebi_semantic_image_search/src/monitoring - Infra · Container, Kubernetes, and reverse-proxy deployment assets
komorebi_semantic_image_search/infra
Connections 23
- Client → API App: HTTP requests · Clients call the application entrypoint
- API App → Middleware: applies · Middleware runs before handlers
- API App → Routes: registers · Routes are mounted into the app
- Routes → Schemas: validates · Handlers use request and response schemas
- Routes → Image Flow: ingest/search · Upload and indexing endpoints invoke service orchestration
- Routes → Vector Service: retrieve · Search endpoints access vector retrieval
- Image Flow → Preprocess: prepare · Raw images are normalized before inference
- Image Flow → Embeddings: encode · Images are converted into semantic vectors
- Image Flow → Captioning: enrich · Captions can be generated during ingestion
- Preprocess → Object Storage: read/write · Image files are loaded from and stored to object storage
- Embeddings → Model Loader: loads models · Embedding generation depends on selected ML backends
- Captioning → Model Loader: loads models · Caption generation depends on multimodal backends
- Image Flow → Vector Store: persist · Embeddings and metadata are written to the vector layer
- Vector Service → Vector Store: query · Similarity search is delegated to the backend store
- Vector Service → Query Rewrite: normalize · Queries may be expanded before retrieval
- Vector Service → Reranker: refine · Initial matches are reranked into final results
- Workers → Image Flow: batch ingest · Workers reuse the ingestion pipeline for async jobs
- Workers → Vector Store: cleanup/index · Background jobs maintain stored vectors and metadata
- Monitoring → API App: instrument · Observability spans the request service
- Monitoring → Workers: instrument · Background processing is also observed
- Infra → API App: deploys · Container and Kubernetes assets package the service
- Infra → Object Storage: configures · Deployment can target local or cloud object storage
- Infra → Vector Store: configures · Deployment can target local or external vector search