How mihaidavid05/text-to-image-search works
Main components
• Frontend/UI Layer: Represented by the Jinja templates found under the “templates” directory. These templates handle the display of the search form (form_template.html), the data report (data_report_template.html), and the images display (images_template.html).
• API Layer / FastAPI Backend: The FastAPI application is kick-started by “service.py” and supports API interactions using request/response schemas defined in “src/schemas.py”. This component receives user queries from the UI.
• Embedding & Search Engine: This includes the algorithms that use the CLIP-ViT-B-32 model to create text and image embeddings. The embedding process (both for text and images) is applied before querying the database. The “utils/search.py” module is responsible for transforming the input text into an embedding and executing an Approximate Nearest Neighbor (ANN) search in Qdrant.
• Data Preparation & Evaluation: Two additional scripts – “prepare.py” and “evaluate.py” – handle data ingestion, report generation, image embedding creation, population of the Qdrant database, and performance evaluation using metrics like precision@k.
• External Vector Database (Qdrant): The project relies on an external Qdrant service (run via Docker) to store image embeddings and perform vector similarity searches.
• Utilities: The “utils” folder (with data.py and utils.py) provides additional support routines to manage data, perform validations, and process other general tasks.
Relationships and interactions
• User Interaction: The UI (Jinja templates) captures text queries and displays results. The templates send form data to the FastAPI endpoint.
• API/Backend Processing: When a POST request arrives at the FastAPI endpoint (service.py), it uses the defined schemas (src/schemas.py) to validate and process it.
• Query Handling: The backend calls the search utility (utils/search.py) to embed the input text (using the CLIP model) and performs a vector search against Qdrant.
• Data Flow: For report generation and data updates, the “prepare.py” script uses the “utils/data.py” module to download, organize, and extract relevant data (as well as to build and store the embeddings).
• Evaluation: The “evaluate.py” script ingests labels from the provided file (docs/labels.txt) to test the retrieval accuracy of the system.
Architectural patterns and technologies
• Separation of Concerns: The design clearly partitions the user interface (Jinja templates), business logic (FastAPI service and associated schemas), and data processing (utility modules and external service integration).
• Microservice-like Endpoints: Although this is a single project, the distinct scripts (prepare.py, service.py, evaluate.py) act like separate services for data handling, user service, and evaluation.
• External Integration: The use of a Dockerized Qdrant instance for ANN search is a key external dependency.
• Technologies: FastAPI for the backend, Jinja for templating, a CLIP model for generating embeddings, and Qdrant for vector storage and ANN retrieval.
• Clear Labels for Each Component:
• “Jinja Templates (UI)” representing the presentation layer.
• “FastAPI Service (service.py)” as the API endpoint.
• “Schemas (src/schemas.py)” assisting in input/output validation.
• “Search Utilities (utils/search.py)” for embedding and search logic.
• “Data Utilities (utils/data.py & utils/utils.py)” for preprocessing and auxiliary functions.
• “Data Preparation & Report Generation (prepare.py)” for initial data ingestion.
• “Evaluation Service (evaluate.py)” for performance testing.
• “Qdrant Vector Database” showing the external vector storage.
• Differentiate the flow for data preparation (which runs as a separate process) versus the live search queries.
• Additional Annotations:
• Brief notes near each component that reference the key technologies involved (e.g., “CLIP-ViT-B-32 for embeddings”, “Dockerized Qdrant”, etc.).
Overall detail
• Maintain a high-level system overview while providing enough granularity (e.g., distinguishing between API endpoints and backend utility modules).