# krishnakaushik25/similar-images-finder architecture

> Architecture diagram of the GitHub repository krishnakaushik25/similar-images-finder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/krishnakaushik25/similar-images-finder
- Repository: https://github.com/krishnakaushik25/similar-images-finder

## Overview

##### Project type and purpose
- This is an open-source computer vision tool (“Similar Images Finder”) that implements an image-to-image search engine.
- It ingests a product image dataset, extracts deep features via a pretrained MobileNetV2, indexes them in Elasticsearch with a KNN plugin, and allows querying a new image to retrieve its top-k visually similar images.

##### Main components
a. Data Ingestion & Storage
- Input: `input/train.json` metadata and downloaded product images (in `input/` or a data folder created by `downloading_images.py`).
- Output: Raw image files on disk.
b. ML Pipeline (src/ML_Pipeline)
- downloading_images.py: parses the Kaggle JSON, fetches images, writes to local disk.
- feature_extraction.py: loads MobileNetV2 (TensorFlow/Keras), computes and flattens image embeddings.
- elasticsearch_indexing.py: connects to an Elasticsearch cluster with the KNN plugin, creates an index per label or a unified index, and ingests embedding vectors with metadata (image ID, path).
c. Query Engine (Engine.py)
- Reads a user-specified input image, invokes feature_extraction on it, then calls Elasticsearch’s KNN search API to retrieve the top-N similar vectors.
- Formats results (image URLs/paths + similarity scores) into `output/response.json`.
d. External Services
- Elasticsearch cluster with the KNN plugin (hosted locally or remotely).
- Optional: if images are downloaded from external URLs (web requests).
e. Notebooks (for prototyping)
- Jupyter notebooks under `notebooks/` illustrate each pipeline stage interactively (downloading, feature extraction, indexing, querying).

##### Relationships & data flow
1) Dataset JSON → downloading_images.py → Local image store
2) Image store → feature_extraction.py → Numeric embedding vectors
3) Embeddings + metadata → elasticsearch_indexing.py → ES index
4) User image → Engine.py → feature_extraction → ES KNN query → JSON output

##### Architectural patterns and design principles
- Modular pipeline: Each stage is encapsulated in its own Python module.
- Separation of concerns: Data ingestion, feature computation, indexing, and query logic are distinct.
- Reproducibility & prototyping: Jupyter notebooks parallel the src pipeline.
- Externalized state: Elasticsearch holds the indexed vectors rather than in-memory.

##### Technologies & frameworks
- Python 3.x
- TensorFlow & Keras (MobileNetV2 pretrained on ImageNet)
- NumPy, Pandas for data handling
- Elasticsearch with KNN plugin (e.g., `elasticsearch-py`)
- `requests` or `urllib` for HTTP image downloads
- Optional environment: virtualenv or Conda

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion Stage
    subgraph "Data Ingestion"
        DataSource["Kaggle Metadata & Image URLs"]:::external
        InputJSON["train.json (input/train.json)"]:::storage
        Downloader["Image Downloader\n(src/ML_Pipeline/downloading_images.py)"]:::compute
        Disk["Local Image Store"]:::storage
    end

    DataSource -->|"metadata JSON"| InputJSON
    InputJSON -->|"parsed URLs"| Downloader
    Downloader -->|"downloaded images"| Disk

    %% Indexing Pipeline
    subgraph "Indexing Pipeline"
        Extractor["Feature Extractor\n(src/ML_Pipeline/feature_extraction.py)\nTensorFlow/Keras"]:::compute
        Indexer["Elasticsearch Indexer\n(src/ML_Pipeline/elasticsearch_indexing.py)\nKNN plugin enabled"]:::compute
        ES["Elasticsearch Cluster\n(KNN plugin)"]:::external
    end

    Disk -->|"image files"| Extractor
    Extractor -->|"feature vectors"| Indexer
    Indexer -->|"indexed embeddings"| ES

    %% Query Pipeline
    subgraph "Query Pipeline"
        QueryImage["User Input Image"]:::external
        Engine["Query Engine\n(src/Engine.py)"]:::compute
        QExtractor["Feature Extractor\n(src/ML_Pipeline/feature_extraction.py)"]:::compute
        OutputWriter["Output Writer\n(output/response.json)"]:::storage
    end

    QueryImage -->|"input image"| Engine
    Engine -->|"preprocessed image"| QExtractor
    QExtractor -->|"query vector"| ES
    ES -->|"top-N results"| OutputWriter

    %% Developer Artifacts
    subgraph "Developer Artifacts"
        NI["Downloading-Images Notebook"]:::dev
        NF["Feature-Extraction Notebook"]:::dev
        NI2["Elasticsearch Indexing Notebook"]:::dev
        NT["Test ImageSearch Notebook"]:::dev
    end

    NI -.-> Downloader
    NF -.-> Extractor
    NI2 -.-> Indexer
    NT -.-> Engine

    %% Documentation & Config
    subgraph "Documentation & Config"
        Req["requirements.txt"]:::dev
        Readme["Readme.md"]:::dev
        Theory["Complete_theory_steps_overview.pdf"]:::dev
        Methodology["Solution_Methodology.pdf"]:::dev
    end

    %% Click Events
    click Downloader "https://github.com/krishnakaushik25/similar-images-finder/blob/main/src/ML_Pipeline/downloading_images.py"
    click Extractor "https://github.com/krishnakaushik25/similar-images-finder/blob/main/src/ML_Pipeline/feature_extraction.py"
    click Indexer "https://github.com/krishnakaushik25/similar-images-finder/blob/main/src/ML_Pipeline/elasticsearch_indexing.py"
    click Engine "https://github.com/krishnakaushik25/similar-images-finder/blob/main/src/Engine.py"
    click InputJSON "https://github.com/krishnakaushik25/similar-images-finder/blob/main/input/train.json"
    click OutputWriter "https://github.com/krishnakaushik25/similar-images-finder/blob/main/output/response.json"
    click NI "https://github.com/krishnakaushik25/similar-images-finder/blob/main/notebooks/Downloading-Images.ipynb"
    click NF "https://github.com/krishnakaushik25/similar-images-finder/blob/main/notebooks/Feature-Extraction.ipynb"
    click NI2 "https://github.com/krishnakaushik25/similar-images-finder/blob/main/notebooks/elasticsearch-indexing.ipynb"
    click NT "https://github.com/krishnakaushik25/similar-images-finder/blob/main/notebooks/test-ImageSearch.ipynb"
    click Req "https://github.com/krishnakaushik25/similar-images-finder/blob/main/requirements.txt"
    click Readme "https://github.com/krishnakaushik25/similar-images-finder/blob/main/Readme.md"
    click Theory "https://github.com/krishnakaushik25/similar-images-finder/blob/main/Complete_theory_steps_overview.pdf"
    click Methodology "https://github.com/krishnakaushik25/similar-images-finder/blob/main/Solution_Methodology.pdf"

    %% Styles
    classDef storage fill:#D0E6F5,stroke:#4A90E2,stroke-width:2px
    classDef compute fill:#DFF5D0,stroke:#7CB342,stroke-width:2px
    classDef external fill:#FFE4B5,stroke:#F39C12,stroke-width:2px
    classDef dev fill:#F0F0F0,stroke:#888888,stroke-dasharray: 5 5,stroke-width:1px
    class DataSource,ES,QueryImage external
    class Disk,InputJSON,OutputWriter storage
    class Downloader,Extractor,Indexer,Engine compute
    class NI,NF,NI2,NT,Req,Readme,Theory,Methodology dev
```
