# larrysman/e-commerce-with-nlp architecture

> Architecture diagram of the GitHub repository larrysman/e-commerce-with-nlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/larrysman/e-commerce-with-nlp
- Repository: https://github.com/larrysman/e-commerce-with-nlp

## Overview

##### Project Type and Purpose
- This is a self-contained, local “smart search” CLI tool for an e-commerce product catalog.
- It leverages NLP (TF-IDF + cosine similarity) to extract constraints and rank products.
- Purpose: Demonstrate modular NLP pipelines, easy to extend or swap in production components (vector DB, embeddings, UI).

##### File Structure
Top-level:
- README.md, requirements.txt, project_structure.py
- e-commerce-with-nlp/
Under e-commerce-with-nlp/:
- data/
– product.py (data model or sample catalog)
- src/
– main_product_search.py (CLI entry point)
– product_loader.py (loads JSON catalog into DataFrame)
– searchable_text.py (builds “search_text” per product)
– user_query_extraction.py (parses price/rating/category constraints)
– product_filter_by_extracted_user_query.py (applies filters)
– product_pipeline.py (orchestrates full pipeline: load → transform → rank)
– __init__.py
– experiment.ipynb (proof-of-concept notebook)

##### Extract Key Architecture Insights from README
- Separation of concerns: each module does one job (loader, extractor, filter, ranker).
- In-memory TF-IDF vectorizer + cosine similarity.
- Interactive CLI interface.
- Future extensibility points: vector DB, REST/UI layer.

##### Main Components
A. CLI Interface
– Component: main_product_search.py
– Receives raw user query, displays results.
B. Query Extraction
– Component: user_query_extraction
– Outputs price/rating/category constraints + free-text.
C. Data Ingestion
– Component: product_loader
– Loads PRODUCT_CATALOG into an in-memory DataFrame.
D. Searchable Text Builder
– Component: searchable_text
– Combines name/description/category/price/rating into a corpus string.
E. TF-IDF Vectorizer
– Component: product_pipeline (initializes vectorizer, builds TFIDF_matrix)
– Stores: vocabulary, document vectors.
F. Filtering
– Component: product_filter_by_extracted_user_query
– Takes DataFrame + constraints, outputs filtered subset.
G. Ranking
– Component: product_pipeline (or separate ranking module)
– Computes cosine similarity between query vector and product vectors.
H. Presentation
– CLI prints top-N products with metadata and scores.

##### Relationships & Data Flows
1. User types query → CLI
2. CLI → user_query_extraction → parsed constraints + text fragment
3. product_loader → DataFrame of all products
4. searchable_text → DataFrame enriched with “search_text” column
5. product_pipeline (startup) → TF-IDF vectorizer → TFIDF_matrix
6. CLI passes query text → TF-IDF vectorizer → query_vector
7. Filter module applies constraints → filtered DataFrame
8. Ranking module computes similarities → sorted results
9. CLI renders output

##### Architectural Patterns & Principles
- Pipeline pattern: clearly defined stages (load → transform → filter → rank → output)
- Separation of concerns: each Python module is single responsibility
- Dependency injection: pass vectorizer and DataFrame into ranking
- Extensibility points: swap TF-IDF for embeddings, add UI layer

##### Technologies & Frameworks
- Core: Python, pandas, scikit-learn (TfidfVectorizer), NumPy
- CLI: built-in argparse or input() loop
- Data model: simple JSON loaded via pandas
- Future: Streamlit/Flask + React, FAISS/Milvus

## Diagram (Mermaid)

```mermaid
flowchart TD
    User((User)):::external

    subgraph "Startup (Pipeline Build)"
        Loader["Product Loader"]:::data
        Catalog(("Product Catalog")):::data
        Searchable["Searchable Text Builder"]:::pipeline
        VectorizerInit["Vectorizer Init (TF-IDF)"]:::pipeline
    end

    subgraph "Per-Query Pipeline"
        CLI["CLI Interface"]:::cli
        Extraction["Query Extraction"]:::pipeline
        QueryVector["Query Vectorization"]:::pipeline
        Filter["Filter Constraints"]:::pipeline
        Rank["Ranking (Cosine Similarity)"]:::pipeline
        Output["Presentation (CLI Output)"]:::cli
    end

    subgraph "Future Extensions"
        FutureEmbedding["Embedding Service"]:::future
        FutureUI["Web UI"]:::future
    end

    Catalog --> Loader
    Loader -->|"DataFrame"| Searchable
    Searchable -->|"enriched DataFrame"| VectorizerInit

    User -->|"raw query text"| CLI
    CLI -->|"raw query text"| Extraction
    Extraction -->|"parsed constraints & text"| Filter
    Extraction -->|"free-text fragment"| QueryVector
    VectorizerInit -->|"TFIDF matrix"| QueryVector
    Filter -->|"filtered DataFrame"| Rank
    QueryVector -->|"query_vector"| Rank
    Rank -->|"scored results"| Output

    FutureEmbedding -.-> VectorizerInit
    FutureEmbedding -.-> QueryVector
    FutureUI -.-> CLI

    click CLI "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/main_product_search.py"
    click Extraction "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/user_query_extraction.py"
    click Loader "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/product_loader.py"
    click Searchable "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/searchable_text.py"
    click VectorizerInit "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/product_pipeline.py"
    click Rank "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/product_pipeline.py"
    click Filter "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/src/product_filter_by_extracted_user_query.py"
    click Catalog "https://github.com/larrysman/e-commerce-with-nlp/blob/main/e-commerce-with-nlp/data/product.py"

    classDef cli fill:#f9f,stroke:#333,stroke-width:1px
    classDef data fill:#afa,stroke:#333,stroke-width:1px
    classDef pipeline fill:#aaf,stroke:#333,stroke-width:1px
    classDef future fill:#ffa,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef external fill:#eee,stroke:#333,stroke-width:1px
```
