# marticasasn/irwa-2026-search-engine architecture

> Architecture diagram of the GitHub repository marticasasn/irwa-2026-search-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marticasasn/irwa-2026-search-engine
- Repository: https://github.com/marticasasn/irwa-2026-search-engine

## Overview

This repository is a course project for a fashion-product search engine, with a documented preprocessing/EDA notebook and Python modules for indexing, conjunctive retrieval, TF-IDF ranking, and evaluation. The main workflow prepares the raw product corpus, writes processed JSONL, builds an inverted index, and ranks matching products for a query. The README also describes EDA and labelled-query sanity checks. RAG, filtering, and UI are planned project stages, not evidenced as implemented runtime components. Notebook details are represented from README documentation; its source was not sampled.

Main entry points: [`src/ranking.py`](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/ranking.py) (TF-IDF ranking), [`IRWA_Part1_Text_Processing_EDA.ipynb`](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/IRWA_Part1_Text_Processing_EDA.ipynb) (Preprocessing and EDA), [`src/indexing.py`](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/indexing.py) (Inverted index), [`src/preprocessing.py`](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/preprocessing.py) (Term processing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_preparation["Preparation and analysis"]
  node_prep_eda["Preprocessing and EDA"]
  node_processed_corpus["Processed corpus"]
end

subgraph group_retrieval["Search runtime"]
  node_text_processing["Term processing<br/>[preprocessing.py]"]
  node_indexing["Inverted index<br/>[indexing.py]"]
  node_retrieval_search["Conjunctive retrieval<br/>[indexing.py]"]
  node_tfidf_ranking["TF-IDF ranking<br/>[ranking.py]"]
end

subgraph group_assessment["Evaluation"]
  node_validation_labels["Validation labels"]
  node_evaluation["Ranking metrics<br/>[evaluation.py]"]
end

node_user(("Search user"))
node_raw_corpus["Product corpus"]

node_raw_corpus -->|"loads for processing"| node_prep_eda
node_prep_eda -->|"writes processed data"| node_processed_corpus
node_validation_labels -->|"checks query matches"| node_prep_eda
node_processed_corpus -->|"loads corpus"| node_indexing
node_indexing -->|"uses terms"| node_text_processing
node_user -.->|"submits query"| node_tfidf_ranking
node_tfidf_ranking -->|"requests candidates"| node_retrieval_search
node_retrieval_search -->|"processes query"| node_text_processing
node_tfidf_ranking -->|"processes query terms"| node_text_processing
node_tfidf_ranking -->|"scores postings"| node_indexing
node_evaluation -->|"computes metrics"| node_evaluation

click node_raw_corpus "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/fashion_products_dataset.json"
click node_prep_eda "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/IRWA_Part1_Text_Processing_EDA.ipynb"
click node_processed_corpus "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/processed_corpus.json"
click node_validation_labels "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/validation_labels.csv"
click node_text_processing "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/preprocessing.py"
click node_indexing "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/indexing.py"
click node_retrieval_search "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/indexing.py"
click node_tfidf_ranking "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/ranking.py"
click node_evaluation "https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/evaluation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prep_eda,node_processed_corpus,node_user toneBlue
class node_text_processing,node_indexing,node_retrieval_search,node_tfidf_ranking toneAmber
class node_validation_labels,node_evaluation toneMint
class node_raw_corpus toneTeal
```

## Components

### Preparation and analysis

- [Preprocessing and EDA](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/IRWA_Part1_Text_Processing_EDA.ipynb): `IRWA_Part1_Text_Processing_EDA.ipynb`
- [Processed corpus](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/processed_corpus.json): `processed_corpus.json`

### Search runtime

- [Term processing](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/preprocessing.py): `src/preprocessing.py`
- [Inverted index](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/indexing.py): `src/indexing.py`
- [Conjunctive retrieval](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/indexing.py): `src/indexing.py`
- [TF-IDF ranking](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/ranking.py): `src/ranking.py`

### Evaluation

- [Validation labels](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/validation_labels.csv): `validation_labels.csv`
- [Ranking metrics](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/src/evaluation.py): `src/evaluation.py`

### Other components

- **Search user**
- [Product corpus](https://github.com/marticasasn/irwa-2026-search-engine/blob/main/fashion_products_dataset.json): `fashion_products_dataset.json`

## Connections

- Product corpus → Preprocessing and EDA: loads for processing (evidence: `README.md`)
- Preprocessing and EDA → Processed corpus: writes processed data (evidence: `README.md`)
- Validation labels → Preprocessing and EDA: checks query matches (evidence: `README.md`)
- Processed corpus → Inverted index: loads corpus (evidence: `src/indexing.py`)
- Inverted index → Term processing: uses terms (evidence: `src/indexing.py`)
- Search user → TF-IDF ranking: submits query
- TF-IDF ranking → Conjunctive retrieval: requests candidates (evidence: `src/ranking.py`)
- Conjunctive retrieval → Term processing: processes query (evidence: `src/indexing.py`)
- TF-IDF ranking → Term processing: processes query terms (evidence: `src/ranking.py`)
- TF-IDF ranking → Inverted index: scores postings (evidence: `src/ranking.py`)
- Ranking metrics → Ranking metrics: computes metrics (evidence: `src/evaluation.py`)
