How marticasasn/irwa-2026-search-engine works
This repository is a course project for a fashion-product search engine, with a documented preprocessing/EDA notebook and Python modules for indexing, conjunctive retrieval, TF-IDF ranking, and evaluation. The main workflow prepares the raw product corpus, writes processed JSONL, builds an inverted index, and ranks matching products for a query. The README also describes EDA and labelled-query sanity checks. RAG, filtering, and UI are planned project stages, not evidenced as implemented runtime components. Notebook details are represented from README documentation; its source was not sampled.
Main entry points: ranking.py TF-IDF ranking · IRWA_Part1_Text_Processing_EDA.ipynb Preprocessing and EDA · indexing.py Inverted index · preprocessing.py Term processing
Components 10
Preparation and analysis
- Preprocessing and EDA
IRWA_Part1_Text_Processing_EDA.ipynb - Processed corpus
processed_corpus.json
Search runtime
- Term processing
src/preprocessing.py - Inverted index
src/indexing.py - Conjunctive retrieval
src/indexing.py - TF-IDF ranking
src/ranking.py
Evaluation
- Validation labels
validation_labels.csv - Ranking metrics
src/evaluation.py
Other components
- Search user
- Product corpus
fashion_products_dataset.json
Connections 11
- Product corpus → Preprocessing and EDA: loads for processing
README.md - Preprocessing and EDA → Processed corpus: writes processed data
README.md - Validation labels → Preprocessing and EDA: checks query matches
README.md - Processed corpus → Inverted index: loads corpus
src/indexing.py - Inverted index → Term processing: uses terms
src/indexing.py - Search user → TF-IDF ranking: submits query
- TF-IDF ranking → Conjunctive retrieval: requests candidates
src/ranking.py - Conjunctive retrieval → Term processing: processes query
src/indexing.py - TF-IDF ranking → Term processing: processes query terms
src/ranking.py - TF-IDF ranking → Inverted index: scores postings
src/ranking.py - Ranking metrics → Ranking metrics: computes metrics
src/evaluation.py