# shreyansh200528/two-tower-recommendation-engine architecture

> Architecture diagram of the GitHub repository shreyansh200528/two-tower-recommendation-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shreyansh200528/two-tower-recommendation-engine
- Repository: https://github.com/shreyansh200528/two-tower-recommendation-engine

## Overview

This repository implements a MovieLens recommendation engine whose main user workflow is a request entering interactive inference and receiving ranked, novel movie recommendations. Known users follow two-tower embedding, FAISS candidate retrieval, seen-item filtering, and LightGBM re-ranking; new users take demographic retrieval or anonymous Bayesian popularity fallback. Training separately prepares chronological data, learns the towers, builds the index and ranker, and evaluates ranking quality. The excerpts show these main orchestration paths; some training and inference details are documented but not fully excerpted.

Main entry points: [`src/interactive_inference.py`](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/interactive_inference.py) (Interactive inference), [`src/train_pipeline.py`](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/train_pipeline.py) (Training pipeline), [`src/cold_start.py`](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/cold_start.py) (Demographic fallback), [`src/data_preprocessing.py`](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/data_preprocessing.py) (MovieLens preprocessing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_preprocessing["MovieLens preprocessing"]
  node_temporal["Temporal split and sampling"]
end

subgraph group_learning["Representation learning"]
  node_training["Training pipeline<br/>[train_pipeline.py]"]
  node_towers["Two-tower model<br/>[two_tower_model.py]"]
  node_weights["Tower weights<br/>[two_tower.pt]"]
end

subgraph group_ranking["Retrieval and ranking"]
  node_indexing["FAISS search engine<br/>[vector_search.py]"]
  node_faiss_index[("FAISS HNSW index<br/>[faiss_hnsw.index]")]
  node_seen_filter["Seen-item filtering"]
  node_reranker["LightGBM re-ranker<br/>[reranker.py]"]
  node_ranker_model["Ranker model artifact"]
end

subgraph group_runtime["Inference and evaluation"]
  node_inference["Interactive inference"]
  node_cold_handler["Demographic fallback<br/>[cold_start.py]"]
  node_bayesian["Bayesian popularity fallback<br/>[cold_start.py]"]
  node_evaluation["Ranking evaluation<br/>[evaluate.py]"]
end

node_requester(("Recommendation requester"))

node_requester -->|"requests recommendations"| node_inference
node_inference -->|"loads data"| node_preprocessing
node_preprocessing -->|"splits chronologically"| node_temporal
node_training -->|"prepares data"| node_preprocessing
node_training -->|"trains model"| node_towers
node_training -->|"builds index"| node_indexing
node_training -->|"trains ranker"| node_reranker
node_training -->|"evaluates results"| node_evaluation
node_indexing -->|"writes index"| node_faiss_index
node_inference -->|"loads weights"| node_weights
node_inference -->|"embeds user"| node_towers
node_inference -->|"searches candidates"| node_faiss_index
node_inference -->|"filters seen items"| node_seen_filter
node_inference -->|"re-ranks candidates"| node_reranker
node_inference -->|"handles demographics"| node_cold_handler
node_inference -->|"handles anonymous users"| node_bayesian
node_cold_handler -->|"builds demographic vector"| node_towers
node_cold_handler -->|"retrieves candidates"| node_faiss_index
node_cold_handler -->|"blends quality scores"| node_bayesian

click node_inference "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/interactive_inference.py"
click node_preprocessing "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/data_preprocessing.py"
click node_temporal "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/data_preprocessing.py"
click node_training "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/train_pipeline.py"
click node_towers "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/two_tower_model.py"
click node_weights "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/two_tower.pt"
click node_indexing "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/vector_search.py"
click node_faiss_index "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/faiss_hnsw.index"
click node_seen_filter "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/interactive_inference.py"
click node_reranker "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/reranker.py"
click node_ranker_model "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/lightgbm_reranker.txt"
click node_cold_handler "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/cold_start.py"
click node_bayesian "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/cold_start.py"
click node_evaluation "https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/evaluate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_preprocessing,node_temporal toneBlue
class node_training,node_towers,node_weights toneAmber
class node_indexing,node_faiss_index,node_seen_filter,node_reranker,node_ranker_model,node_requester toneMint
class node_inference,node_cold_handler,node_bayesian,node_evaluation toneRose
```

## Components

### Data preparation

- [MovieLens preprocessing](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/data_preprocessing.py): `src/data_preprocessing.py`
- [Temporal split and sampling](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/data_preprocessing.py): `src/data_preprocessing.py`

### Representation learning

- [Training pipeline](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/train_pipeline.py): `src/train_pipeline.py`
- [Two-tower model](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/two_tower_model.py): `src/two_tower_model.py`
- [Tower weights](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/two_tower.pt): `models_baseline_run1/two_tower.pt`

### Retrieval and ranking

- [FAISS search engine](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/vector_search.py): `src/vector_search.py`
- [FAISS HNSW index](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/faiss_hnsw.index): `models_baseline_run1/faiss_hnsw.index`
- [Seen-item filtering](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/interactive_inference.py): `src/interactive_inference.py`
- [LightGBM re-ranker](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/reranker.py): `src/reranker.py`
- [Ranker model artifact](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/models_baseline_run1/lightgbm_reranker.txt): `models_baseline_run1/lightgbm_reranker.txt`

### Inference and evaluation

- [Interactive inference](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/interactive_inference.py): `src/interactive_inference.py`
- [Demographic fallback](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/cold_start.py): `src/cold_start.py`
- [Bayesian popularity fallback](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/cold_start.py): `src/cold_start.py`
- [Ranking evaluation](https://github.com/shreyansh200528/two-tower-recommendation-engine/blob/main/src/evaluate.py): `src/evaluate.py`

### Other components

- **Recommendation requester**

## Connections

- Recommendation requester → Interactive inference: requests recommendations (evidence: `README.md`)
- Interactive inference → MovieLens preprocessing: loads data (evidence: `src/interactive_inference.py`)
- MovieLens preprocessing → Temporal split and sampling: splits chronologically (evidence: `src/data_preprocessing.py`)
- Training pipeline → MovieLens preprocessing: prepares data (evidence: `src/train_pipeline.py`)
- Training pipeline → Two-tower model: trains model (evidence: `src/train_pipeline.py`)
- Training pipeline → FAISS search engine: builds index (evidence: `src/train_pipeline.py`)
- Training pipeline → LightGBM re-ranker: trains ranker (evidence: `src/train_pipeline.py`)
- Training pipeline → Ranking evaluation: evaluates results (evidence: `src/train_pipeline.py`)
- FAISS search engine → FAISS HNSW index: writes index (evidence: `src/vector_search.py`)
- Interactive inference → Tower weights: loads weights (evidence: `src/interactive_inference.py`)
- Interactive inference → Two-tower model: embeds user (evidence: `src/interactive_inference.py`)
- Interactive inference → FAISS HNSW index: searches candidates (evidence: `src/interactive_inference.py`)
- Interactive inference → Seen-item filtering: filters seen items (evidence: `src/interactive_inference.py`)
- Interactive inference → LightGBM re-ranker: re-ranks candidates (evidence: `README.md`)
- Interactive inference → Demographic fallback: handles demographics (evidence: `src/interactive_inference.py`)
- Interactive inference → Bayesian popularity fallback: handles anonymous users (evidence: `src/interactive_inference.py`)
- Demographic fallback → Two-tower model: builds demographic vector (evidence: `src/cold_start.py`)
- Demographic fallback → FAISS HNSW index: retrieves candidates (evidence: `src/cold_start.py`)
- Demographic fallback → Bayesian popularity fallback: blends quality scores (evidence: `src/cold_start.py`)
