# jayaprabhapalani/search-engine architecture

> Architecture diagram of the GitHub repository jayaprabhapalani/search-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jayaprabhapalani/search-engine
- Repository: https://github.com/jayaprabhapalani/search-engine

## Overview

This is a Hacker News story search product: users search and page through stories, get title autocomplete, inspect search analytics, and can request reindexing. The backend fetches and preprocesses stories, builds in-memory search indexes, serves hybrid search, and records analytics; reindexing can also run as a background Celery task. The graph preserves these distinct stages and the frontend flow. Source excerpts and the index support the shown calls; crawler internals and deployment wiring were not inspected.

Main entry points: [`hn_search/main.py`](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/main.py) (FastAPI routes), [`frontend/src/App.jsx`](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/App.jsx) (App coordinator), [`hn_search/tasks.py`](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/tasks.py) (Reindex task), [`frontend/src/services/api.js`](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/services/api.js) (API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Web client"]
  node_app["App coordinator<br/>[App.jsx]"]
  node_home["Home page<br/>[HomePage.jsx]"]
  node_results["Results page<br/>[ResultsPage.jsx]"]
  node_searchbar["Search bar<br/>[SearchBar.jsx]"]
  node_resultcard["Result card<br/>[ResultCard.jsx]"]
  node_searchhook["Search state<br/>[useSearch.js]"]
  node_autohook["Autocomplete state<br/>[useAutocomplete.js]"]
  node_analyticsui["Analytics views<br/>[AnalyticsPage.jsx]"]
  node_api_client["API client<br/>[api.js]"]
end

subgraph group_api["Search API"]
  node_backend["FastAPI routes<br/>[main.py]"]
  node_search["Hybrid search<br/>[search.py]"]
  node_trie["Title trie<br/>[trie.py]"]
end

subgraph group_indexing["Indexing"]
  node_preprocess["Text preprocessing<br/>[preprocessor.py]"]
  node_crawler["Story crawler<br/>[crawler.py]"]
  node_tasks["Reindex task<br/>[tasks.py]"]
  node_celery["Celery app<br/>[celery_app.py]"]
end

subgraph group_state["State and analytics"]
  node_cache["Result cache<br/>[cache.py]"]
  node_ratelimiter["Rate limiting<br/>[ratelimiter.py]"]
  node_models["Story and analytics models<br/>[models.py]"]
  node_database[("SQL database access<br/>[database.py]")]
end

node_user(("User"))
node_redis[("Redis")]

node_user -->|"uses"| node_app
node_app -->|"renders"| node_home
node_app -->|"renders"| node_results
node_app -->|"renders"| node_analyticsui
node_app -->|"uses"| node_searchhook
node_app -->|"uses"| node_autohook
node_app -->|"requests reindex"| node_api_client
node_home -->|"renders"| node_searchbar
node_results -->|"renders"| node_searchbar
node_results -->|"renders"| node_resultcard
node_searchhook -->|"requests search"| node_api_client
node_autohook -->|"requests suggestions"| node_api_client
node_analyticsui -->|"loads analytics"| node_api_client
node_api_client -->|"calls search routes"| node_backend
node_backend -->|"checks requests"| node_ratelimiter
node_backend -->|"loads and persists"| node_database
node_backend -->|"queries models"| node_models
node_backend -->|"reads and writes"| node_cache
node_backend -->|"runs hybrid search"| node_search
node_search -->|"uses text processing"| node_preprocess
node_backend -->|"fetches stories"| node_crawler
node_backend -->|"processes stories"| node_preprocess
node_backend -->|"builds suggestions"| node_trie
node_backend -->|"dispatches reindex"| node_tasks
node_tasks -->|"registers task"| node_celery
node_tasks -->|"fetches stories"| node_crawler
node_tasks -->|"processes stories"| node_preprocess
node_tasks -->|"saves stories"| node_database
node_cache -.->|"uses cache store"| node_redis
node_ratelimiter -.->|"checks limits"| node_redis

click node_app "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/App.jsx"
click node_home "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/pages/HomePage.jsx"
click node_results "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/pages/ResultsPage.jsx"
click node_searchbar "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/search/SearchBar.jsx"
click node_resultcard "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/search/ResultCard.jsx"
click node_searchhook "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/hooks/useSearch.js"
click node_autohook "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/hooks/useAutocomplete.js"
click node_analyticsui "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/analytics/AnalyticsPage.jsx"
click node_api_client "https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/services/api.js"
click node_backend "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/main.py"
click node_search "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/search.py"
click node_preprocess "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/preprocessor.py"
click node_crawler "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/crawler.py"
click node_tasks "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/tasks.py"
click node_celery "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/celery_app.py"
click node_trie "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/trie.py"
click node_cache "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/cache.py"
click node_ratelimiter "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/ratelimiter.py"
click node_models "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/models.py"
click node_database "https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/database.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_home,node_results,node_searchbar,node_resultcard,node_searchhook,node_autohook,node_analyticsui,node_api_client,node_user toneBlue
class node_backend,node_search,node_trie,node_redis toneAmber
class node_preprocess,node_crawler,node_tasks,node_celery toneMint
class node_cache,node_ratelimiter,node_models,node_database toneRose
```

## Components

### Web client

- [App coordinator](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/App.jsx): `frontend/src/App.jsx`
- [Home page](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/pages/HomePage.jsx): `frontend/src/pages/HomePage.jsx`
- [Results page](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/pages/ResultsPage.jsx): `frontend/src/pages/ResultsPage.jsx`
- [Search bar](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/search/SearchBar.jsx): `frontend/src/components/search/SearchBar.jsx`
- [Result card](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/search/ResultCard.jsx): `frontend/src/components/search/ResultCard.jsx`
- [Search state](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/hooks/useSearch.js): `frontend/src/hooks/useSearch.js`
- [Autocomplete state](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/hooks/useAutocomplete.js): `frontend/src/hooks/useAutocomplete.js`
- [Analytics views](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/components/analytics/AnalyticsPage.jsx): `frontend/src/components/analytics/AnalyticsPage.jsx`
- [API client](https://github.com/jayaprabhapalani/search-engine/blob/main/frontend/src/services/api.js): `frontend/src/services/api.js`

### Search API

- [FastAPI routes](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/main.py): `hn_search/main.py`
- [Hybrid search](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/search.py): `hn_search/search.py`
- [Title trie](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/trie.py): `hn_search/trie.py`

### Indexing

- [Text preprocessing](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/preprocessor.py): `hn_search/preprocessor.py`
- [Story crawler](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/crawler.py): `hn_search/crawler.py`
- [Reindex task](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/tasks.py): `hn_search/tasks.py`
- [Celery app](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/celery_app.py): `hn_search/celery_app.py`

### State and analytics

- [Result cache](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/cache.py): `hn_search/cache.py`
- [Rate limiting](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/ratelimiter.py): `hn_search/ratelimiter.py`
- [Story and analytics models](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/models.py): `hn_search/models.py`
- [SQL database access](https://github.com/jayaprabhapalani/search-engine/blob/main/hn_search/database.py): `hn_search/database.py`

### Other components

- **User**
- **Redis**

## Connections

- User → App coordinator: uses (evidence: `frontend/src/App.jsx`)
- App coordinator → Home page: renders (evidence: `frontend/src/App.jsx`)
- App coordinator → Results page: renders (evidence: `frontend/src/App.jsx`)
- App coordinator → Analytics views: renders (evidence: `frontend/src/App.jsx`)
- App coordinator → Search state: uses (evidence: `frontend/src/App.jsx`)
- App coordinator → Autocomplete state: uses (evidence: `frontend/src/App.jsx`)
- App coordinator → API client: requests reindex (evidence: `frontend/src/App.jsx`)
- Home page → Search bar: renders (evidence: `frontend/src/pages/HomePage.jsx`)
- Results page → Search bar: renders (evidence: `frontend/src/pages/ResultsPage.jsx`)
- Results page → Result card: renders (evidence: `frontend/src/pages/ResultsPage.jsx`)
- Search state → API client: requests search (evidence: `frontend/src/hooks/useSearch.js`)
- Autocomplete state → API client: requests suggestions (evidence: `frontend/src/hooks/useAutocomplete.js`)
- Analytics views → API client: loads analytics (evidence: `frontend/src/components/analytics/AnalyticsPage.jsx`)
- API client → FastAPI routes: calls search routes (evidence: `frontend/src/services/api.js`)
- FastAPI routes → Rate limiting: checks requests (evidence: `hn_search/main.py`)
- FastAPI routes → SQL database access: loads and persists (evidence: `hn_search/main.py`)
- FastAPI routes → Story and analytics models: queries models (evidence: `hn_search/main.py`)
- FastAPI routes → Result cache: reads and writes (evidence: `hn_search/main.py`)
- FastAPI routes → Hybrid search: runs hybrid search (evidence: `hn_search/main.py`)
- Hybrid search → Text preprocessing: uses text processing (evidence: `hn_search/search.py`)
- FastAPI routes → Story crawler: fetches stories (evidence: `hn_search/main.py`)
- FastAPI routes → Text preprocessing: processes stories (evidence: `hn_search/main.py`)
- FastAPI routes → Title trie: builds suggestions (evidence: `hn_search/main.py`)
- FastAPI routes → Reindex task: dispatches reindex (evidence: `hn_search/main.py`)
- Reindex task → Celery app: registers task (evidence: `hn_search/tasks.py`)
- Reindex task → Story crawler: fetches stories (evidence: `hn_search/tasks.py`)
- Reindex task → Text preprocessing: processes stories (evidence: `hn_search/tasks.py`)
- Reindex task → SQL database access: saves stories (evidence: `hn_search/tasks.py`)
- Result cache → Redis: uses cache store
- Rate limiting → Redis: checks limits
