# aemadhavan/lumina architecture

> Architecture diagram of the GitHub repository aemadhavan/lumina, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aemadhavan/lumina
- Repository: https://github.com/aemadhavan/lumina

## Overview

LUMINA is a Perplexity-style research search product: a user asks in the web app and receives a streamed, cited answer using live web results and optionally indexed documents; deep search plans and researches sub-questions before merging results. Threads, memory, Spaces/document ingestion, and run statistics support that workflow. The gateway validates and proxies requests to the agent. README documents major capabilities not fully represented in sampled source; those paths are shown at their existing enclosing directories, and uncertain internal wiring is omitted.

Main entry points: [`backend/agent/src/index.ts`](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/index.ts) (Agent service), [`backend/agent/src/db.ts`](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/db.ts) (MongoDB and files), [`backend/agent/src/tools/index.ts`](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/index.ts) (Research tools), [`backend/agent/src/loop.ts`](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/loop.ts) (Research loop)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Web client"]
  node_app["Search interface<br/>[App.tsx]"]
  node_api["API client<br/>[api.ts]"]
  node_answer_view["Answer and citations<br/>[AnswerText.tsx]"]
end

subgraph group_edge["Gateway"]
  node_gateway["HTTP gateway<br/>[index.ts]"]
  node_rate_limit["Rate limiter<br/>[rate_limiter.ts]"]
end

subgraph group_research["Research agent"]
  node_agent_service["Agent service<br/>[index.ts]"]
  node_quick_loop["Research loop<br/>[loop.ts]"]
  node_deep_loop["Deep search<br/>[deep_loop.ts]"]
  node_planner["Research planner<br/>[plan_research.ts]"]
  node_tool_registry["Research tools<br/>[index.ts]"]
  node_web_search["Web search<br/>[web_search.ts]"]
  node_page_fetch["Page fetch<br/>[fetch_page.ts]"]
  node_memory_tools["Memory tools<br/>[memory.ts]"]
  node_llm["LLM streaming<br/>[llm.ts]"]
end

subgraph group_knowledge["Knowledge and jobs"]
  node_doc_search["Document retrieval"]
  node_embedding["Embeddings<br/>[embeddings.ts]"]
  node_spaces_routes["Spaces and uploads<br/>[spaces.ts]"]
  node_worker["Ingestion worker<br/>[worker.ts]"]
end

subgraph group_state["Persistence"]
  node_thread_routes["Threads<br/>[threads.ts]"]
  node_memory_routes["Memory API<br/>[memory.ts]"]
  node_stats_routes["Usage statistics<br/>[stats.ts]"]
  node_run_log["Run records<br/>[runs.ts]"]
  node_search_cache["Search cache<br/>[cache.ts]"]
  node_database[("MongoDB and files<br/>[db.ts]")]
end

node_user(("Researcher"))
node_web_provider["Web search provider"]
node_llm_provider["LLM provider"]
node_embedding_provider["Embedding provider"]

node_user -->|"asks"| node_app
node_app -->|"submits query"| node_api
node_api -->|"sends requests"| node_gateway
node_gateway -->|"limits requests"| node_rate_limit
node_gateway -->|"proxies requests"| node_agent_service
node_agent_service -->|"dispatches ask"| node_quick_loop
node_quick_loop -->|"invokes tools"| node_tool_registry
node_quick_loop -->|"streams answer"| node_llm
node_tool_registry -->|"dispatches search"| node_web_search
node_tool_registry -->|"dispatches fetch"| node_page_fetch
node_tool_registry -->|"dispatches retrieval"| node_doc_search
node_tool_registry -->|"dispatches memory"| node_memory_tools
node_web_search -->|"searches web"| node_web_provider
node_llm -->|"streams completion"| node_llm_provider
node_doc_search -->|"embeds query"| node_embedding
node_embedding -->|"requests vectors"| node_embedding_provider
node_doc_search -->|"reads chunks and documents"| node_database
node_quick_loop -->|"checks cache"| node_search_cache
node_quick_loop -->|"records run"| node_run_log
node_agent_service -->|"serves thread routes"| node_thread_routes
node_agent_service -->|"serves Spaces routes"| node_spaces_routes
node_agent_service -->|"serves memory routes"| node_memory_routes
node_agent_service -->|"serves stats routes"| node_stats_routes
node_spaces_routes -->|"stores uploads and jobs"| node_database
node_worker -->|"claims jobs and stores chunks"| node_database
node_worker -->|"embeds chunks"| node_embedding
node_agent_service -.->|"queues ingestion"| node_worker
node_deep_loop -->|"plans sub-questions"| node_planner
node_planner -->|"requests plan"| node_llm
node_deep_loop -->|"researches sub-questions"| node_tool_registry
node_agent_service -->|"dispatches deep search"| node_deep_loop
node_thread_routes -->|"stores threads"| node_database
node_memory_routes -->|"stores memories"| node_database
node_stats_routes -->|"reads statistics"| node_database
node_memory_tools -->|"reads and writes memory"| node_database
node_app -->|"renders answer"| node_answer_view
node_gateway -->|"streams response"| node_app

click node_app "https://github.com/aemadhavan/lumina/blob/main/web/src/App.tsx"
click node_api "https://github.com/aemadhavan/lumina/blob/main/web/src/api.ts"
click node_answer_view "https://github.com/aemadhavan/lumina/blob/main/web/src/components/AnswerText.tsx"
click node_gateway "https://github.com/aemadhavan/lumina/blob/main/backend/gateway/src/index.ts"
click node_rate_limit "https://github.com/aemadhavan/lumina/blob/main/backend/gateway/src/rate_limiter.ts"
click node_agent_service "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/index.ts"
click node_quick_loop "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/loop.ts"
click node_deep_loop "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/deep_loop.ts"
click node_planner "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/plan_research.ts"
click node_tool_registry "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/index.ts"
click node_web_search "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/web_search.ts"
click node_page_fetch "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/fetch_page.ts"
click node_doc_search "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/search_documents.ts"
click node_memory_tools "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/memory.ts"
click node_llm "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/llm.ts"
click node_embedding "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/embeddings.ts"
click node_spaces_routes "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/spaces.ts"
click node_worker "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/worker.ts"
click node_thread_routes "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/threads.ts"
click node_memory_routes "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/memory.ts"
click node_stats_routes "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/stats.ts"
click node_run_log "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/runs.ts"
click node_search_cache "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/cache.ts"
click node_database "https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/db.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_api,node_answer_view toneBlue
class node_gateway,node_rate_limit toneAmber
class node_agent_service,node_quick_loop,node_deep_loop,node_planner,node_tool_registry,node_web_search,node_page_fetch,node_memory_tools,node_llm toneMint
class node_doc_search,node_embedding,node_spaces_routes,node_worker toneRose
class node_thread_routes,node_memory_routes,node_stats_routes,node_run_log,node_search_cache,node_database,node_user,node_web_provider,node_llm_provider,node_embedding_provider toneIndigo
```

## Components

### Web client

- [Search interface](https://github.com/aemadhavan/lumina/blob/main/web/src/App.tsx): `web/src/App.tsx`
- [API client](https://github.com/aemadhavan/lumina/blob/main/web/src/api.ts): `web/src/api.ts`
- [Answer and citations](https://github.com/aemadhavan/lumina/blob/main/web/src/components/AnswerText.tsx): `web/src/components/AnswerText.tsx`

### Gateway

- [HTTP gateway](https://github.com/aemadhavan/lumina/blob/main/backend/gateway/src/index.ts): `backend/gateway/src/index.ts`
- [Rate limiter](https://github.com/aemadhavan/lumina/blob/main/backend/gateway/src/rate_limiter.ts): `backend/gateway/src/rate_limiter.ts`

### Research agent

- [Agent service](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/index.ts): `backend/agent/src/index.ts`
- [Research loop](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/loop.ts): `backend/agent/src/loop.ts`
- [Deep search](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/deep_loop.ts): `backend/agent/src/deep_loop.ts`
- [Research planner](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/plan_research.ts): `backend/agent/src/tools/plan_research.ts`
- [Research tools](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/index.ts): `backend/agent/src/tools/index.ts`
- [Web search](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/web_search.ts): `backend/agent/src/tools/web_search.ts`
- [Page fetch](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/fetch_page.ts): `backend/agent/src/tools/fetch_page.ts`
- [Memory tools](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/memory.ts): `backend/agent/src/tools/memory.ts`
- [LLM streaming](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/llm.ts): `backend/agent/src/llm.ts`

### Knowledge and jobs

- [Document retrieval](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/tools/search_documents.ts): `backend/agent/src/tools/search_documents.ts`
- [Embeddings](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/embeddings.ts): `backend/agent/src/embeddings.ts`
- [Spaces and uploads](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/spaces.ts): `backend/agent/src/routes/spaces.ts`
- [Ingestion worker](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/worker.ts): `backend/agent/src/worker.ts`

### Persistence

- [Threads](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/threads.ts): `backend/agent/src/routes/threads.ts`
- [Memory API](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/memory.ts): `backend/agent/src/routes/memory.ts`
- [Usage statistics](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/routes/stats.ts): `backend/agent/src/routes/stats.ts`
- [Run records](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/runs.ts): `backend/agent/src/runs.ts`
- [Search cache](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/cache.ts): `backend/agent/src/cache.ts`
- [MongoDB and files](https://github.com/aemadhavan/lumina/blob/main/backend/agent/src/db.ts): `backend/agent/src/db.ts`

### Other components

- **Researcher**
- **Web search provider**
- **LLM provider**
- **Embedding provider**

## Connections

- Researcher → Search interface: asks
- Search interface → API client: submits query
- API client → HTTP gateway: sends requests
- HTTP gateway → Rate limiter: limits requests
- HTTP gateway → Agent service: proxies requests
- Agent service → Research loop: dispatches ask
- Research loop → Research tools: invokes tools
- Research loop → LLM streaming: streams answer
- Research tools → Web search: dispatches search
- Research tools → Page fetch: dispatches fetch
- Research tools → Document retrieval: dispatches retrieval
- Research tools → Memory tools: dispatches memory
- Web search → Web search provider: searches web
- LLM streaming → LLM provider: streams completion
- Document retrieval → Embeddings: embeds query
- Embeddings → Embedding provider: requests vectors
- Document retrieval → MongoDB and files: reads chunks and documents
- Research loop → Search cache: checks cache
- Research loop → Run records: records run
- Agent service → Threads: serves thread routes
- Agent service → Spaces and uploads: serves Spaces routes
- Agent service → Memory API: serves memory routes
- Agent service → Usage statistics: serves stats routes
- Spaces and uploads → MongoDB and files: stores uploads and jobs
- Ingestion worker → MongoDB and files: claims jobs and stores chunks
- Ingestion worker → Embeddings: embeds chunks
- Agent service → Ingestion worker: queues ingestion
- Deep search → Research planner: plans sub-questions
- Research planner → LLM streaming: requests plan
- Deep search → Research tools: researches sub-questions
- Agent service → Deep search: dispatches deep search
- Threads → MongoDB and files: stores threads
- Memory API → MongoDB and files: stores memories
- Usage statistics → MongoDB and files: reads statistics
- Memory tools → MongoDB and files: reads and writes memory
- Search interface → Answer and citations: renders answer
- HTTP gateway → Search interface: streams response