How aemadhavan/lumina works
LUMINA is a Perplexity-style research search product: a user asks in the web app and receives a streamed, cited answer using live web results and optionally indexed documents; deep search plans and researches sub-questions before merging results. Threads, memory, Spaces/document ingestion, and run statistics support that workflow. The gateway validates and proxies requests to the agent. README documents major capabilities not fully represented in sampled source; those paths are shown at their existing enclosing directories, and uncertain internal wiring is omitted.
Main entry points: src/index.ts Agent service · db.ts MongoDB and files · tools/index.ts Research tools · loop.ts Research loop
Components 28
Web client
- Search interface
web/src/App.tsx - API client
web/src/api.ts - Answer and citations
web/src/components/AnswerText.tsx
Gateway
- HTTP gateway
backend/gateway/src/index.ts - Rate limiter
backend/gateway/src/rate_limiter.ts
Research agent
- Agent service
backend/agent/src/index.ts - Research loop
backend/agent/src/loop.ts - Deep search
backend/agent/src/deep_loop.ts - Research planner
backend/agent/src/tools/plan_research.ts - Research tools
backend/agent/src/tools/index.ts - Web search
backend/agent/src/tools/web_search.ts - Page fetch
backend/agent/src/tools/fetch_page.ts - Memory tools
backend/agent/src/tools/memory.ts - LLM streaming
backend/agent/src/llm.ts
Knowledge and jobs
- Document retrieval
backend/agent/src/tools/search_documents.ts - Embeddings
backend/agent/src/embeddings.ts - Spaces and uploads
backend/agent/src/routes/spaces.ts - Ingestion worker
backend/agent/src/worker.ts
Persistence
- Threads
backend/agent/src/routes/threads.ts - Memory API
backend/agent/src/routes/memory.ts - Usage statistics
backend/agent/src/routes/stats.ts - Run records
backend/agent/src/runs.ts - Search cache
backend/agent/src/cache.ts - MongoDB and files
backend/agent/src/db.ts
Other components
- Researcher
- Web search provider
- LLM provider
- Embedding provider
Connections 37
- Researcher → Search interface: asks
- Search interface → API client: submits query
- API client → HTTP gateway: sends requests
- HTTP gateway → Rate limiter: limits requests
- HTTP gateway → Agent service: proxies requests
- Agent service → Research loop: dispatches ask
- Research loop → Research tools: invokes tools
- Research loop → LLM streaming: streams answer
- Research tools → Web search: dispatches search
- Research tools → Page fetch: dispatches fetch
- Research tools → Document retrieval: dispatches retrieval
- Research tools → Memory tools: dispatches memory
- Web search → Web search provider: searches web
- LLM streaming → LLM provider: streams completion
- Document retrieval → Embeddings: embeds query
- Embeddings → Embedding provider: requests vectors
- Document retrieval → MongoDB and files: reads chunks and documents
- Research loop → Search cache: checks cache
- Research loop → Run records: records run
- Agent service → Threads: serves thread routes
- Agent service → Spaces and uploads: serves Spaces routes
- Agent service → Memory API: serves memory routes
- Agent service → Usage statistics: serves stats routes
- Spaces and uploads → MongoDB and files: stores uploads and jobs
- Ingestion worker → MongoDB and files: claims jobs and stores chunks
- Ingestion worker → Embeddings: embeds chunks
- Agent service → Ingestion worker: queues ingestion
- Deep search → Research planner: plans sub-questions
- Research planner → LLM streaming: requests plan
- Deep search → Research tools: researches sub-questions
- Agent service → Deep search: dispatches deep search
- Threads → MongoDB and files: stores threads
- Memory API → MongoDB and files: stores memories
- Usage statistics → MongoDB and files: reads statistics
- Memory tools → MongoDB and files: reads and writes memory
- Search interface → Answer and citations: renders answer
- HTTP gateway → Search interface: streams response