How parmeshg-17/notebooklm-lite works
NotebookLM Lite is a citation-first document research workspace. Users create notebooks, upload documents, and ask questions grounded in the notebook corpus; answers stream with citations that lead to source pages and quoted evidence. Ingestion preserves page context while parsing and chunking, then embeds and indexes content. At query time, hybrid retrieval feeds answer generation and citation validation, with an explicit refusal for unsupported questions. The map also includes document reading and golden-set evaluation capabilities. Some router and UI wiring is inferred from the README because their implementation was not sampled.
Main entry points: pipeline.py Ingestion pipeline · chat.py Chat API · main.py FastAPI application · document_service.py Document service
Components 29
Research workspace
- Research UI
frontend/app/page.tsx - Notebook view
frontend/components/NotebookView.tsx - Evidence inspector
frontend/components/CitationViewerPanel.tsx - Document reader
frontend/components/FullDocumentReader.tsx
API and services
- FastAPI application
backend/app/main.py - Notebook API
backend/app/api/notebooks.py - Document API
backend/app/api/documents.py - Chat API
backend/app/api/chat.py - Evaluation API
backend/app/api/evals.py - Notebook service
backend/app/services/notebook_service.py - Document service
backend/app/services/document_service.py - Object storage
backend/app/services/storage.py
Document ingestion
- Ingestion pipeline
backend/app/ingestion/pipeline.py - Page parser
backend/app/ingestion/parser.py - Page chunker
backend/app/ingestion/chunker.py - Embedding provider
backend/app/rag/embeddings.py - Ingestion task
backend/app/workers/tasks.py
Grounded Q&A
- Hybrid retrieval
backend/app/rag/retrieval.py - Answer generator
backend/app/rag/generator.py - Citation validator
backend/app/rag/citations.py - Citation formatting
backend/app/rag/citations.py
Data and evaluation
- PostgreSQL corpus
backend/app/models/models.py - Document repository
backend/app/db/document_repo.py - Notebook repository
backend/app/db/notebook_repo.py - Evaluation runner
backend/app/evaluations/eval_service.py
Other components
- Researcher
- S3 or local files
- LLM provider
- Redis broker
Connections 38
- Researcher → Research UI: uses
- Research UI → FastAPI application: HTTP and SSE
- FastAPI application → Notebook API: dispatches
- FastAPI application → Document API: dispatches
- FastAPI application → Chat API: dispatches
- FastAPI application → Evaluation API: dispatches
- Notebook API → Notebook service: calls
- Notebook service → Notebook repository: calls
- Notebook repository → PostgreSQL corpus: reads and writes
- Document API → Document service: calls
- Document service → Document repository: calls
- Document repository → PostgreSQL corpus: reads and writes
- Document service → Object storage: stores files
- Object storage → S3 or local files: reads and writes
- Document service → Ingestion pipeline: starts ingestion
- Ingestion task → Ingestion pipeline: runs
- Ingestion task → Redis broker: broker access
- Ingestion pipeline → Object storage: downloads source
- Ingestion pipeline → Page parser: parses pages
- Ingestion pipeline → Page chunker: chunks pages
- Ingestion pipeline → Embedding provider: embeds chunks
- Ingestion pipeline → PostgreSQL corpus: stores indexed chunks
- Chat API → Hybrid retrieval: retrieves evidence
- Hybrid retrieval → Embedding provider: embeds query
- Hybrid retrieval → PostgreSQL corpus: searches corpus
- Chat API → Answer generator: generates answer
- Answer generator → LLM provider: requests completion
- Chat API → Citation validator: validates answer
- Chat API → Citation formatting: formats citations
- Chat API → Research UI: streams answer
- Research UI → Notebook view: shows notebook
- Notebook view → Evidence inspector: opens evidence
- Evidence inspector → Document reader: jumps to page
- Document reader → Document API: fetches page
- Evaluation API → Evaluation runner: runs benchmark
- Evaluation runner → Hybrid retrieval: retrieves test evidence
- Evaluation runner → Answer generator: generates test answer
- Evaluation runner → Citation validator: scores citations