How nilsjennissen/langgraph works
1. Project shape
- This repository is a local RAG chatbot application centered on two runtime entrypoints: app.py and app_streamlit.py.
- The rest of the repo is supporting material: a PDF document corpus in docs/, exploratory work in notebooks/, and tests in tests/.
2. Primary purpose
- The system answers user questions by combining retrieval over local PDFs with LLM generation.
- It is not a generic app scaffold; the architecture is specifically optimized for adaptive question answering with routing, retrieval, rewriting, and answer grading.
3. Main orchestration layer
- The core logic is described as a LangGraph state machine.
- The graph holds a small shared state: the user question, retrieved documents, the generated answer, and retry count.
- This graph is the central control plane that decides whether to answer directly or invoke retrieval, and whether to rewrite and retry.
4. Question routing boundary
- The first architectural decision is a router that classifies each query into one of two paths:
- direct LLM response for general questions,
- retrieval-augmented path for questions that should be answered from the document corpus.
- This boundary is important because it separates “knowledge in the model” from “knowledge in the PDFs.”
5. Retrieval subsystem
- The retrieval path is multi-stage:
- document retrieval,
- document relevance grading,
- query rewriting if retrieval quality is poor.
- The README describes an ensemble retriever combining keyword search (BM25) and semantic search via a vectorstore-backed approach.
- The document source is the docs/ directory, which currently contains many domain PDFs and acts as the local knowledge base.
6. Generation and self-evaluation loop
- Once relevant documents are selected, the graph generates an answer using retrieved context.
- The system then grades the generation for quality/grounding.
- If the output is weak, the workflow can loop back through query transformation and retrieval again, making the system iterative rather than single-pass.
7. LLM provider abstraction
- The chatbot is intentionally provider-agnostic and can be wired to OpenAI, Ollama, Groq, or Hugging Face integrations through LangChain.
- This means the model layer is an interchangeable dependency rather than a fixed implementation.
- Architecturally, the LLM backend is a pluggable service boundary beneath the graph nodes.
8. Presentation layer
- app.py appears to be the main script for running the chatbot logic from Python.
- app_streamlit.py is a separate UI entrypoint for an interactive Streamlit front end.
- These are thin presentation surfaces over the same underlying retrieval/generation workflow.
9. Data flow from user to answer
- User question enters the app entrypoint.
- The graph routes the question.
- If retrieval is needed, the system loads/uses the PDF corpus, searches it, filters candidate passages, and may rewrite the query.
- The selected context is passed to the LLM, the answer is graded, and the final response is returned to the UI.
10. Corpus and local assets
- The repository stores its knowledge source locally as PDFs under docs/.
- This is the main durable data boundary in the repo: there is no separate database or remote knowledge store shown.
- The document set is domain-specific and determines the chatbot’s factual coverage.
11. Development and experimentation
- notebooks/pdf-chat.ipynb is for step-by-step experimentation and likely mirrors or probes the graph pipeline interactively.
- This notebook layer is outside the production path and is best viewed as a development sandbox for retrieval and prompting behavior.
12. Testing and quality boundaries
- tests/tests.py provides a minimal automated validation layer.
- Given the architecture, tests likely focus on graph behavior, document loading, routing decisions, or utility functions rather than end-to-end model determinism.
13. What is not present
- There is no evidence of a separate server, database, queue, or microservice decomposition.
- The architecture is therefore a single-process application with local document storage and outbound API calls to LLM providers.
14. Diagram guidance
- A good architecture diagram should show:
- user/UI on the left,
- an orchestration graph in the center,
- local PDF corpus as the retrieval source,
- optional LLM provider backends on the right,
- and a feedback loop from answer grading back into query rewriting/retrieval.
Main entry points: docs PDF corpus · app.py App · app_streamlit.py Streamlit UI · pdf-chat.ipynb Notebook
Components 18
Presentation
User-facing entrypoints
- App · Python chatbot entrypoint
app.py - Streamlit UI · Interactive frontend for the same workflow
app_streamlit.py
Orchestration
LangGraph control plane and answer loop
- State graph · Central control plane that routes and loops over question answering steps
- Shared state · Question, retrieved docs, generated answer, retry count
- Router · Chooses direct answer vs retrieval-augmented path
- Direct answer · Answers general questions without corpus lookup
- RAG path · Retrieval-grounded answer branch
- Rewrite query · Refines weak queries before retrying retrieval
- Grade docs · Checks whether retrieved passages are useful
- Generate answer · Forms grounded response from selected context
- Grade answer · Self-evaluates grounding and response quality
Retrieval
Local PDF knowledge access and ranking
- Ensemble retriever · Combines lexical BM25 and semantic vector search
- BM25 · Sparse lexical retrieval over the PDF corpus
- Vector store · Embedding-backed semantic retrieval index
Model layer
LLM and embedding dependencies
- LLM provider · OpenAI, Ollama, Groq, or Hugging Face via LangChain
Assets
Local corpus and exploratory material
- PDF corpus · Domain PDFs used as the factual source
docs - Notebook · Interactive exploration of retrieval and prompting
notebooks/pdf-chat.ipynb - Tests · Minimal automated checks for core behavior
tests/tests.py
Connections 21
- App → State graph: starts · Runs the chatbot workflow from Python
- Streamlit UI → State graph: starts · Drives the same pipeline from the UI
- State graph → Shared state: reads/writes · Maintains question, docs, answer, and retries
- State graph → Router: routes · First decision point for each query
- Router → Direct answer: general query · Bypasses retrieval for non-corpus questions
- Router → RAG path: needs corpus · Sends knowledge-seeking questions to retrieval
- RAG path → Ensemble retriever: fetch context · Gets candidate passages from local PDFs
- Ensemble retriever → BM25: lexical · Keyword-based search branch
- Ensemble retriever → Vector store: semantic · Embedding-based search branch
- BM25 → PDF corpus: indexes · Searches the local document set
- Vector store → PDF corpus: built from · Derived from the same PDF corpus
- Ensemble retriever → Grade docs: candidate check · Filters weak retrieval results
- Grade docs → Rewrite query: if poor · Rewrites the question when retrieval quality is low
- Rewrite query → Ensemble retriever: retry search · Loops back into retrieval with improved query
- Grade docs → Generate answer: if useful · Passes relevant context to generation
- Generate answer → LLM provider: calls · Uses a pluggable model backend
- Generate answer → Grade answer: self-check · Evaluates grounding and response quality
- Grade answer → Rewrite query: if weak · Can trigger another retrieval iteration
- Grade answer → State graph: finalize · Returns the finished answer when sufficient
- Notebook → Ensemble retriever: experiments with · Used to probe retrieval and prompting behavior
- Tests → State graph: validates · Checks core workflow behavior