How tomasonjo/kg-rag works
1. What this repo is
- This is a notebook-first educational codebase for building GraphRAG systems: a retrieval pipeline that combines knowledge graphs, vector search, and LLM prompting.
- The repository is organized as a sequence of chapter notebooks rather than a single deployable application, so the architecture is best understood as a set of progressively layered RAG examples.
2. Top-level structure
- notebooks/ contains the main executable artifacts: ch02.ipynb through ch08.ipynb.
- The Python files alongside the notebooks are shared helper modules used across chapters: utils.py, schema_utils.py, text2cypher.py, and chapter-specific helpers like ch05_tools.py, ch07_tools.py, and ch08_tools.py.
- data/ contains supporting assets, including a benchmark dataset and a license agreement.
3. Primary execution model
- The notebooks are the main entrypoints for experimentation, walkthroughs, and reproducing examples.
- Shared Python modules hold reusable logic so the notebooks can focus on narrative, orchestration, and demonstration rather than duplicating plumbing code.
- This separation is important architecturally: notebooks drive the flow, while modules encapsulate repeatable operations such as schema handling, query generation, and utility functions.
4. Core GraphRAG pipeline
- The repository centers on a typical GraphRAG flow: ingest text or source material, extract structured knowledge, build or query a graph, and use retrieved context to answer questions with an LLM.
- Unlike pure vector RAG, graph-based retrieval introduces explicit entities and relationships, which improves traceability and supports multi-hop reasoning.
- The codebase appears to demonstrate both graph-centric and vector-centric retrieval, then combines them into hybrid approaches.
5. Knowledge graph and Cypher boundary
- text2cypher.py strongly indicates a boundary between natural language and graph queries: converting user questions into Cypher-like statements for graph retrieval.
- schema_utils.py suggests the graph schema is treated as a first-class contract, likely used to constrain or validate generated graph queries.
- This implies a Cypher-compatible property graph backend as a key external dependency, even though the exact database is not named in the repo.
6. Retrieval and tooling modules
- ch05_tools.py likely supports vector similarity or hybrid retrieval examples introduced in the middle chapters.
- ch07_tools.py and ch08_tools.py suggest later-stage tooling for agentic RAG and semantic-layer / query-generation workflows.
- These modules are where the reusable retrieval primitives live, while the notebooks show how those primitives are composed into end-to-end examples.
7. Evaluation and benchmarking
- data/benchmark_data.csv indicates there is a benchmarking or evaluation loop for RAG performance.
- This is a distinct subsystem from the demo notebooks: it focuses on measuring retrieval/answer quality rather than just illustrating the mechanics.
- The presence of benchmark data suggests the repo treats performance and quality as first-class concerns, not just proof-of-concept functionality.
8. Data and governance
- data/license_agreement.txt implies at least one dataset or benchmark source has usage restrictions that matter operationally.
- That makes the data layer a governed boundary: some assets are not just sample inputs, but licensed materials that must be respected when reproducing experiments.
- For architecture diagrams, treat data/ as an input domain with both content and compliance concerns.
9. Environment and dependencies
- notebooks/requirements.txt defines the Python runtime dependencies for the notebook environment.
- .env.example shows the project expects configuration via environment variables, likely for API keys and backend connection settings.
- This means external services are configured outside the notebooks, reinforcing that the examples depend on runtime credentials and infrastructure rather than being self-contained.
10. Likely external systems
- The repo’s design implies at least two external service types: an LLM provider for generation/extraction, and a graph database or graph query engine for Cypher-based retrieval.
- Vector retrieval examples also suggest a vector search component, though the specific engine is not named in the tree.
- These services are used as backends; the notebooks and helper modules act as the orchestration layer.
11. How the chapters fit together
- Early notebooks (ch02–ch04) likely establish concepts and baseline RAG behavior.
- Mid-course notebooks (ch05–ch06) appear to cover vector similarity and evaluation.
- Later notebooks (ch07–ch08) move toward agentic workflows, semantic layers, and structured query generation over the graph.
12. Architectural takeaway
- This repository is not an application server; it is a reference implementation and experimentation environment.
- Its architecture is best drawn as: chapter notebooks at the top, shared utility modules in the middle, and external LLM/vector/graph services plus benchmark data at the bottom.
- The key design boundary is between unstructured language input and structured graph/schema-constrained retrieval.
Main entry points: utils.py Utils · ch08.ipynb Ch 08 · ch07.ipynb Ch 07 · .env.example Env example
Components 20
Chapter notebooks
Main executable walkthroughs and experiments across the chapter sequence.
- Ch 02 · Early chapter notebook introducing the GraphRAG workflow.
notebooks/ch02.ipynb - Ch 03 · Chapter notebook for the next step in the learning sequence.
notebooks/ch03.ipynb - Ch 04 · Chapter notebook expanding the baseline retrieval pattern.
notebooks/ch04.ipynb - Ch 05 · Chapter notebook covering mid-course retrieval examples.
notebooks/ch05.ipynb - Ch 06 · Chapter notebook focused on evaluation or comparative experiments.
notebooks/ch06.ipynb - Ch 07 · Later chapter notebook for agentic or structured query workflows.
notebooks/ch07.ipynb - Ch 08 · Final chapter notebook for advanced GraphRAG composition.
notebooks/ch08.ipynb
Shared modules
Reusable notebook helpers for retrieval, schema handling, and query generation.
- Utils · General reusable notebook utilities.
notebooks/utils.py - Schema utils · Schema handling and validation helpers for graph-focused workflows.
notebooks/schema_utils.py - Text to Cypher · Converts natural language into graph query form.
notebooks/text2cypher.py - Ch 05 tools · Helpers supporting the chapter 5 retrieval examples.
notebooks/ch05_tools.py - Ch 07 tools · Helpers for later chapter agentic or semantic-layer workflows.
notebooks/ch07_tools.py - Ch 08 tools · Helpers for the final chapter's advanced GraphRAG examples.
notebooks/ch08_tools.py - Requirements · Notebook environment dependencies.
notebooks/requirements.txt - Env example · Example environment variable configuration for runtime credentials and backends.
.env.example
Data assets
Benchmark and governance inputs used by the examples and evaluation.
- Benchmark data · Evaluation dataset for measuring retrieval and answer quality.
data/benchmark_data.csv - License note · Usage constraints for governed data assets.
data/license_agreement.txt
External services
Runtime backends implied by the notebooks and helper modules.
- LLM provider · Generation and extraction backend used by the notebooks.
- Graph DB · Cypher-compatible property graph backend for structured retrieval.
- Vector search · Embedding-based retrieval backend used in hybrid examples.
Connections 24
- Ch 02 → Utils: uses · Notebook orchestration relies on shared utility functions.
- Ch 03 → Utils: uses · Notebook orchestration relies on shared utility functions.
- Ch 04 → Utils: uses · Notebook orchestration relies on shared utility functions.
- Ch 05 → Ch 05 tools: uses · Mid-course retrieval notebook delegates reusable logic to chapter tools.
- Ch 05 → Utils: uses · Notebook orchestration relies on shared utility functions.
- Ch 06 → Benchmark data: evaluates · Evaluation notebook measures performance against benchmark data.
- Ch 06 → Utils: uses · Notebook orchestration relies on shared utility functions.
- Ch 07 → Ch 07 tools: uses · Later notebook composes structured and agentic helpers.
- Ch 07 → Text to Cypher: calls · Natural language questions are converted into graph queries.
- Ch 07 → Schema utils: constrains · Schema helpers bound query generation to graph structure.
- Ch 08 → Ch 08 tools: uses · Final chapter notebook delegates advanced workflow helpers.
- Ch 08 → Text to Cypher: calls · Natural language questions are converted into graph queries.
- Ch 08 → Schema utils: constrains · Schema helpers bound query generation to graph structure.
- Text to Cypher → Graph DB: targets · Generated Cypher is executed against the graph backend.
- Ch 05 tools → Vector search: retrieves · Retrieval helpers connect to vector-based search.
- Ch 07 tools → LLM provider: orchestrates · Agentic helpers depend on generation and reasoning from the LLM backend.
- Ch 08 tools → LLM provider: orchestrates · Advanced workflow helpers depend on generation and reasoning from the LLM backend.
- Env example → LLM provider: configures · Environment variables hold credentials and runtime settings.
- Env example → Graph DB: configures · Environment variables hold credentials and runtime settings.
- Env example → Vector search: configures · Environment variables hold credentials and runtime settings.
- Requirements → Ch 02: supports · Notebook dependencies support the chapter runtime.
- Requirements → Ch 08: supports · Notebook dependencies support the chapter runtime.
- License note → Benchmark data: governs · Governance note applies to the benchmark asset.
- Benchmark data → Ch 06: feeds · Benchmark data is consumed by evaluation-oriented notebooks.