How thegreatvoid/learnkg works
This repository turns a text corpus into an interactive, directed knowledge graph. The principal workflow loads and chunks documents, asks a local LLM for directional concept relations, adds same-chunk proximity links, combines the relations into a weighted graph, and renders it for browser exploration. README also describes community coloring and graph analysis. The sampled pipeline supports the main processing and visualization links; some intermediate pipeline code was omitted, so those stage boundaries are based on the README’s documented workflow.
Main entry points: run_pipeline.py Pipeline runner · df_helpers.py Chunk records · run_directed.py Directed graph renderer · prompts.py Relation prompting
Components 17
Corpus processing
- Pipeline runner
run_pipeline.py - Document loading
run_pipeline.py - Chunk records
helpers/df_helpers.py
Relation extraction
- Relation prompting
helpers/prompts.py - Ollama client
ollama/client.py - Relation dataframe
helpers/df_helpers.py
Graph construction
- Proximity edges
run_pipeline.py - Edge consolidation
run_pipeline.py - Directed graph
run_pipeline.py - Precomputed graph
run_directed.py
Analysis and display
- Community analysis
run_pipeline.py - PageRank sizing
run_pipeline.py - Interactive renderer
run_pipeline.py - Directed graph renderer
run_directed.py - Browser graph
run_directed.py
Other components
- Corpus user
- Ollama model service
Connections 18
- Corpus user → Pipeline runner: starts
README.md - Pipeline runner → Document loading: loads corpus
README.md - Document loading → Chunk records: splits text
README.md - Pipeline runner → Chunk records: converts pages
run_pipeline.py - Chunk records → Relation prompting: extracts relations
helpers/df_helpers.py - Relation prompting → Ollama client: requests generation
helpers/prompts.py - Ollama client → Ollama model service: posts prompt
ollama/client.py - Chunk records → Relation dataframe: builds relation rows
helpers/df_helpers.py - Pipeline runner → Relation dataframe: converts extracted graph
run_pipeline.py - Pipeline runner → Proximity edges: calculates co-occurrence
run_pipeline.py - Pipeline runner → Edge consolidation: groups and sums edges
run_pipeline.py - Pipeline runner → Directed graph: builds DiGraph
run_pipeline.py - Pipeline runner → Community analysis: detects communities
run_pipeline.py - Pipeline runner → PageRank sizing: sizes nodes
run_pipeline.py - Pipeline runner → Interactive renderer: renders graph
run_pipeline.py - Corpus user → Directed graph renderer: runs renderer
run_directed.py - Precomputed graph → Directed graph renderer: loads graph data
run_directed.py - Directed graph renderer → Browser graph: writes visualization
run_directed.py