How niethesan96/graph_traversal works
This repository is a staged Wikispeedia navigation-policy training and evaluation pipeline. Its workflow loads the directed article graph and metadata, embeds article leads and samples shortest-path-labeled pairs, extracts hop-level features, trains a ranking policy, then performs greedy traversal and compares it with search and heuristic baselines. The README documents additional evaluation and diagnostic capabilities; the sampled excerpts confirm parts of the implementation, but not every phase’s internal wiring.
Main entry points: phase1.py Graph loading · phase6.py Baseline evaluation · phase5.py Greedy traversal · phase6_diagnostic.py Hub-drift diagnostics
Components 19
Wikispeedia data
- Directed links
links.tsv - Article list
articles.tsv - Article categories
categories.tsv - Article text
plaintext_articles
Graph and supervision
- Graph loading
phase1.py - Graph checkpoint
- Embedding and oracle
phase2.py - Labeled hop data
Policy learning
- Feature extraction
phase3.py - Feature matrices
- Policy training
phase4.py - Policy and scaler
Navigation and evaluation
- Greedy traversal
phase5.py - Traversal results
- Baseline evaluation
phase6.py - Comparison results
- Hub-drift diagnostics
phase6_diagnostic.py
Other components
- Pipeline user
- Sentence transformer
Connections 28
- Pipeline user → Graph loading: runs phases
- Directed links → Graph loading: loads edges
- Article list → Graph loading: loads articles
- Article categories → Graph loading: loads categories
- Article text → Graph loading: loads text
- Graph loading → Graph checkpoint: saves graph data
- Graph checkpoint → Embedding and oracle: loads graph and text
- Sentence transformer → Embedding and oracle: embeds article leads
- Embedding and oracle → Labeled hop data: saves pairs and embeddings
- Graph checkpoint → Feature extraction: reads graph and categories
- Labeled hop data → Feature extraction: reads hop examples
- Feature extraction → Feature matrices: saves feature sets
- Feature matrices → Policy training: provides matrices
- Labeled hop data → Policy training: provides labels
- Policy training → Policy and scaler: saves policy
- Graph checkpoint → Greedy traversal: provides graph and categories
- Labeled hop data → Greedy traversal: provides embeddings and test pairs
- Policy and scaler → Greedy traversal: provides policy
- Greedy traversal → Traversal results: saves traversals
- Traversal results → Baseline evaluation: provides learned results
- Graph checkpoint → Baseline evaluation: provides graph
- Labeled hop data → Baseline evaluation: provides pairs and embeddings
- Policy and scaler → Baseline evaluation: provides learned policy
- Baseline evaluation → Comparison results: saves comparisons
- Feature matrices → Hub-drift diagnostics: provides features
- Labeled hop data → Hub-drift diagnostics: provides examples and pairs
- Graph checkpoint → Hub-drift diagnostics: provides graph
- Hub-drift diagnostics → Policy and scaler: saves reduced policy