# niethesan96/graph_traversal architecture

> Architecture diagram of the GitHub repository niethesan96/graph_traversal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/niethesan96/graph_traversal
- Repository: https://github.com/niethesan96/graph_traversal

## Overview

This repository is a staged Wikispeedia navigation-policy training and evaluation pipeline. Its workflow loads the directed article graph and metadata, embeds article leads and samples shortest-path-labeled pairs, extracts hop-level features, trains a ranking policy, then performs greedy traversal and compares it with search and heuristic baselines. The README documents additional evaluation and diagnostic capabilities; the sampled excerpts confirm parts of the implementation, but not every phase’s internal wiring.

Main entry points: [`phase1.py`](https://github.com/niethesan96/graph_traversal/blob/main/phase1.py) (Graph loading), [`phase6.py`](https://github.com/niethesan96/graph_traversal/blob/main/phase6.py) (Baseline evaluation), [`phase5.py`](https://github.com/niethesan96/graph_traversal/blob/main/phase5.py) (Greedy traversal), [`phase6_diagnostic.py`](https://github.com/niethesan96/graph_traversal/blob/main/phase6_diagnostic.py) (Hub-drift diagnostics)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Wikispeedia data"]
  node_links["Directed links<br/>[links.tsv]"]
  node_articles["Article list<br/>[articles.tsv]"]
  node_categories["Article categories<br/>[categories.tsv]"]
  node_plaintext["Article text"]
end

subgraph group_prep["Graph and supervision"]
  node_phase1["Graph loading<br/>[phase1.py]"]
  node_graphckpt["Graph checkpoint"]
  node_phase2["Embedding and oracle<br/>[phase2.py]"]
  node_trainckpt["Labeled hop data"]
end

subgraph group_learning["Policy learning"]
  node_phase3["Feature extraction<br/>[phase3.py]"]
  node_featureckpt["Feature matrices"]
  node_phase4["Policy training<br/>[phase4.py]"]
  node_modelckpt["Policy and scaler"]
end

subgraph group_runtime["Navigation and evaluation"]
  node_phase5["Greedy traversal<br/>[phase5.py]"]
  node_inferckpt["Traversal results"]
  node_phase6["Baseline evaluation<br/>[phase6.py]"]
  node_evalckpt["Comparison results"]
  node_diagnostic["Hub-drift diagnostics"]
end

node_engineer(("Pipeline user"))
node_embedmodel["Sentence transformer"]

node_engineer -->|"runs phases"| node_phase1
node_links -->|"loads edges"| node_phase1
node_articles -->|"loads articles"| node_phase1
node_categories -->|"loads categories"| node_phase1
node_plaintext -->|"loads text"| node_phase1
node_phase1 -->|"saves graph data"| node_graphckpt
node_graphckpt -->|"loads graph and text"| node_phase2
node_embedmodel -.->|"embeds article leads"| node_phase2
node_phase2 -->|"saves pairs and embeddings"| node_trainckpt
node_graphckpt -->|"reads graph and categories"| node_phase3
node_trainckpt -->|"reads hop examples"| node_phase3
node_phase3 -->|"saves feature sets"| node_featureckpt
node_featureckpt -->|"provides matrices"| node_phase4
node_trainckpt -->|"provides labels"| node_phase4
node_phase4 -->|"saves policy"| node_modelckpt
node_graphckpt -->|"provides graph and categories"| node_phase5
node_trainckpt -->|"provides embeddings and test pairs"| node_phase5
node_modelckpt -->|"provides policy"| node_phase5
node_phase5 -->|"saves traversals"| node_inferckpt
node_inferckpt -->|"provides learned results"| node_phase6
node_graphckpt -->|"provides graph"| node_phase6
node_trainckpt -->|"provides pairs and embeddings"| node_phase6
node_modelckpt -->|"provides learned policy"| node_phase6
node_phase6 -->|"saves comparisons"| node_evalckpt
node_featureckpt -->|"provides features"| node_diagnostic
node_trainckpt -->|"provides examples and pairs"| node_diagnostic
node_graphckpt -->|"provides graph"| node_diagnostic
node_diagnostic -->|"saves reduced policy"| node_modelckpt

click node_links "https://github.com/niethesan96/graph_traversal/blob/main/links.tsv"
click node_articles "https://github.com/niethesan96/graph_traversal/blob/main/articles.tsv"
click node_categories "https://github.com/niethesan96/graph_traversal/blob/main/categories.tsv"
click node_plaintext "https://github.com/niethesan96/graph_traversal/tree/main/plaintext_articles"
click node_phase1 "https://github.com/niethesan96/graph_traversal/blob/main/phase1.py"
click node_phase2 "https://github.com/niethesan96/graph_traversal/blob/main/phase2.py"
click node_phase3 "https://github.com/niethesan96/graph_traversal/blob/main/phase3.py"
click node_phase4 "https://github.com/niethesan96/graph_traversal/blob/main/phase4.py"
click node_phase5 "https://github.com/niethesan96/graph_traversal/blob/main/phase5.py"
click node_phase6 "https://github.com/niethesan96/graph_traversal/blob/main/phase6.py"
click node_diagnostic "https://github.com/niethesan96/graph_traversal/blob/main/phase6_diagnostic.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_links,node_articles,node_categories,node_plaintext,node_engineer toneBlue
class node_phase1,node_graphckpt,node_phase2,node_trainckpt toneAmber
class node_phase3,node_featureckpt,node_phase4,node_modelckpt toneMint
class node_phase5,node_inferckpt,node_phase6,node_evalckpt,node_diagnostic toneRose
class node_embedmodel toneIndigo
```

## Components

### Wikispeedia data

- [Directed links](https://github.com/niethesan96/graph_traversal/blob/main/links.tsv): `links.tsv`
- [Article list](https://github.com/niethesan96/graph_traversal/blob/main/articles.tsv): `articles.tsv`
- [Article categories](https://github.com/niethesan96/graph_traversal/blob/main/categories.tsv): `categories.tsv`
- [Article text](https://github.com/niethesan96/graph_traversal/tree/main/plaintext_articles): `plaintext_articles`

### Graph and supervision

- [Graph loading](https://github.com/niethesan96/graph_traversal/blob/main/phase1.py): `phase1.py`
- **Graph checkpoint**
- [Embedding and oracle](https://github.com/niethesan96/graph_traversal/blob/main/phase2.py): `phase2.py`
- **Labeled hop data**

### Policy learning

- [Feature extraction](https://github.com/niethesan96/graph_traversal/blob/main/phase3.py): `phase3.py`
- **Feature matrices**
- [Policy training](https://github.com/niethesan96/graph_traversal/blob/main/phase4.py): `phase4.py`
- **Policy and scaler**

### Navigation and evaluation

- [Greedy traversal](https://github.com/niethesan96/graph_traversal/blob/main/phase5.py): `phase5.py`
- **Traversal results**
- [Baseline evaluation](https://github.com/niethesan96/graph_traversal/blob/main/phase6.py): `phase6.py`
- **Comparison results**
- [Hub-drift diagnostics](https://github.com/niethesan96/graph_traversal/blob/main/phase6_diagnostic.py): `phase6_diagnostic.py`

### Other components

- **Pipeline user**
- **Sentence transformer**

## Connections

- Pipeline user → Graph loading: runs phases
- Directed links → Graph loading: loads edges
- Article list → Graph loading: loads articles
- Article categories → Graph loading: loads categories
- Article text → Graph loading: loads text
- Graph loading → Graph checkpoint: saves graph data
- Graph checkpoint → Embedding and oracle: loads graph and text
- Sentence transformer → Embedding and oracle: embeds article leads
- Embedding and oracle → Labeled hop data: saves pairs and embeddings
- Graph checkpoint → Feature extraction: reads graph and categories
- Labeled hop data → Feature extraction: reads hop examples
- Feature extraction → Feature matrices: saves feature sets
- Feature matrices → Policy training: provides matrices
- Labeled hop data → Policy training: provides labels
- Policy training → Policy and scaler: saves policy
- Graph checkpoint → Greedy traversal: provides graph and categories
- Labeled hop data → Greedy traversal: provides embeddings and test pairs
- Policy and scaler → Greedy traversal: provides policy
- Greedy traversal → Traversal results: saves traversals
- Traversal results → Baseline evaluation: provides learned results
- Graph checkpoint → Baseline evaluation: provides graph
- Labeled hop data → Baseline evaluation: provides pairs and embeddings
- Policy and scaler → Baseline evaluation: provides learned policy
- Baseline evaluation → Comparison results: saves comparisons
- Feature matrices → Hub-drift diagnostics: provides features
- Labeled hop data → Hub-drift diagnostics: provides examples and pairs
- Graph checkpoint → Hub-drift diagnostics: provides graph
- Hub-drift diagnostics → Policy and scaler: saves reduced policy