# muhammadsaqlainaslam/my-llm-wiki architecture

> Architecture diagram of the GitHub repository muhammadsaqlainaslam/my-llm-wiki, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muhammadsaqlainaslam/my-llm-wiki
- Repository: https://github.com/muhammadsaqlainaslam/my-llm-wiki

## Overview

#### Purpose and Entry Points
This repository is an AI/ML research wiki: Obsidian-compatible Markdown notes become a static, searchable, citation-aware web knowledge graph. Main operational entry points are `agent.py` (research ingestion/maintenance), `build_wiki.py` (PDF-to-note generation), and `build.py` (site/data generation). The user-facing artifacts are `docs/index.html` and `docs/graph.html`.

#### Canonical Knowledge Store
`wiki/` is the source of truth: each Markdown file is a paper or concept note with YAML metadata, prose, and Obsidian `[[wikilinks]]`. Navigation/index surfaces live alongside content in `wiki/Home.md`, `wiki/000 Index.md`, and `wiki/Citation Map.md`. The graph boundary is explicit: Markdown links encode relationships; frontmatter encodes searchable/citation metadata.

#### Research Ingestion Agent
`agent.py` is the autonomous-but-manually-triggered acquisition layer. Its modes search topical content, monitor arXiv/GitHub/blog sources, discover citing papers, refresh citation counts, and audit citation identities. It uses Claude for relevance evaluation and note generation decisions, while citation refresh and arXiv verification use deterministic external API calls. Run history is persisted in `agent_log.json`; Claude-facing operational guidance is in `skills/wiki-agent/SKILL.md`.

#### PDF-to-Note Generation
`build_wiki.py` transforms source papers into structured notes, using PyMuPDF for extraction and the Anthropic Claude API for summarization and wiki-note authoring. Its output enters `wiki/`, rather than directly modifying the site, preserving a reviewable content boundary. Runtime credentials and optional vault location are configured through `.env.example`.

#### Static Build Pipeline
`build.py` compiles `wiki/*.md` into browser-consumable data and rendered note HTML. It parses YAML, resolves wikilinks, protects/restores LaTex math, converts Markdown/callouts/embeds, and emits metadata including links, citations, arXiv IDs, and rendered body content. Primary generated payload: `docs/notes.json`; presentation artifacts: `docs/index.html` and `docs/graph.html`.

#### Browser Application Surfaces
The site is a dependency-light static client: `docs/index.html` provides browsing, search, filtering, timeline, citation exploration, note rendering, backlinks, comparisons, and local reading state. `docs/graph.html` loads the generated notes data into a D3 v7 force-directed relationship graph. KaTeX renders math client-side; browser `localStorage` stores appearance and reading-progress state. No application server or database is present.

#### Deployment and External Services
GitHub Actions deploys generated `docs/` content to GitHub Pages on pushes, defined in `.github/workflows/deploy.yml`. External integration boundaries are Anthropic/Vertex Claude for generation, arXiv for paper search/identity data, Semantic Scholar for citations, GitHub APIs for repository content, and optional YouTube transcript extraction. `setup_cron.sh` exists for optional scheduling, but the intended control model is manual agent invocation followed by generated-content commits.

#### End-to-End Control Flow
New content flows from PDFs or agent-discovered sources → `build_wiki.py`/`agent.py` → versioned Markdown in `wiki/` → `build.py` → static `docs/` assets → GitHub Actions → GitHub Pages. Citation maintenance follows Semantic Scholar → YAML frontmatter in `wiki/` → rebuild of `docs/notes.json`. Wikilinks and metadata are therefore authored once in the vault and consumed consistently by browse, search, citation, and graph views.

Main entry points: [`agent.py`](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/agent.py) (Research agent), [`docs/index.html`](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/index.html) (Browse application), [`docs/notes.json`](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/notes.json) (Notes payload), [`build_wiki.py`](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/build_wiki.py) (PDF-to-note builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_research["Research acquisition"]
  node_agent["Research agent<br/>Python entry point<br/>[agent.py]"]
  node_agent_skill["Agent guidance<br/>Claude instructions<br/>[SKILL.md]"]
  node_agent_log["Run history<br/>JSON log<br/>[agent_log.json]"]
  node_sources{{"Research sources<br/>External APIs"}}
  node_claude{{"Claude<br/>AI generation service"}}
  node_pdf_builder["PDF-to-note builder<br/>Python entry point<br/>[build_wiki.py]"]
  node_env["Runtime settings<br/>Environment template<br/>[.env.example]"]
end

subgraph group_knowledge["Canonical knowledge"]
  node_vault[("Wiki vault<br/>Markdown knowledge store")]
  node_home["Navigation surfaces<br/>Wiki indexes<br/>[Home.md]"]
end

subgraph group_build["Static build"]
  node_builder["Static builder<br/>Python entry point<br/>[build.py]"]
  node_notes_data["Notes payload<br/>Generated JSON<br/>[notes.json]"]
end

subgraph group_site["Published site"]
  node_browse_app["Browse application<br/>Static browser client<br/>[index.html]"]
  node_graph_app["Relationship graph<br/>D3 browser client<br/>[graph.html]"]
  node_browser_state[("Reading state<br/>Browser localStorage")]
  node_katex{{"KaTeX<br/>Browser math renderer"}}
  node_deploy["Pages deployment<br/>GitHub Actions workflow<br/>[deploy.yml]"]
  node_pages{{"GitHub Pages<br/>Static hosting"}}
end

node_agent_skill -.->|"guides"| node_agent
node_env -.->|"configures"| node_agent
node_env -.->|"configures"| node_pdf_builder
node_sources -->|"papers and citations"| node_agent
node_claude -->|"evaluates and drafts"| node_agent
node_agent -->|"records runs"| node_agent_log
node_agent -->|"creates and updates notes"| node_vault
node_sources -->|"source PDFs"| node_pdf_builder
node_claude -->|"summarizes and authors"| node_pdf_builder
node_pdf_builder -->|"writes structured notes"| node_vault
node_home -->|"navigates notes"| node_vault
node_vault -->|"Markdown, metadata, wikilinks"| node_builder
node_builder -->|"generates"| node_notes_data
node_builder -->|"renders note HTML"| node_browse_app
node_notes_data -->|"loads"| node_browse_app
node_notes_data -->|"loads relationships"| node_graph_app
node_browse_app -->|"persists preferences"| node_browser_state
node_katex -->|"renders math"| node_browse_app
node_katex -->|"renders math"| node_graph_app
node_browse_app -->|"published docs"| node_deploy
node_graph_app -->|"published docs"| node_deploy
node_notes_data -->|"published data"| node_deploy
node_deploy -->|"deploys"| node_pages

click node_agent "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/agent.py"
click node_agent_skill "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/skills/wiki-agent/SKILL.md"
click node_agent_log "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/agent_log.json"
click node_pdf_builder "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/build_wiki.py"
click node_home "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/wiki/Home.md"
click node_builder "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/build.py"
click node_notes_data "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/notes.json"
click node_browse_app "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/index.html"
click node_graph_app "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/graph.html"
click node_deploy "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/.github/workflows/deploy.yml"
click node_env "https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/.env.example"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent,node_agent_skill,node_agent_log,node_sources,node_claude,node_pdf_builder,node_env toneBlue
class node_vault,node_home toneAmber
class node_builder,node_notes_data toneMint
class node_browse_app,node_graph_app,node_browser_state,node_katex,node_deploy,node_pages toneRose
```

## Components

### Research acquisition

- [Research agent](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/agent.py): Python entry point · Manually triggered discovery and citation maintenance. · `agent.py`
- [Agent guidance](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/skills/wiki-agent/SKILL.md): Claude instructions · `skills/wiki-agent/SKILL.md`
- [Run history](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/agent_log.json): JSON log · `agent_log.json`
- **Research sources**: External APIs · arXiv, Semantic Scholar, GitHub, blogs, and optional YouTube.
- **Claude**: AI generation service · Anthropic or Vertex Claude evaluates and authors notes.
- [PDF-to-note builder](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/build_wiki.py): Python entry point · Extracts papers and generates reviewable wiki notes. · `build_wiki.py`
- [Runtime settings](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/.env.example): Environment template · Credentials and optional vault location. · `.env.example`

### Canonical knowledge

- **Wiki vault**: Markdown knowledge store · Versioned notes with YAML metadata and Obsidian wikilinks are the source of truth.
- [Navigation surfaces](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/wiki/Home.md): Wiki indexes · Home, index, and citation map organize vault navigation. · `wiki/Home.md`

### Static build

- [Static builder](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/build.py): Python entry point · Parses frontmatter, resolves links, and renders Markdown into static data. · `build.py`
- [Notes payload](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/notes.json): Generated JSON · Contains rendered bodies, metadata, citations, and resolved links. · `docs/notes.json`

### Published site

- [Browse application](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/index.html): Static browser client · Search, filtering, citations, backlinks, and reading state. · `docs/index.html`
- [Relationship graph](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/docs/graph.html): D3 browser client · Force-directed graph over generated note links. · `docs/graph.html`
- **Reading state**: Browser localStorage · Stores appearance and reading progress locally.
- **KaTeX**: Browser math renderer
- [Pages deployment](https://github.com/muhammadsaqlainaslam/my-llm-wiki/blob/main/.github/workflows/deploy.yml): GitHub Actions workflow · Deploys generated docs on pushes. · `.github/workflows/deploy.yml`
- **GitHub Pages**: Static hosting

## Connections

- Agent guidance → Research agent: guides
- Runtime settings → Research agent: configures
- Runtime settings → PDF-to-note builder: configures
- Research sources → Research agent: papers and citations
- Claude → Research agent: evaluates and drafts
- Research agent → Run history: records runs
- Research agent → Wiki vault: creates and updates notes
- Research sources → PDF-to-note builder: source PDFs
- Claude → PDF-to-note builder: summarizes and authors
- PDF-to-note builder → Wiki vault: writes structured notes
- Navigation surfaces → Wiki vault: navigates notes
- Wiki vault → Static builder: Markdown, metadata, wikilinks
- Static builder → Notes payload: generates
- Static builder → Browse application: renders note HTML
- Notes payload → Browse application: loads
- Notes payload → Relationship graph: loads relationships
- Browse application → Reading state: persists preferences
- KaTeX → Browse application: renders math
- KaTeX → Relationship graph: renders math
- Browse application → Pages deployment: published docs
- Relationship graph → Pages deployment: published docs
- Notes payload → Pages deployment: published data
- Pages deployment → GitHub Pages: deploys