# he-yufeng/repowiki architecture

> Architecture diagram of the GitHub repository he-yufeng/repowiki, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/he-yufeng/repowiki
- Repository: https://github.com/he-yufeng/repowiki

## Overview

1. **Repo shape: a Python backend plus a separate browser frontend.**  
   The core product lives under `src/repowiki/` and can run as a CLI, a FastAPI server, or a terminal chat tool. The `frontend/` directory is a React + Vite TypeScript UI that talks to the backend API. This is not a single monolith; it is a pipeline-oriented backend with a thin web presentation layer.

2. **Primary entry points.**  
   `repowiki/__main__.py` and `cli.py` provide command-line commands such as `scan`, `serve`, `chat`, and `config`. The same backend capabilities are exposed through `server/app.py` and routers under `server/routers/`. The web app is a consumer of those backend endpoints rather than an independent analysis engine.

3. **Repository ingestion boundary.**  
   Input comes from either a local filesystem path (`ingest/local.py`) or a GitHub URL (`ingest/github.py`). That boundary matters because everything downstream operates on a normalized project snapshot, regardless of source. The README indicates scans respect `.gitignore`, `.repowikiignore`, and common secret-file exclusions.

4. **Scanning and file selection.**  
   `core/scanner.py` is the first major processing stage. It walks the tree, detects languages, identifies candidate files, and filters out binaries, generated bundles, and oversized or irrelevant content. This stage is the main “context control” gate before any graph or LLM work happens.

5. **Dependency graph and file ranking.**  
   `core/graph.py` builds a cross-file dependency graph and applies PageRank-style ranking to identify important files and a guided reading path. The README explicitly notes import-aware resolution for Python package-relative imports and JavaScript/TypeScript relative modules, so the graph is a core architectural subsystem rather than a cosmetic feature.

6. **LLM analysis pipeline.**  
   `core/analyzer.py` orchestrates multi-pass structured analysis of the scanned repo. It feeds selected file content and tree context into the LLM layer, likely producing project overview, module summaries, architecture notes, and reading guidance. Prompt templates live in `llm/prompts.py`, and the actual model access is abstracted by `llm/client.py` around `litellm`.

7. **Wiki assembly and document model.**  
   `core/wiki_builder.py` turns analysis artifacts into a coherent wiki structure. Supporting types in `core/models.py` define the data shapes passed across scan, graph, analysis, and export layers. This is the internal “content product” boundary: raw repo data becomes structured documentation objects.

8. **Caching and repeat scans.**  
   `core/cache.py` provides SQLite-backed caching keyed by content hash, allowing re-scans to skip unchanged files. The README positions SQLite as the only persistence dependency, which keeps the architecture local-first and avoids external databases.

9. **Retrieval and Q&A path.**  
   `core/rag.py` implements terminal Q&A retrieval using TF-IDF rather than embeddings. That means the chat path is separate from the documentation-generation path: it indexes repo text, retrieves relevant snippets, and then asks the LLM to answer grounded in those snippets. This is the architecture behind `repowiki chat`.

10. **Export layer with three output modes.**  
   `export/markdown.py`, `export/json_export.py`, and `export/html.py` render the wiki into different delivery formats. Markdown is a directory of files, JSON is structured machine-readable output, and HTML is a self-contained artifact with Mermaid diagrams included. Export is a distinct boundary after analysis, not part of scanning.

11. **Web backend and frontend responsibilities.**  
   `server/app.py` and routers `scan.py`, `wiki.py`, and `chat.py` expose the analysis pipeline over HTTP. The frontend pages in `frontend/src/pages/` (`Home`, `WikiView`, `ChatView`) and components like `WikiSidebar`, `WikiContent`, and `MermaidDiagram` provide the browsing experience. The frontend is therefore a presentation and navigation layer over wiki data, not the source of truth.

12. **Configuration and external dependencies.**  
   `config.py` centralizes runtime settings, including model selection and API keys. The README shows config precedence from CLI flags to environment variables to a user config file, with provider-specific API keys for LLM access. The key external integration is the LLM provider layer via `litellm`; everything else is local filesystem, GitHub fetch/clone, SQLite, and browser UI.

13. **Quality gates and delivery pipeline.**  
   Tests cover scanner, graph, RAG, and export behavior (`tests/test_scanner.py`, `test_graph.py`, `test_rag.py`, `test_markdown_export.py`, `test_html_export.py`). GitHub Actions workflows in `.github/workflows/ci.yml` and `publish.yml` indicate the project is packaged and released as a distributable Python tool with CI validation.

14. **Useful diagram boundaries to draw.**  
   A good architecture diagram would show: input sources → ingestion → scanning/filtering → dependency graph + ranking → LLM analysis → wiki assembly → export/API responses; with a side path for retrieval/chat and a separate presentation boundary for the React frontend. The biggest internal seams are ingestion vs analysis, analysis vs export, and offline CLI processing vs online server/UI consumption.

Main entry points: [`src/repowiki/core/wiki_builder.py`](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/wiki_builder.py) (Wiki builder), [`src/repowiki/core/analyzer.py`](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/analyzer.py) (Analyzer), [`frontend/src/App.tsx`](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/App.tsx) (UI app), [`frontend/src/lib/api.ts`](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/lib/api.ts) (API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_backend["Python backend"]
  node_n_cli["CLI entry<br/>command entrypoint<br/>[__main__.py]"]
  node_n_ingest_local["Local ingest<br/>source adapter<br/>[local.py]"]
  node_n_ingest_github["GitHub ingest<br/>source adapter<br/>[github.py]"]
  node_n_scanner["Scanner<br/>file selection<br/>[scanner.py]"]
  node_n_graph{{"Dep graph<br/>ranking graph<br/>[graph.py]"}}
  node_n_cache[("Cache<br/>[cache.py]")]
  node_n_analyzer["Analyzer<br/>llm pipeline<br/>[analyzer.py]"]
  node_n_prompts["Prompts<br/>prompt templates<br/>[prompts.py]"]
  node_n_llm["LLM client<br/>model adapter<br/>[client.py]"]
  node_n_models["Models<br/>data contracts<br/>[models.py]"]
  node_n_builder["Wiki builder<br/>assembly stage<br/>[wiki_builder.py]"]
  node_n_rag["RAG chat<br/>retrieval qa<br/>[rag.py]"]
  node_n_md_export["Markdown export<br/>export format<br/>[markdown.py]"]
  node_n_json_export["JSON export<br/>export format<br/>[json_export.py]"]
  node_n_html_export["HTML export<br/>export format<br/>[html.py]"]
  node_n_server["API server<br/>fastapi app<br/>[app.py]"]
  node_n_scan_router["Scan API<br/>api router<br/>[scan.py]"]
  node_n_wiki_router["Wiki API<br/>api router<br/>[wiki.py]"]
  node_n_chat_router["Chat API<br/>api router<br/>[chat.py]"]
end

subgraph group_g_frontend["Browser frontend"]
  node_n_frontend_app["UI app<br/>react app<br/>[App.tsx]"]
  node_n_frontend_api["API client<br/>frontend client<br/>[api.ts]"]
  node_n_home["Home<br/>page<br/>[Home.tsx]"]
  node_n_wiki_view["Wiki view<br/>page<br/>[WikiView.tsx]"]
  node_n_chat_view["Chat view<br/>page<br/>[ChatView.tsx]"]
  node_n_sidebar["Sidebar<br/>ui component<br/>[WikiSidebar.tsx]"]
  node_n_content["Content pane<br/>ui component<br/>[WikiContent.tsx]"]
  node_n_mermaid["Diagram view<br/>ui component<br/>[MermaidDiagram.tsx]"]
end

subgraph group_g_ops["Delivery"]
  node_n_config["Config<br/>runtime settings<br/>[config.py]"]
  node_n_ci["CI/CD<br/>workflow<br/>[ci.yml]"]
end

node_n_cli -->|"reads settings"| node_n_config
node_n_cli -->|"scan local"| node_n_ingest_local
node_n_cli -->|"scan remote"| node_n_ingest_github
node_n_ingest_local -->|"snapshot"| node_n_scanner
node_n_ingest_github -->|"snapshot"| node_n_scanner
node_n_scanner -->|"memoize"| node_n_cache
node_n_scanner -->|"selected files"| node_n_graph
node_n_graph -->|"ranked path"| node_n_analyzer
node_n_graph -->|"structural data"| node_n_models
node_n_analyzer -->|"uses templates"| node_n_prompts
node_n_analyzer -->|"model calls"| node_n_llm
node_n_llm -->|"provider config"| node_n_config
node_n_analyzer -->|"reuse results"| node_n_cache
node_n_analyzer -->|"artifacts"| node_n_builder
node_n_builder -->|"populate docs"| node_n_models
node_n_rag -->|"answer with context"| node_n_llm
node_n_rag -->|"reuse index"| node_n_cache
node_n_builder -->|"render markdown"| node_n_md_export
node_n_builder -->|"render json"| node_n_json_export
node_n_builder -->|"render html"| node_n_html_export
node_n_server -->|"mounts"| node_n_scan_router
node_n_server -->|"mounts"| node_n_wiki_router
node_n_server -->|"mounts"| node_n_chat_router
node_n_scan_router -->|"accepts local"| node_n_ingest_local
node_n_scan_router -->|"accepts remote"| node_n_ingest_github
node_n_wiki_router -->|"serve wiki"| node_n_builder
node_n_chat_router -->|"serve chat"| node_n_rag
node_n_frontend_app -->|"uses client"| node_n_frontend_api
node_n_frontend_app -->|"navigates"| node_n_home
node_n_frontend_app -->|"navigates"| node_n_wiki_view
node_n_frontend_app -->|"navigates"| node_n_chat_view
node_n_wiki_view -->|"shows"| node_n_sidebar
node_n_wiki_view -->|"shows"| node_n_content
node_n_wiki_view -->|"shows diagrams"| node_n_mermaid
node_n_frontend_api -->|"fetch wiki"| node_n_wiki_router
node_n_frontend_api -->|"fetch chat"| node_n_chat_router
node_n_frontend_api -->|"trigger scan"| node_n_scan_router
node_n_ci -->|"tests package"| node_n_cli
node_n_ci -->|"builds ui"| node_n_frontend_app

click node_n_cli "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/__main__.py"
click node_n_config "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/config.py"
click node_n_ingest_local "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/ingest/local.py"
click node_n_ingest_github "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/ingest/github.py"
click node_n_scanner "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/scanner.py"
click node_n_graph "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/graph.py"
click node_n_cache "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/cache.py"
click node_n_analyzer "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/analyzer.py"
click node_n_prompts "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/llm/prompts.py"
click node_n_llm "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/llm/client.py"
click node_n_models "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/models.py"
click node_n_builder "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/wiki_builder.py"
click node_n_rag "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/rag.py"
click node_n_md_export "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/markdown.py"
click node_n_json_export "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/json_export.py"
click node_n_html_export "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/html.py"
click node_n_server "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/app.py"
click node_n_scan_router "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/scan.py"
click node_n_wiki_router "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/wiki.py"
click node_n_chat_router "https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/chat.py"
click node_n_frontend_app "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/App.tsx"
click node_n_frontend_api "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/lib/api.ts"
click node_n_home "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/Home.tsx"
click node_n_wiki_view "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/WikiView.tsx"
click node_n_chat_view "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/ChatView.tsx"
click node_n_sidebar "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/WikiSidebar.tsx"
click node_n_content "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/WikiContent.tsx"
click node_n_mermaid "https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/MermaidDiagram.tsx"
click node_n_ci "https://github.com/he-yufeng/repowiki/blob/main/.github/workflows/ci.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_cli,node_n_ingest_local,node_n_ingest_github,node_n_scanner,node_n_graph,node_n_cache,node_n_analyzer,node_n_prompts,node_n_llm,node_n_models,node_n_builder,node_n_rag,node_n_md_export,node_n_json_export,node_n_html_export,node_n_server,node_n_scan_router,node_n_wiki_router,node_n_chat_router toneBlue
class node_n_frontend_app,node_n_frontend_api,node_n_home,node_n_wiki_view,node_n_chat_view,node_n_sidebar,node_n_content,node_n_mermaid toneAmber
class node_n_config,node_n_ci toneMint
```

## Components

### Python backend

CLI, server, ingestion, analysis, export, and retrieval pipeline.

- [CLI entry](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/__main__.py): command entrypoint · Command-line interface and main executable entrypoints. · `src/repowiki/__main__.py`
- [Local ingest](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/ingest/local.py): source adapter · Loads a repository snapshot from a local filesystem path. · `src/repowiki/ingest/local.py`
- [GitHub ingest](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/ingest/github.py): source adapter · Fetches or clones repository content from a GitHub URL. · `src/repowiki/ingest/github.py`
- [Scanner](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/scanner.py): file selection · Walks the tree, filters noise, detects languages, and selects candidate files. · `src/repowiki/core/scanner.py`
- [Dep graph](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/graph.py): ranking graph · Builds cross-file dependency links and ranks important files. · `src/repowiki/core/graph.py`
- [Cache](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/cache.py): sqlite cache · SQLite-backed cache for reusing unchanged scan and analysis results. · `src/repowiki/core/cache.py`
- [Analyzer](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/analyzer.py): llm pipeline · Runs multi-pass structured analysis over selected files and repo context. · `src/repowiki/core/analyzer.py`
- [Prompts](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/llm/prompts.py): prompt templates · Prompt templates used by the analysis pipeline. · `src/repowiki/llm/prompts.py`
- [LLM client](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/llm/client.py): model adapter · Abstracts model/provider access through litellm. · `src/repowiki/llm/client.py`
- [Models](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/models.py): data contracts · Shared data shapes passed between scan, graph, analysis, and export stages. · `src/repowiki/core/models.py`
- [Wiki builder](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/wiki_builder.py): assembly stage · Converts analysis artifacts into a structured wiki document. · `src/repowiki/core/wiki_builder.py`
- [RAG chat](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/core/rag.py): retrieval qa · TF-IDF based retrieval and grounded terminal Q&A path. · `src/repowiki/core/rag.py`
- [Markdown export](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/markdown.py): export format · Renders the wiki as a directory of Markdown files. · `src/repowiki/export/markdown.py`
- [JSON export](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/json_export.py): export format · Produces structured machine-readable wiki output. · `src/repowiki/export/json_export.py`
- [HTML export](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/export/html.py): export format · Builds a self-contained HTML artifact with diagrams. · `src/repowiki/export/html.py`
- [API server](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/app.py): fastapi app · HTTP application exposing scan, wiki, and chat endpoints. · `src/repowiki/server/app.py`
- [Scan API](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/scan.py): api router · Server routes for repository scanning and ingestion. · `src/repowiki/server/routers/scan.py`
- [Wiki API](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/wiki.py): api router · Server routes for wiki retrieval and document access. · `src/repowiki/server/routers/wiki.py`
- [Chat API](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/server/routers/chat.py): api router · Server routes for chat and Q&A interactions. · `src/repowiki/server/routers/chat.py`

### Browser frontend

React/Vite UI that consumes backend API responses.

- [UI app](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/App.tsx): react app · Vite/React browser application shell. · `frontend/src/App.tsx`
- [API client](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/lib/api.ts): frontend client · Browser-side API wrapper for backend requests. · `frontend/src/lib/api.ts`
- [Home](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/Home.tsx): page · Frontend landing and entry page. · `frontend/src/pages/Home.tsx`
- [Wiki view](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/WikiView.tsx): page · Frontend page for browsing wiki content. · `frontend/src/pages/WikiView.tsx`
- [Chat view](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/pages/ChatView.tsx): page · Frontend page for conversational Q&A. · `frontend/src/pages/ChatView.tsx`
- [Sidebar](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/WikiSidebar.tsx): ui component · Navigation component for wiki structure. · `frontend/src/components/WikiSidebar.tsx`
- [Content pane](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/WikiContent.tsx): ui component · Displays rendered wiki text and sections. · `frontend/src/components/WikiContent.tsx`
- [Diagram view](https://github.com/he-yufeng/repowiki/blob/main/frontend/src/components/MermaidDiagram.tsx): ui component · Renders Mermaid diagrams in the browser. · `frontend/src/components/MermaidDiagram.tsx`

### Delivery

Configuration, packaging, and CI workflows.

- [Config](https://github.com/he-yufeng/repowiki/blob/main/src/repowiki/config.py): runtime settings · Central runtime configuration and provider/API settings. · `src/repowiki/config.py`
- [CI/CD](https://github.com/he-yufeng/repowiki/blob/main/.github/workflows/ci.yml): workflow · Build, test, and publish workflows. · `.github/workflows/ci.yml`

## Connections

- CLI entry → Config: reads settings · CLI resolves runtime configuration and model selection.
- CLI entry → Local ingest: scan local · CLI can start from a local filesystem repository.
- CLI entry → GitHub ingest: scan remote · CLI can start from a GitHub repository source.
- Local ingest → Scanner: snapshot · Local source is normalized before scanning.
- GitHub ingest → Scanner: snapshot · GitHub source is normalized before scanning.
- Scanner → Cache: memoize · Scanner consults cache to skip unchanged work.
- Scanner → Dep graph: selected files · Scan output feeds dependency graph construction.
- Dep graph → Analyzer: ranked path · Important files and reading order guide analysis.
- Dep graph → Models: structural data · Graph produces shared repository structure objects.
- Analyzer → Prompts: uses templates · Analysis is driven by structured prompt templates.
- Analyzer → LLM client: model calls · Analyzer delegates generation to the model client.
- LLM client → Config: provider config · Model client uses configured API keys and model settings.
- Analyzer → Cache: reuse results · Analysis can reuse cached file and repo outputs.
- Analyzer → Wiki builder: artifacts · Structured analysis is assembled into wiki content.
- Wiki builder → Models: populate docs · Wiki builder emits typed document structures.
- RAG chat → LLM client: answer with context · Retrieval pipeline sends grounded snippets to the model.
- RAG chat → Cache: reuse index · Chat path can reuse cached repo state.
- Wiki builder → Markdown export: render markdown · Wiki structure can be exported as Markdown files.
- Wiki builder → JSON export: render json · Wiki structure can be exported as machine-readable JSON.
- Wiki builder → HTML export: render html · Wiki structure can be exported as standalone HTML.
- API server → Scan API: mounts · FastAPI app mounts scan routes.
- API server → Wiki API: mounts · FastAPI app mounts wiki routes.
- API server → Chat API: mounts · FastAPI app mounts chat routes.
- Scan API → Local ingest: accepts local · HTTP scan API can ingest local repositories.
- Scan API → GitHub ingest: accepts remote · HTTP scan API can ingest GitHub repositories.
- Wiki API → Wiki builder: serve wiki · Wiki endpoints expose built documentation.
- Chat API → RAG chat: serve chat · Chat endpoints expose retrieval-based Q&A.
- UI app → API client: uses client · React app communicates through the API wrapper.
- UI app → Home: navigates · App shell hosts the home page.
- UI app → Wiki view: navigates · App shell hosts the wiki browser page.
- UI app → Chat view: navigates · App shell hosts the chat page.
- Wiki view → Sidebar: shows · Wiki page uses the navigation sidebar.
- Wiki view → Content pane: shows · Wiki page renders wiki content.
- Wiki view → Diagram view: shows diagrams · Wiki page renders architecture diagrams.
- API client → Wiki API: fetch wiki · Browser retrieves wiki data from the backend API.
- API client → Chat API: fetch chat · Browser sends chat requests to the backend API.
- API client → Scan API: trigger scan · Browser can trigger repository scans through the API.
- CI/CD → CLI entry: tests package · CI validates the Python tool entrypoints and workflows.
- CI/CD → UI app: builds ui · CI validates the browser frontend build.