# terry-xu-666/noderag architecture

> Architecture diagram of the GitHub repository terry-xu-666/noderag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/terry-xu-666/noderag
- Repository: https://github.com/terry-xu-666/noderag

## Overview

##### Project type and purpose
- NodeRAG is a Python-based, full-stack RAG (Retrieval-Augmented Generation) system.
- It provides:
- A back-end library for building and querying a heterogeneous graph of “nodes.”
- A document ingestion/indexing pipeline.
- A retrieval & answer-generation pipeline (search + LLM integration).
- Visualization tooling (static HTML & WebUI).

##### File structure
Top-level modules:
- NodeRAG/build – document-to-graph ingestion pipeline (text splitting, embedding, graph construction, incremental updates).
- NodeRAG/search – graph retrieval logic.
- NodeRAG/LLM – LLM wrappers, routing, and state management.
- NodeRAG/storage – graph persistence and ID/graph mappings.
- NodeRAG/Vis – standalone HTML visualizations.
- NodeRAG/WebUI – a Python-based web interface (app.py).
- NodeRAG/config – YAML & Python config.
- NodeRAG/logging – standardized logging.
- NodeRAG/utils – helper functions (HNSW, PPR, prompts, YAML ops, token utils, etc.).

##### README for additional insights
- The system supports two primary flows:
a. Indexing (ingesting raw text → heterogeneous graph)
b. Answering (user query → retrieval → LLM generation)
- It offers both a WebUI and HTML-based visual exploration tool.
- Emphasizes heterogeneous node types, incremental graph updates, and explainable retrieval.

##### Elements
- Clear labels for each component (e.g., “TextSplitter”, “HNSWGraphBuilder”, “LLMRouter”)

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion"
        direction TB
        RawDocs["Raw Documents"]:::external
        Ingest["Ingestion Pipeline"]:::service
        TextSplitter["TextSplitter"]:::service
        EmbeddingGen["Embedding Generator"]:::service
        AttrGen["Attribute Generator"]:::service
        SumGen["Summary Generator"]:::service
        GraphBuilder["Graph Constructor & HNSWGraphBuilder"]:::service

        RawDocs --> Ingest
        Ingest --> TextSplitter
        TextSplitter --> EmbeddingGen
        EmbeddingGen --> AttrGen
        AttrGen --> SumGen
        SumGen --> GraphBuilder
        GraphBuilder --> Storage
    end

    %% Graph Storage Layer
    subgraph "Graph Storage"
        direction TB
        Storage["Graph Storage"]:::storage
        IDGen["ID Generator"]:::service
        GraphMap["Graph Mapping"]:::service
        StoreIF["Storage Interface"]:::service

        Storage --> IDGen
        Storage --> GraphMap
        Storage --> StoreIF
    end

    %% Retrieval Engine Layer
    subgraph "Retrieval Engine"
        direction TB
        SearchLogic["Search Logic"]:::service
        HNSW["HNSW Index"]:::service
        PPR["Personalized PageRank"]:::service
        AnsBase["Answer Base"]:::service

        SearchLogic --> HNSW
        SearchLogic --> PPR
        SearchLogic --> Storage
        SearchLogic --> AnsBase
    end

    %% LLM Module Layer
    subgraph "LLM Module"
        direction TB
        LLMBase["LLM_base"]:::service
        LLMAPI["LLM Router"]:::service
        LLMState["LLM State Manager"]:::service
        PromptMgr["Prompt Manager"]:::service

        LLMBase --> LLMAPI
        LLMAPI --> LLMState
        LLMState --> PromptMgr
        LLMAPI --> ExternalLLM
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        HTMLVis["HTML Visualizer"]:::frontend
        WebUI["WebUI/API"]:::frontend
    end

    %% External Services and Flows
    ExternalLLM["External LLM API"]:::external
    UserQuery["User Query"]:::external

    UserQuery --> SearchLogic
    AnsBase --> LLMBase
    LLMAPI --> WebUI
    HTMLVis --> Storage
    WebUI -->|serves UI| UserQuery

    %% Cross-Cutting Config & Logging
    Config["Config Loader"]:::config
    Logger["Logger & Metrics"]:::logging

    Config --> Ingest
    Config --> Storage
    Config --> SearchLogic
    Config --> LLMBase
    Config --> WebUI
    Config --> HTMLVis

    Ingest --> Logger
    Storage --> Logger
    SearchLogic --> Logger
    LLMBase --> Logger
    WebUI --> Logger
    HTMLVis --> Logger

    %% Click Events
    click Ingest "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/build/__main__.py"
    click TextSplitter "https://github.com/terry-xu-666/noderag/tree/main/NodeRAG/build/component/"
    click EmbeddingGen "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/build/pipeline/embedding.py"
    click AttrGen "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/build/pipeline/attribute_generation.py"
    click SumGen "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/build/pipeline/summary_generation.py"
    click GraphBuilder "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/build/pipeline/HNSW_graph.py"
    click IDGen "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/storage/genid.py"
    click GraphMap "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/storage/graph_mapping.py"
    click StoreIF "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/storage/storage.py"
    click SearchLogic "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/search/search.py"
    click AnsBase "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/search/Answer_base.py"
    click HNSW "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/utils/HNSW.py"
    click PPR "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/utils/PPR.py"
    click LLMBase "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/LLM/LLM_base.py"
    click LLMAPI "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/LLM/LLM_route.py"
    click LLMState "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/LLM/LLM_state.py"
    click PromptMgr "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/utils/prompt/prompt_manager.py"
    click HTMLVis "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/Vis/html/visual_html.py"
    click WebUI "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/WebUI/app.py"
    click Config "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/config/Node_config.yaml"
    click Logger "https://github.com/terry-xu-666/noderag/blob/main/NodeRAG/logging/logger.py"

    %% Styles
    classDef service fill:#ADD8E6,stroke:#000;
    classDef storage fill:#90EE90,stroke:#000;
    classDef external fill:#FFA500,stroke:#000;
    classDef frontend fill:#FFFFE0,stroke:#000;
    classDef config fill:#98FB98,stroke:#000;
    classDef logging fill:#D3D3D3,stroke:#000;
```
