# graphrag/ms-graphrag architecture

> Architecture diagram of the GitHub repository graphrag/ms-graphrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/graphrag/ms-graphrag
- Repository: https://github.com/graphrag/ms-graphrag

## Overview

##### Main Components
- Input Ingestion and Preprocessing:
– Represented by components in the “graphrag/index/input” area. These modules support various file types (CSV, JSON, TXT, etc.) and also have a “data_model” to normalize data.
– Include connections from external data sources (e.g., local files, Azure blob storage via “unified-search-app/app/knowledge_loader/data_sources”) into the pipeline.

- Indexing & Graph Construction:
– Components in “graphrag/index/operations” such as building noun graphs, extractors (for covariates, graph extraction, community reports), chunking text, and text-splitting.
– The process typically starts from raw data, passes through several transformation steps (e.g., chunking, text embedding, summarization), and then constructs a graph (community, relationships, etc.).

- Embedding Generation & Language Model Integration:
– Represent modules under “graphrag/language_model” including providers (e.g., fnllm), response handling, and LLM manager.
– This layer handles integration with external LLM services to generate embeddings and process prompts, as well as support prompt tuning (see “graphrag/prompt_tune” and “graphrag/prompts”).

- Caching, Vector Stores, and Storage:
– Show the caching layer (e.g., “graphrag/cache” with different strategies such as memory, file, and JSON cache).
– Include storage interfaces like “graphrag/storage” and “graphrag/vector_stores” (integrations with Azure AI search, Cosmos DB, and LanceDB). These enable persistent storage and fast retrieval of processed data and embeddings.

- Query Processing & Context Building:
– Modules under “graphrag/query” (structured search, local/global search, drift search, and context builders) are crucial for receiving search queries and returning results using processed graphs and embeddings.
– Outline how query inputs are converted via context builders and routed to the appropriate pipelines.

- Unified Search Application (Frontend/API Interface):
– The “unified-search-app” directory contains its own set of files (Dockerfile, README, app logic, UI pages, state management, and a knowledge loader). This acts as an end-user front end (or API layer) which ties into the GraphRAG engine.

##### Relationships & Data Flow
– Show a clear flow from data ingestion → preprocessing → graph/embedding generation. Illustrate that the pipeline is modular and each step has well-defined outputs that feed into the next stage.
– Emphasize that query requests (possibly from the unified-search-app UI or API) are sent through a query processor, which uses context builders and language model interactions to retrieve and present relevant results.

##### Architectural Patterns & Design Principles
– The structure appears modular with clear separation of concerns. Each major operation (data ingestion, graph processing, storage, language model interaction) is isolated in its own package.
– Note the use of caching and pluggable vector store backends which indicate patterns like factory and strategy are in play.
– There is also a clear CLI and API entry point under “graphrag/cli” and “graphrag/api” reinforcing external interfacing via both command-line and programmatic calls.

##### Technology and Frameworks
– Deployment and CI/CD aspects (shown in the .github/workflows and Dockerfile under unified-search-app) indicate continuous integration and containerized deployment. They can be represented as supporting infrastructure.

##### Detailing the Architecture
– For managing configurations and the evolving pipeline state, mention the “graphrag/config” modules.

This detailed view should help any principal software engineer quickly grasp the overall system design, its components, and their interactions for further development and maintenance.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Ingestion & Preprocessing
    subgraph "Input Ingestion & Preprocessing"
        id2["External Data Sources"]:::ingestion
        id1["File Input & Data Normalization"]:::ingestion
    end

    %% Indexing & Graph Construction
    subgraph "Indexing & Graph Construction"
        id5["Text Chunking"]:::indexing
        id6["Text Splitting"]:::indexing
        id4["Noun Graph Builder"]:::indexing
        id3["Graph Operations"]:::indexing
    end

    %% Embedding Generation & Language Model Integration
    subgraph "Embedding Generation & Language Model Integration"
        id7["LLM Integration"]:::llm
        id8["LLM Provider (fnllm)"]:::llm
        id9["Prompt Tuning"]:::llm
        id10["Prompts & Templates"]:::llm
    end

    %% Caching, Vector Stores, and Storage
    subgraph "Caching, Vector Stores, and Storage"
        id11["Caching Layer"]:::caching
        id12["Storage Interface"]:::caching
        id13["Vector Stores"]:::caching
    end

    %% Query Processing & Context Building
    subgraph "Query Processing & Context Building"
        id14["Query Engine"]:::query
        id15["Context Builder"]:::query
        id16["Structured Search"]:::query
    end

    %% Unified Search Application (Frontend/API)
    subgraph "Unified Search Application (Frontend/API)"
        id17["Unified Search App"]:::frontend
    end

    %% Connections & Data Flow
    id2 --> id1
    id1 --> id5
    id5 --> id6
    id6 --> id4
    id4 --> id3
    id3 --> id7
    id7 --> id8
    id8 --> id9
    id9 --> id10
    id10 --> id11
    id11 --> id12
    id12 --> id13
    id13 --> id14
    id14 --> id15
    id15 --> id16
    id16 --> id17
    id17 -->|"query"| id14

    %% Click Events
    click id1 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/index/input"
    click id2 "https://github.com/graphrag/ms-graphrag/tree/main/unified-search-app/app/knowledge_loader/data_sources"
    click id3 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/index/operations"
    click id4 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/index/operations/build_noun_graph"
    click id5 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/index/operations/chunk_text"
    click id6 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/index/text_splitting"
    click id7 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/language_model"
    click id8 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/language_model/providers/fnllm"
    click id9 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/prompt_tune"
    click id10 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/prompts"
    click id11 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/cache"
    click id12 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/storage"
    click id13 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/vector_stores"
    click id14 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/query"
    click id15 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/query/context_builder"
    click id16 "https://github.com/graphrag/ms-graphrag/tree/main/graphrag/query/structured_search"
    click id17 "https://github.com/graphrag/ms-graphrag/tree/main/unified-search-app"

    %% Styles
    classDef ingestion fill:#add8e6,stroke:#000,stroke-width:1px;
    classDef indexing fill:#90ee90,stroke:#000,stroke-width:1px;
    classDef llm fill:#ffcccb,stroke:#000,stroke-width:1px;
    classDef caching fill:#ffe4b5,stroke:#000,stroke-width:1px;
    classDef query fill:#d3d3d3,stroke:#000,stroke-width:1px;
    classDef frontend fill:#f4a460,stroke:#000,stroke-width:1px;
```
