# xuchenhao001/drag architecture

> Architecture diagram of the GitHub repository xuchenhao001/drag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xuchenhao001/drag
- Repository: https://github.com/xuchenhao001/drag

## Overview

##### Project type and purpose
- Type: Python-based simulation framework for distributed Retrieval-Augmented Generation (RAG)
- Purpose: Orchestrate a network of “peer” agents, each with its own knowledge base, that retrieve documents, call a local LLM service (Ollama), evaluate results, log metrics, and produce figures.

##### File structure
- Top-level entry point:
- simulator.py
- auto_scripts/ (default.sh, demo.sh)
- Configuration:
- config/data/*.yaml (datasets: medical, MMLU, news)
- config/llm/*.yaml (model settings: llama32, gemma2, qwen25)
- config/rag.yaml (RAG network parameters)
- Core logic modules (modules/*.py):
- data_types.py (data models)
- knowledge_base.py (builds/shards docs)
- rag_network.py (distributed retrieval)
- peer.py (peer node abstraction)
- llm.py (Ollama API wrapper + templates)
- evaluator.py (metrics)
- exp_logger.py (logging)
- options.py (CLI flags)
- Templates: templates/*.tmpl (prompt formats)
- Visualization: figures/*.py (plot generation)

##### README for additional insights
- Requires Python 3.10, Ollama LLM service, NLTK data
- Uses Hugging Face datasets and Ollama LLM models
- Run via `python simulator.py`

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration Layer"
        Orchestrator["Orchestrator\n(simulator.py)"]:::compute
        Options["Options Parser\n(modules/options.py)"]:::compute
        ConfigLoader["Config Loader"]:::compute
        AutoScripts["Automation Scripts\n(auto_scripts/default.sh,\nauto_scripts/demo.sh)"]:::external
    end

    %% Configuration Storage
    subgraph "Configuration"
        DataConfig["Data YAML\n(config/data/*.yaml)"]:::storage
        LLMConfig["LLM YAML\n(config/llm/*.yaml)"]:::storage
        RAGConfig["RAG Params\n(config/rag.yaml)"]:::storage
    end

    %% Core Processing
    subgraph "Core Layer"
        DataModels["Data Models\n(modules/data_types.py)"]:::compute
        DatasetSource["Dataset Source\n(Hugging Face\nvia modules/knowledge_base.py)"]:::external
        KnowledgeBase["Knowledge Base\n(modules/knowledge_base.py)"]:::storage
        RAGController["RAG Network Controller\n(modules/rag_network.py)"]:::compute
        subgraph "Peer Nodes"
            Peer1["Peer Node\n(modules/peer.py)"]:::compute
        end
        LLMWrapper["LLM Service Wrapper\n(modules/llm.py)"]:::compute
        Templates["Prompt Templates\n(templates/*.tmpl)"]:::storage
        Ollama["Ollama LLM Service"]:::external
        Evaluator["Evaluator\n(modules/evaluator.py)"]:::compute
        Logger["Experiment Logger\n(modules/exp_logger.py)"]:::storage
    end

    %% Visualization Pipeline
    subgraph "Visualization Pipeline"
        PlotMain["Main Plot Script\n(figures/main.py)"]:::compute
        PlotUtils["Plot Utils\n(figures/plot.py)"]:::compute
        FigData["Figures Data\n(figures/data/)"]:::storage
        FigOutput["Figures Output\n(figures/output/)"]:::storage
    end

    %% Relationships
    AutoScripts --> Orchestrator
    Orchestrator --> Options
    Options --> ConfigLoader
    ConfigLoader --> DataConfig
    ConfigLoader --> LLMConfig
    ConfigLoader --> RAGConfig

    ConfigLoader --> RAGController
    RAGController --> Peer1

    Peer1 --> DatasetSource
    Peer1 --> KnowledgeBase
    Peer1 --> LLMWrapper
    LLMWrapper --> Templates
    LLMWrapper --> Ollama

    Peer1 --> Evaluator
    Evaluator --> Logger

    Logger --> PlotMain
    PlotMain --> PlotUtils
    PlotUtils --> FigData
    PlotUtils --> FigOutput

    %% Click Events
    click Orchestrator "https://github.com/xuchenhao001/drag/blob/main/simulator.py"
    click Options "https://github.com/xuchenhao001/drag/blob/main/modules/options.py"
    click DataConfig "https://github.com/xuchenhao001/drag/blob/main/config/data/*.yaml"
    click LLMConfig "https://github.com/xuchenhao001/drag/blob/main/config/llm/*.yaml"
    click RAGConfig "https://github.com/xuchenhao001/drag/blob/main/config/rag.yaml"
    click DataModels "https://github.com/xuchenhao001/drag/blob/main/modules/data_types.py"
    click DatasetSource "https://github.com/xuchenhao001/drag/blob/main/modules/knowledge_base.py"
    click KnowledgeBase "https://github.com/xuchenhao001/drag/blob/main/modules/knowledge_base.py"
    click RAGController "https://github.com/xuchenhao001/drag/blob/main/modules/rag_network.py"
    click Peer1 "https://github.com/xuchenhao001/drag/blob/main/modules/peer.py"
    click LLMWrapper "https://github.com/xuchenhao001/drag/blob/main/modules/llm.py"
    click Templates "https://github.com/xuchenhao001/drag/blob/main/templates/*.tmpl"
    click Evaluator "https://github.com/xuchenhao001/drag/blob/main/modules/evaluator.py"
    click Logger "https://github.com/xuchenhao001/drag/blob/main/modules/exp_logger.py"
    click PlotMain "https://github.com/xuchenhao001/drag/blob/main/figures/main.py"
    click PlotUtils "https://github.com/xuchenhao001/drag/blob/main/figures/plot.py"
    click FigData "https://github.com/xuchenhao001/drag/tree/main/figures/data/"
    click FigOutput "https://github.com/xuchenhao001/drag/tree/main/figures/output/"
    click AutoScripts "https://github.com/xuchenhao001/drag/blob/main/auto_scripts/default.sh"
    click AutoScripts "https://github.com/xuchenhao001/drag/blob/main/auto_scripts/demo.sh"

    %% Styles
    classDef compute fill:#D0E6FF,stroke:#0366D6,stroke-width:1px;
    classDef storage fill:#E6F4EA,stroke:#22863A,stroke-width:1px;
    classDef external fill:#FFF5B1,stroke:#D4A72C,stroke-width:1px;
```
