# hhhuang/cag architecture

> Architecture diagram of the GitHub repository hhhuang/cag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hhhuang/cag
- Repository: https://github.com/hhhuang/cag

## Overview

→ Dataset loading → Knowledge Base preprocessing
→ Context window population
→ Model inference
→ Results output

→ KV-Cache preloading
→ Runtime parameter caching
→ Direct inference access

##### Key Elements to Highlight

a. Context Window Management
- Size limits (16K-32K tokens)
- Knowledge preloading mechanism
- Cache optimization

b. Comparison Points
- CAG direct path (bold/highlighted)

##### Implementation Details

a. Configuration Components (Small rectangles)
- Environment settings
- Model parameters
- Dataset configurations

b. Output Handlers (Parallelograms)
- Results directory structure
- Performance metrics storage
- Evaluation outputs

##### Notes Section (Bottom)
- Context window limitations
- Supported dataset sizes
- Performance characteristics

- Primary Components: #2E86C1 (Blue)
- Cache Operations: #27AE60 (Green)
- Model Components: #8E44AD (Purple)
- Evaluation: #E67E22 (Orange)

##### The simplified architecture compared to RAG

Keep the abstraction level focused on the key architectural components while maintaining enough detail to show the unique aspects of CAG compared to traditional RAG systems.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Data
        DS1["SQuAD Dataset"]
        DS2["HotpotQA Dataset"]
        DS3["Sample QA Dataset"]
        DC["Data Cache"]
        KB["Knowledge Base Storage"]
    end

    subgraph Processing
        CAG["CAG Engine"]
        KVC["KV-Cache Manager"]
        CWM["Context Window Manager"]
    end

    subgraph Model
        LLM["LLaMA 3.1-8B"]
        MPC["Model Parameter Cache"]
    end

    subgraph Evaluation
        BS["BERTScore Calculator"]
        PMC["Performance Metrics"]
        RES1["SQuAD Results"]
        RES2["HotpotQA Results"]
    end

    subgraph Scripts
        EX1["Execution Script"]
        TS1["SQuAD Test"]
        TS2["HotpotQA Test K16-K80"]
        TS3["HotpotQA Test K24-K64"]
        TS4["HotpotQA Test K32-K48"]
    end

    DS1 --> CAG
    DS2 --> CAG
    DS3 --> CAG
    CAG --> KVC
    KVC --> CWM
    CWM --> LLM
    LLM --> MPC
    MPC -.-> KVC
    CAG --> BS
    BS --> PMC
    PMC --> RES1
    PMC --> RES2
    DC -.-> CAG
    KB -.-> CAG

    %% Click events for component mapping
    click DS1 "https://github.com/hhhuang/CAG/tree/main/datasets/squad/"
    click DS2 "https://github.com/hhhuang/CAG/tree/main/datasets/hotpotqa/"
    click DS3 "https://github.com/hhhuang/CAG/blob/main/datasets/rag_sample_qas_from_kis.csv"
    click DC "https://github.com/hhhuang/CAG/tree/main/data_cache/"
    click KVC "https://github.com/hhhuang/CAG/blob/main/kvcache.py"
    click CAG "https://github.com/hhhuang/CAG/blob/main/rag.py"
    click EX1 "https://github.com/hhhuang/CAG/blob/main/scripts/run.sh"
    click RES1 "https://github.com/hhhuang/CAG/tree/main/results/squad-train/"
    click RES2 "https://github.com/hhhuang/CAG/tree/main/results/hotpotqa-train/"
    click TS1 "https://github.com/hhhuang/CAG/blob/main/scripts/random-squad.sh"
    click TS2 "https://github.com/hhhuang/CAG/blob/main/scripts/random-hotpot-k16-k80.sh"
    click TS3 "https://github.com/hhhuang/CAG/blob/main/scripts/random-hotpot-k24-k64.sh"
    click TS4 "https://github.com/hhhuang/CAG/blob/main/scripts/random-hotpot-k32-k48.sh"

    %% Styling
    classDef data fill:#2E86C1,stroke:#333,stroke-width:2px
    classDef processing fill:#27AE60,stroke:#333,stroke-width:2px
    classDef model fill:#8E44AD,stroke:#333,stroke-width:2px
    classDef eval fill:#E67E22,stroke:#333,stroke-width:2px
    classDef script fill:#95A5A6,stroke:#333,stroke-width:2px

    class DS1,DS2,DS3,DC,KB data
    class CAG,KVC,CWM processing
    class LLM,MPC model
    class BS,PMC,RES1,RES2 eval
    class EX1,TS1,TS2,TS3,TS4 script

    %% Legend
    subgraph Legend
        L1["Data Storage"]:::data
        L2["Processing Units"]:::processing
        L3["Model Components"]:::model
        L4["Evaluation Tools"]:::eval
        L5["Scripts"]:::script
    end
```
