# weaviate/structured-rag architecture

> Architecture diagram of the GitHub repository weaviate/structured-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/weaviate/structured-rag
- Repository: https://github.com/weaviate/structured-rag

## Overview

##### Main Components

a) Core Components:
- Input Processing Layer
- WikiQuestions Dataset Handler
- JSON Template Manager
- Prompt Formation System

b) Model Integration Layer:
- LLM Connectors
- Gemini Interface
- Llama3 (Ollama) Interface
- GPT-4 Interface
- Claude Interface

c) Testing Framework:
- Task Executors
- String-based Tasks
- Integer-based Tasks
- Boolean Tasks
- -based Tasks
- Complex Object Tasks
- Evaluation System
- Metrics Calculator
- Results Aggregator

d) Visualization Layer:
- Results Processing
- Performance Visualization
- Comparative Analysis

##### Component Relationships

Use the following connection types:
- Double lines: API interactions

##### Specific Implementation Details

a) Data Flow Path:
**Input Data → Prompt Formation → LLM Processing → Response Parsing → Evaluation → Visualization**

b) Testing Pipeline:
**Task Selection → Template Application → Model Execution → Result Validation → Metrics Calculation**

##### Key Architectural Elements

- Parallel processing capabilities for different models
- Structured output formats (JSON templates)
- Various task types:
- GenerateAnswer
- RateContext
- AssessAnswerability
- ParaphraseQuestions
- RAGAS
- GenerateAnswerWithConfidence
- GenerateAnswersWithConfidence

##### Integration Points

- Connections to external LLM APIs
- Result storage and aggregation points
- Visualization pipeline connections

##### Model comparison capabilities (Gemini vs. Llama3)

This will create a comprehensive visualization of how the StructuredRAG system processes, evaluates, and compares different LLM responses in structured JSON formats.

## Diagram (Mermaid)

```mermaid
graph TB
    classDef core fill:#b3d1ff,stroke:#333
    classDef model fill:#90EE90,stroke:#333
    classDef testing fill:#FFB366,stroke:#333
    classDef viz fill:#DDA0DD,stroke:#333
    classDef storage fill:#E6E6E6,stroke:#333

    subgraph Input
        WQ["WikiQuestions Dataset"]:::core
        TM["JSON Template Manager"]:::core
        PF["Prompt Formation System"]:::core
    end

    subgraph Models
        direction TB
        LLM["LLM Base Classes"]:::model
        GEM["Gemini Interface"]:::model
        LLAMA["Llama3 Interface"]:::model
        GPT["GPT-4 Interface"]:::model
        CLAUDE["Claude Interface"]:::model
        VLLM["VLLM Integration"]:::model
    end

    subgraph Testing
        direction TB
        TE["Task Executors"]:::testing
        BT["Batch Testing"]:::testing
        ES["Evaluation System"]:::testing
        TH["Test Helpers"]:::testing
    end

    subgraph Visualization
        direction TB
        RP["Results Processing"]:::viz
        PV["Performance Visualization"]:::viz
        SA["Single Result Analysis"]:::viz
        EA["Error Analysis"]:::viz
    end

    subgraph Storage
        RD["Results Directory"]:::storage
        ER["Experimental Results"]:::storage
    end

    WQ --> PF
    TM --> PF
    PF --> LLM

    LLM --> |API| GEM
    LLM --> |API| LLAMA
    LLM --> |API| GPT
    LLM --> |API| CLAUDE
    LLM --> |API| VLLM

    GEM & LLAMA & GPT & CLAUDE & VLLM --> TE
    TE --> BT
    BT --> ES
    TH --> ES
    ES --> RP
    RP --> PV
    RP --> SA
    RP --> EA
    ES --> RD
    RD --> ER

    click WQ "https://github.com/weaviate/structured-rag/blob/main/data/WikiQuestions.json"
    click PF "https://github.com/weaviate/structured-rag/blob/main/structured_rag/mock_gfl/fstring_prompts.py"
    click LLM "https://github.com/weaviate/structured-rag/blob/main/structured_rag/models.py"
    click VLLM "https://github.com/weaviate/structured-rag/tree/main/structured_rag/mock_gfl/modal_vllm_outlines/"
    click TE "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/run_scripts/run_test.py"
    click BT "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/run_scripts/run_batch_test.py"
    click ES "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/utils_and_metrics/metrics.py"
    click TH "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/utils_and_metrics/helpers.py"
    click RP "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/result_visualization/aggregate_result_jsons.py"
    click PV "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/result_visualization/visualize.py"
    click SA "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/result_visualization/visualize_single_result.py"
    click EA "https://github.com/weaviate/structured-rag/blob/main/structured_rag/run_test/result_visualization/dspy_error_analysis.py"
    click RD "https://github.com/weaviate/structured-rag/tree/main/structured_rag/run_test/results/"
    click ER "https://github.com/weaviate/structured-rag/tree/main/structured_rag/run_test/results/experimental-results-9-11-24/"

    %% Legend
    classDef legendClass fill:none,stroke:none
    subgraph Legend
        L1["Core Components"]:::core
        L2["Model Components"]:::model
        L3["Testing Components"]:::testing
        L4["Visualization Components"]:::viz
        L5["Storage Components"]:::storage
    end
```
