How weaviate/structured-rag works
Main Components
a) Core Components:
• Input Processing Layer
• WikiQuestions Dataset Handler
• JSON Template Manager
• Prompt Formation System
b) Model Integration Layer:
• LLM Connectors
• Gemini Interface
• Llama3 (Ollama) Interface
• GPT-4 Interface
• Claude Interface
c) Testing Framework:
• Task Executors
• String-based Tasks
• Integer-based Tasks
• Boolean Tasks
• -based Tasks
• Complex Object Tasks
• Evaluation System
• Metrics Calculator
• Results Aggregator
d) Visualization Layer:
• Results Processing
• Performance Visualization
• Comparative Analysis
Component Relationships
Use the following connection types:
• Double lines: API interactions
Specific Implementation Details
a) Data Flow Path:
Input Data → Prompt Formation → LLM Processing → Response Parsing → Evaluation → Visualization
b) Testing Pipeline:
Task Selection → Template Application → Model Execution → Result Validation → Metrics Calculation
Key Architectural Elements
• Parallel processing capabilities for different models
• Structured output formats (JSON templates)
• Various task types:
• GenerateAnswer
• RateContext
• AssessAnswerability
• ParaphraseQuestions
• RAGAS
• GenerateAnswerWithConfidence
• GenerateAnswersWithConfidence
Integration Points
• Connections to external LLM APIs
• Result storage and aggregation points
• Visualization pipeline connections
Model comparison capabilities (Gemini vs. Llama3)
This will create a comprehensive visualization of how the StructuredRAG system processes, evaluates, and compares different LLM responses in structured JSON formats.