How qhjqhj00/memorag works
Main Components Identification
a) Memory Model
• Handles global context understanding
• Processes up to 1M tokens
• Provides memory-based knowledge discovery
• Includes caching mechanism for optimization
b) Retrieval System
• Integrates with memory model for enhanced retrieval
• Uses embedding models (e.g., BGE-M3)
• Manages document chunking and indexing
• Implements FAISS for efficient similarity search
c) Generation Model
• Supports multiple LLM options (local models or APIs)
• Integrates with memory and retrieval components
• Handles final response generation
Key Relationships and Data Flow
a) Primary Pipeline Flow:
Input Context → Memory Model → Retrieval System → Generation Model → Final Output
b) Memory Processing Flow:
Raw Input → Chunking → Indexing → KV Cache Storage → Memory Model
Architectural Patterns
Key architectural features:
• Cache Management System
• KV cache storage
• Chunked passages storage
• FAISS index storage
• Model Integration Layer
• How different models can be plugged in
• API integration points
Technical Implementation Details
Important technical elements:
• File Storage System
• Cache directory structure
• Model weights storage
• Index storage
• Model Configuration
• Memory model settings
• Retrieval model configuration
• Generator model options
┌─ Input Layer ─────────────────────────┐
• Context Documents / User Queries │
• ┬─────────────────────┘
• ↓
┌─ Memory Layer ──────────────────────────────────┐
• Global Memory Model │
• Context Processing │
• Caching System │
• ┬──────────────────────────────┘
• ↓
┌─ Retrieval Layer ────────────────────────────┐
• Document Chunking │
• Embedding Generation │
• FAISS Index │
• ┬────────────────────────────┘
• ↓
┌─ Generation Layer ───────────────────────────┐
• LLM Integration │
• API Support │
• Response Generation │
• ┬────────────────────────────┘
• ↓
┌─ Output Layer ──────────────────────────┐
• Generated Responses │
• ┘
Additional Elements
• Version information and supported model types
• Configuration options and parameters
• Performance optimization points
Implementation Notes
• Input/output formats at each stage
• Cache hit/miss paths