How hhhuang/cag works
→ Dataset loading → Knowledge Base preprocessing
→ Context window population
→ Model inference
→ Results output
→ KV-Cache preloading
→ Runtime parameter caching
→ Direct inference access
Key Elements to Highlight
a. Context Window Management
• Size limits (16K-32K tokens)
• Knowledge preloading mechanism
• Cache optimization
b. Comparison Points
• CAG direct path (bold/highlighted)
Implementation Details
a. Configuration Components (Small rectangles)
• Environment settings
• Model parameters
• Dataset configurations
b. Output Handlers (Parallelograms)
• Results directory structure
• Performance metrics storage
• Evaluation outputs
Notes Section (Bottom)
• Context window limitations
• Supported dataset sizes
• Performance characteristics
• Primary Components: #2E86C1 (Blue)
• Cache Operations: #27AE60 (Green)
• Model Components: #8E44AD (Purple)
• Evaluation: #E67E22 (Orange)
The simplified architecture compared to RAG
Keep the abstraction level focused on the key architectural components while maintaining enough detail to show the unique aspects of CAG compared to traditional RAG systems.