How tikv/raft-engine works
Main Components
a. Core Components:
• Memtable System
• Log File Management
• Write Pipeline
• Garbage Collection System
• Storage Layer
• Compression Layer (LZ4)
b. Supporting Components:
• Configuration Management
• Error Handling
• Metrics Collection
• File System Interface
Key Relationships to Highlight
• Write flow: Client → Log Batch → Write Pipeline → Log File
• Read flow: Client → Memtable → Log File (if needed)
• GC flow: Purge Trigger → Log File Scanner → Compaction Process
Special Elements
• Concurrent Write Queue Management
• Compression Pipeline
• File Rotation Logic
• GC Collaboration System
Important Details to Capture
• The log-structured nature of the storage
• The shared log stream between Raft Groups
• The separation between logical and physical deletion
• The collaborative GC mechanism
• The atomic write guarantees
Component Relationships
Clear boundaries between:
• In-memory components (Memtable)
• On-disk components (Log Files)
• Temporary states (Write Queue)
• Background processes (GC)
Configuration Points
Configurable elements:
• Log file rotation thresholds
• Memory limits (when swap feature is enabled)
• Compression settings
• GC parameters
Testing/Monitoring Points
Indicators for:
• Metrics collection points
• Failpoint locations (when enabled)
• Benchmark measurement points