How go-graphite/go-carbon works
Main Components Identification
a. Receivers (Multiple Input Protocols):
• TCP/UDP (Plain text protocol)
• Pickle Protocol
• HTTP
• Kafka
• PubSub
• Protocol Buffers
b. Core Components:
• Cache System
• Persister
• CarbonServer (HTTP API)
• CarbonLink (Query Interface)
• Tags System
• gRPC API
c. Storage:
• Whisper Files
• File System
[External Clients/Data Sources] → [Input Layer] → [Processing Layer] → [Storage Layer]
• ↑
• [Query/API Layer]
Component Relationships
a. Data Ingestion Flow:
• External sources → Receivers
• Receivers → Cache
• Cache → Persister
• Persister → Whisper Files
b. Query Flow:
• Client Queries → CarbonServer/CarbonLink/gRPC API
• APIs → Cache (for recent data)
• APIs → Whisper Files (for historical data)
Detailed Component Specifications
Following details for each component:
a. Cache:
• Max size configuration
• Write strategy options
• Bloom filter capability
b. Persister:
• Multiple worker support
• Whisper file management
• Schema and aggregation handling
c. CarbonServer:
• HTTP API endpoints
• Query cache capabilities
• Index types (Trigram/Trie)
d. Receivers:
• Supported protocols
• Buffer configurations
• Rate limiting options
Additional Elements
Supporting elements:
a. Configuration Components:
• Storage schemas
• Storage aggregation
• Storage quotas
b. Monitoring/Metrics:
• Internal statistics
• Prometheus metrics
• Performance metrics
Notes Section
Important notes about:
• Scalability considerations
• Performance optimizations
• OS tuning recommendations
• File system requirements
• Multiple input protocol support
• Efficient caching system
• Flexible storage options
• Various query interfaces
• Performance monitoring capabilities
This design accurately represents the project's architecture as shown in the file structure and described in the README, particularly focusing on the core components and their interactions in the Graphite/Carbon server implementation.