How stormaref/kafkastorm works
Main Components
a) Core Components:
• Producer System
• ProducerHostedService
• Producer Service
• Message Queue Store
• Consumer System
• ConsumerHostedService
• Consumer Registration
• Message Processing Pipeline
• Registration/Configuration Layer
b) External Dependencies:
• Confluent.Kafka (as the underlying Kafka client)
• Kafka Broker(s)
Special Features to Emphasize
Create dedicated sections showing:
• Message queueing mechanism for failed messages
• Concurrent consumer handling
• Producer queue system
• Assembly scanning for automatic consumer registration
Detailed Component Breakdown
Producer Side:
[Application] → [ProducerRegistrationFactory]
• ↓
[Producer Service] ← → [Message Store]
• ↓
[ProducerHostedService]
• ↓
[Confluent.Kafka Producer]
• ↓
[Kafka Broker]
Consumer Side:
[Kafka Broker]
• ↑
[Confluent.Kafka Consumer]
• ↑
[ConsumerHostedService]
• ↑
[Consumer Registration]
• ↑
[Application Consumers]
Important Notes
• Notes about configuration options (e.g., queue limits, bootstrap servers)
• Error handling paths and retry mechanisms
• Where concurrent processing occurs
• Interface boundaries (IConsumer, IProducer, IMessage, IMessageStore)
Implementation Details
• How the message queue limit (65536) is implemented
• Where JSON serialization/deserialization occurs
• Points where dependency injection is used
• Assembly scanning mechanism for automatic consumer registration
Remember to include a section showing how the library integrates with standard .NET dependency injection, as this is a key aspect of the implementation.