# stormaref/kafkastorm architecture

> Architecture diagram of the GitHub repository stormaref/kafkastorm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/stormaref/kafkastorm
- Repository: https://github.com/stormaref/kafkastorm

## Overview

##### Main Components

a) Core Components:
- Producer System
- ProducerHostedService
- Producer Service
- Message Queue Store
- Consumer System
- ConsumerHostedService
- Consumer Registration
- Message Processing Pipeline
- Registration/Configuration Layer

b) External Dependencies:
- Confluent.Kafka (as the underlying Kafka client)
- Kafka Broker(s)

##### Special Features to Emphasize

Create dedicated sections showing:
- Message queueing mechanism for failed messages
- Concurrent consumer handling
- Producer queue system
- Assembly scanning for automatic consumer registration

##### Detailed Component Breakdown

Producer Side:
**[Application] → [ProducerRegistrationFactory]**
- ↓
**[Producer Service] ← → [Message Store]**
- ↓
**[ProducerHostedService]**
- ↓
**[Confluent.Kafka Producer]**
- ↓
**[Kafka Broker]**

Consumer Side:
**[Kafka Broker]**
- ↑
**[Confluent.Kafka Consumer]**
- ↑
**[ConsumerHostedService]**
- ↑
**[Consumer Registration]**
- ↑
**[Application Consumers]**

##### Important Notes

- Notes about configuration options (e.g., queue limits, bootstrap servers)
- Error handling paths and retry mechanisms
- Where concurrent processing occurs
- Interface boundaries (IConsumer, IProducer, IMessage, IMessageStore)

##### Implementation Details

- How the message queue limit (65536) is implemented
- Where JSON serialization/deserialization occurs
- Points where dependency injection is used
- Assembly scanning mechanism for automatic consumer registration

Remember to include a section showing how the library integrates with standard .NET dependency injection, as this is a key aspect of the implementation.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client["Client Application Layer"]
        APP[Application]
    end

    subgraph Core["KafkaStorm Core Layer"]
        subgraph Registration["Registration Components"]
            KRF["KafkaStorm Registration Factory"]
            CRF["Consumer Registration Factory"]
            PRF["Producer Registration Factory"]
            REG["Registration Extensions"]
        end

        subgraph Producer["Producer System"]
            PS["Producer Service"]
            PHS["Producer Hosted Service"]
            PI["Producer Interface"]
        end

        subgraph Consumer["Consumer System"]
            CHS["Consumer Hosted Service"]
            CI["Consumer Interface"]
        end

        subgraph MessageStore["Message Queue Store"]
            MS["Message Store Service"]
            MSI["Message Store Interface"]
            MI["Message Interface"]
            SM["Stored Message Model"]
        end

        subgraph ErrorHandling["Error Handling"]
            CNE["Consumer Exceptions"]
            PCE["Producer Exceptions"]
            MNE["Message Exceptions"]
        end
    end

    subgraph External["External Layer"]
        CK[Confluent.Kafka]
        KB[Kafka Broker]
    end

    %% Relationships
    APP -->|Uses| KRF
    KRF -->|Creates| PRF
    KRF -->|Creates| CRF
    PRF -->|Configures| PS
    CRF -->|Configures| CHS
    PS -->|Uses| MS
    PS -.->|Implements| PI
    CHS -.->|Implements| CI
    MS -.->|Implements| MSI
    PS -->|Uses| PHS
    PHS -->|Uses| CK
    CHS -->|Uses| CK
    CK -->|Connects| KB

    %% Error Handling
    PS -.->|Throws| PCE
    CHS -.->|Throws| CNE
    MS -.->|Throws| MNE

    %% Click Events
    click PS "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Services/Producer.cs"
    click PHS "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Services/ProducerHostedService.cs"
    click PI "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Interfaces/IProducer.cs"
    click PRF "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Registration/ProducerRegistrationFactory.cs"
    click CHS "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Services/ConsumerHostedService.cs"
    click CI "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Interfaces/IConsumer.cs"
    click CRF "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Registration/ConsumerRegistrationFactory.cs"
    click MS "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Services/MessageStore.cs"
    click MSI "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Interfaces/IMessageStore.cs"
    click MI "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Interfaces/IMessage.cs"
    click SM "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Models/StoredMessage.cs"
    click KRF "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Registration/KafkaStormRegistrationFactory.cs"
    click REG "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Registration/Extensions.cs"
    click CNE "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Exceptions/ConsumerNotConfiguredException.cs"
    click PCE "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Exceptions/ProducerConfigNullException.cs"
    click MNE "https://github.com/stormaref/KafkaStorm/blob/master/KafkaStorm/Exceptions/MessageNullException.cs"

    %% Styling
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px
    classDef external fill:#d3d3d3,stroke:#333
    classDef core fill:#b3e0ff,stroke:#333
    classDef error fill:#ffcccc,stroke:#333
    class CK,KB external
    class PS,PHS,CHS,MS core
    class CNE,PCE,MNE error
```
