# confluentinc/librdkafka architecture

> Architecture diagram of the GitHub repository confluentinc/librdkafka, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/confluentinc/librdkafka
- Repository: https://github.com/confluentinc/librdkafka

## Overview

librdkafka is a C/C++ Apache Kafka client library. Applications configure a client, then produce messages, consume messages, or issue administrative requests to Kafka brokers; the library also provides transactions, security, compression, metrics, and a C++ API. The README establishes these product capabilities, but most implementation files were not sampled, so internal relationships are omitted or marked unverified rather than inferred. Nanopb and generated OpenTelemetry types are represented as an internal telemetry-encoding layer.

Main entry points: [`src/rdkafka.c`](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka.c) (Producer), [`src/rdkafka_broker.c`](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_broker.c) (Broker Protocol), [`src-cpp/rdkafkacpp.h`](https://github.com/confluentinc/librdkafka/blob/master/src-cpp/rdkafkacpp.h) (C++ API), [`src/opentelemetry/metrics.pb.h`](https://github.com/confluentinc/librdkafka/blob/master/src/opentelemetry/metrics.pb.h) (OTel Metrics Types)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client APIs"]
  node_c_api["C API<br/>[rdkafka.h]"]
  node_cpp_api["C++ API<br/>[rdkafkacpp.h]"]
  node_cpp_impl["C++ Bindings<br/>[rdkafkacpp_int.h]"]
end

subgraph group_runtime["Kafka Runtime"]
  node_producer["Producer<br/>[rdkafka.c]"]
  node_consumer["Consumers<br/>[rdkafka_cgrp.c]"]
  node_admin["Admin Client<br/>[rdkafka_admin.c]"]
  node_broker["Broker Protocol<br/>[rdkafka_broker.c]"]
  node_configuration["Configuration<br/>[rdkafka_conf.c]"]
  node_assignment["Group Assignment<br/>[rdkafka_assignor.c]"]
  node_mock["Mock Broker"]
end

subgraph group_capabilities["Client Capabilities"]
  node_transactions["Transactions<br/>[rdkafka_txnmgr.c]"]
  node_security["Security<br/>[rdkafka_sasl.c]"]
  node_oidc["OIDC Tokens"]
  node_compression["Compression<br/>[rdkafka_lz4.c]"]
  node_telemetry["Metrics"]
end

subgraph group_encoding["Telemetry Encoding"]
  node_otel_types["OTel Metrics Types<br/>[metrics.pb.h]"]
  node_nanopb["Nanopb Codec<br/>[pb.h]"]
  node_common_types["OTel Common Types<br/>[common.pb.h]"]
end

node_application(("Application"))
node_kafka[("Kafka Brokers")]

node_application -->|"uses"| node_c_api
node_application -->|"uses"| node_cpp_api
node_c_api -->|"Kafka protocol"| node_kafka
node_cpp_api -->|"Kafka client"| node_kafka
node_application -.->|"produces messages"| node_producer
node_application -.->|"consumes messages"| node_consumer
node_application -.->|"admin requests"| node_admin
node_producer -.->|"dispatches"| node_broker
node_consumer -.->|"fetches from"| node_broker
node_admin -.->|"requests"| node_broker
node_cpp_api -.->|"implemented by"| node_cpp_impl
node_transactions -.->|"extends"| node_producer
node_security -.->|"authenticates"| node_broker
node_oidc -.->|"provides tokens"| node_security
node_compression -.->|"compresses messages"| node_producer
node_telemetry -.->|"encodes metrics"| node_otel_types
node_otel_types -->|"uses types"| node_common_types
node_otel_types -->|"uses codec types"| node_nanopb
node_mock -.->|"simulates broker"| node_kafka

click node_c_api "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka.h"
click node_cpp_api "https://github.com/confluentinc/librdkafka/blob/master/src-cpp/rdkafkacpp.h"
click node_cpp_impl "https://github.com/confluentinc/librdkafka/blob/master/src-cpp/rdkafkacpp_int.h"
click node_producer "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka.c"
click node_consumer "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_cgrp.c"
click node_admin "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_admin.c"
click node_broker "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_broker.c"
click node_configuration "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_conf.c"
click node_assignment "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_assignor.c"
click node_transactions "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_txnmgr.c"
click node_security "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_sasl.c"
click node_oidc "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_sasl_oauthbearer_oidc.c"
click node_compression "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_lz4.c"
click node_telemetry "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_telemetry.c"
click node_otel_types "https://github.com/confluentinc/librdkafka/blob/master/src/opentelemetry/metrics.pb.h"
click node_nanopb "https://github.com/confluentinc/librdkafka/blob/master/src/nanopb/pb.h"
click node_common_types "https://github.com/confluentinc/librdkafka/blob/master/src/opentelemetry/common.pb.h"
click node_mock "https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_mock_handlers.c"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_c_api,node_cpp_api,node_cpp_impl toneBlue
class node_producer,node_consumer,node_admin,node_broker,node_configuration,node_assignment,node_mock,node_kafka toneAmber
class node_transactions,node_security,node_oidc,node_compression,node_telemetry toneMint
class node_otel_types,node_nanopb,node_common_types toneRose
class node_application toneIndigo
```

## Components

### Client APIs

- [C API](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka.h): `src/rdkafka.h`
- [C++ API](https://github.com/confluentinc/librdkafka/blob/master/src-cpp/rdkafkacpp.h): `src-cpp/rdkafkacpp.h`
- [C++ Bindings](https://github.com/confluentinc/librdkafka/blob/master/src-cpp/rdkafkacpp_int.h): `src-cpp/rdkafkacpp_int.h`

### Kafka Runtime

- [Producer](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka.c): `src/rdkafka.c`
- [Consumers](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_cgrp.c): `src/rdkafka_cgrp.c`
- [Admin Client](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_admin.c): `src/rdkafka_admin.c`
- [Broker Protocol](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_broker.c): `src/rdkafka_broker.c`
- [Configuration](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_conf.c): `src/rdkafka_conf.c`
- [Group Assignment](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_assignor.c): `src/rdkafka_assignor.c`
- [Mock Broker](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_mock_handlers.c): `src/rdkafka_mock_handlers.c`

### Client Capabilities

- [Transactions](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_txnmgr.c): `src/rdkafka_txnmgr.c`
- [Security](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_sasl.c): `src/rdkafka_sasl.c`
- [OIDC Tokens](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_sasl_oauthbearer_oidc.c): `src/rdkafka_sasl_oauthbearer_oidc.c`
- [Compression](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_lz4.c): `src/rdkafka_lz4.c`
- [Metrics](https://github.com/confluentinc/librdkafka/blob/master/src/rdkafka_telemetry.c): `src/rdkafka_telemetry.c`

### Telemetry Encoding

- [OTel Metrics Types](https://github.com/confluentinc/librdkafka/blob/master/src/opentelemetry/metrics.pb.h): `src/opentelemetry/metrics.pb.h`
- [Nanopb Codec](https://github.com/confluentinc/librdkafka/blob/master/src/nanopb/pb.h): `src/nanopb/pb.h`
- [OTel Common Types](https://github.com/confluentinc/librdkafka/blob/master/src/opentelemetry/common.pb.h): `src/opentelemetry/common.pb.h`

### Other components

- **Application**
- **Kafka Brokers**

## Connections

- Application → C API: uses (evidence: `README.md`)
- Application → C++ API: uses (evidence: `README.md`)
- C API → Kafka Brokers: Kafka protocol (evidence: `README.md`)
- C++ API → Kafka Brokers: Kafka client (evidence: `README.md`)
- Application → Producer: produces messages (evidence: `README.md`)
- Application → Consumers: consumes messages (evidence: `README.md`)
- Application → Admin Client: admin requests (evidence: `README.md`)
- Producer → Broker Protocol: dispatches
- Consumers → Broker Protocol: fetches from
- Admin Client → Broker Protocol: requests
- C++ API → C++ Bindings: implemented by (evidence: `src-cpp/rdkafkacpp_int.h`)
- Transactions → Producer: extends
- Security → Broker Protocol: authenticates
- OIDC Tokens → Security: provides tokens
- Compression → Producer: compresses messages
- Metrics → OTel Metrics Types: encodes metrics
- OTel Metrics Types → OTel Common Types: uses types (evidence: `src/opentelemetry/metrics.pb.h`)
- OTel Metrics Types → Nanopb Codec: uses codec types (evidence: `src/opentelemetry/metrics.pb.h`)
- Mock Broker → Kafka Brokers: simulates broker
