How confluentinc/librdkafka works
librdkafka is a native C/C++ Apache Kafka client library. Applications configure a client and use producer, consumer, share-consumer, or admin APIs to exchange messages or perform administrative operations with Kafka brokers. Its runtime also covers transactions, security, compression, telemetry, and statistics. The README documents these capabilities, but most implementation files were not sampled; accordingly, internal call relationships are omitted or shown as unverified rather than inferred from filenames. The C++ implementation’s dependency on the C API is visible in the sampled internal header.
Main entry points: rdkafka.c Client Core · rdkafka_cgrp.c Consumer · rdkafka.h C API · rdkafka_sasl_oauthbearer.c SASL OAuth
Components 19
Client APIs
- C API
src/rdkafka.h - C++ API
src-cpp/rdkafkacpp.h - C++ Bindings
src-cpp/rdkafkacpp_int.h
Messaging Clients
- Producer
src/rdkafka.c - Consumer
src/rdkafka_cgrp.c - Share Consumer
src/rdkafka_share_acknowledgement.c - Admin Client
src/rdkafka_admin.c - Transactions
src/rdkafka_txnmgr.c
Kafka Runtime
- Client Core
src/rdkafka.c - Broker Protocol
src/rdkafka_broker.c - Metadata
src/rdkafka_metadata.c - Fetch and Queues
src/rdkafka_fetcher.c
Cross-Cutting Features
- SASL OAuth
src/rdkafka_sasl_oauthbearer.c - OIDC Token
src/rdkafka_sasl_oauthbearer_oidc.c - Compression
src/rdkafka_lz4.c - Telemetry
src/rdkafka_telemetry.c - Protobuf Decode
src/nanopb/pb_decode.c
Other components
- Application
- Kafka Cluster
Connections 24
- Application → C API: uses
- Application → C++ API: uses
- C++ Bindings → C API: calls C API
src-cpp/rdkafkacpp_int.h - C API → Client Core: dispatches to
- C++ API → C++ Bindings: implemented by
src-cpp/rdkafkacpp_int.h - Client Core → Producer: provides
- Client Core → Consumer: provides
- Client Core → Share Consumer: provides
- Client Core → Admin Client: provides
- Client Core → Transactions: supports
- Producer → Kafka Cluster: produces messages
- Consumer → Kafka Cluster: consumes messages
- Admin Client → Kafka Cluster: administers
- Share Consumer → Kafka Cluster: consumes shares
- Transactions → Kafka Cluster: supports EOS
- Client Core → Broker Protocol: uses protocol
- Broker Protocol → Kafka Cluster: exchanges requests
- Consumer → Fetch and Queues: fetches via
- Consumer → Metadata: uses metadata
- SASL OAuth → OIDC Token: references OIDC
src/rdkafka_sasl_oauthbearer.c - SASL OAuth → Kafka Cluster: authenticates
- Compression → Kafka Cluster: compresses messages
- Client Core → Telemetry: supports metrics
- Telemetry → Protobuf Decode: decodes telemetry