How shchelchkov/r-function works
r-function is a configurable Kafka message-processing service. Its main workflow consumes Kafka work, dispatches it through a pipeline of configured processors, runs user logic in Wasm or plugins, and publishes results back to Kafka. Git-backed settings describe catalogs, consumers, functions, function values, and streams; the processor also has read/write access to runtime values. The service exposes HTTP and WebSocket routes for settings, values, state, and refresh operations. The README documents the major runtime branches; sampled source confirms pipeline dispatch, API routing, outbound send dispatch, and example function behavior, but not all wiring between subsystems.
Main entry points: setting_store.rs Git settings · mod.rs Runtime host · routes.rs HTTP API · worker.rs Pipeline workers
Components 22
Message ingress
- Kafka consumer
r-consumer/src/kafka/consumer.rs - Pipeline
src/pipeline.rs - Work dispatcher
src/pipeline/dispatcher.rs - Pipeline workers
src/pipeline/worker.rs
Processing runtime
- Process engine
r-process/src/process/processor.rs - Process chain
r-process/src/process/chain.rs - Wasm runtime
r-runtime/src/runtime/wasm.rs - Runtime host
r-runtime/src/runtime/host/mod.rs - Plugin runtime
r-plugin/src/plugin/plugin.rs - Plugin host
r-plugin/src/host/mod.rs
Settings and state
- Git settings
r-setting/src/git/setting_store.rs - Pipeline settings
r-setting/src/catalogs/catalog.rs - Function settings
r-setting/src/functions/function_setting.rs - Stream settings
r-setting/src/streams/stream_setting.rs - Function values
r-setting/src/functions/functions_value.rs - Runtime values
r-db/src/db/db.rs
Message egress
- Kafka producer
r-producer/src/kafka/producer.rs - Send pipeline
r-producer/src/host/send_pipeline.rs
Service management
- HTTP API
r-router/src/route/routes.rs - Service bootstrap
src/bootstrap.rs
Other components
- Kafka
- API client
Connections 25
- Kafka → Kafka consumer: delivers messages
- Kafka consumer → Pipeline: submits work
- Pipeline → Work dispatcher: starts dispatcher
- Work dispatcher → Pipeline workers: dispatches work
- Pipeline workers → Process engine: processes work
- Process engine → Process chain: runs chain
- Process chain → Wasm runtime: executes Wasm
- Process chain → Plugin runtime: executes plugin
- Wasm runtime → Runtime host: calls host
- Plugin runtime → Plugin host: calls host
- Git settings → Pipeline settings: provides settings
- Git settings → Function settings: provides settings
- Git settings → Stream settings: provides settings
- Git settings → Function values: provides values
- Runtime host → Function settings: reads settings
- Runtime host → Function values: reads values
- Runtime host → Runtime values: reads and writes
- Plugin host → Runtime values: reads and writes
- Pipeline workers → Send pipeline: submits sends
- Send pipeline → Kafka producer: enqueues messages
- Kafka producer → Kafka: publishes messages
- API client → HTTP API: requests routes
- Service bootstrap → HTTP API: serves routes
- HTTP API → Git settings: refreshes settings
- HTTP API → Runtime values: reads and writes