How jamesolaitan/realtime-observability-monitor works
rtobs is a low-overhead real-time observability monitor for QEC-shaped and robotics control workloads. Its principal workflow generates scheduled input, runs a decision workload, records bounded telemetry, exports observations, and receives them for delay and symptom checks; diagnostics can capture target-side process state. The README also describes in-process SPSC and cross-process shared-memory handoffs, raw/delta encoding, and UDP batching. Experiment and analysis tools compare configurations and report results. Some implementation details are documented but unsampled, so those subsystem links are shown only where the README or excerpts support them.
Main entry points: pipeline.cpp Decision pipeline · rtobs_runner.cpp Runtime runner · run_experiment.py Experiment orchestration · collector.hpp Telemetry collector
Components 19
Control loop
- Runtime runner
apps/rtobs_runner.cpp - Input source
apps/rtobs_runner.cpp - Decision pipeline
src/pipeline.cpp - QEC workload
workloads/qec_like.cpp - PID controller
workloads/pid_controller.cpp - Output sink
apps/rtobs_runner.cpp
Telemetry path
- SPSC handoff
include/rtobs/ring_buffer.hpp - Shared memory
src/shared_memory.cpp - Telemetry collector
include/rtobs/collector.hpp - SHM collector
apps/rtobs_shm_collector.cpp - Batch encoding
src/batch_codec.cpp - UDP transport
src/udp_transport.cpp
Detection and diagnosis
- Telemetry receiver
apps/rtobs_receiver.cpp - Diagnostic capture
src/diagnostics.cpp - Diagnostic service
apps/rtobs_diag.cpp
Evaluation
- Experiment orchestration
experiments/run_experiment.py - Fault qualification
experiments/run_fault_qualification.py - Result analysis
analysis/analyze.py
Other components
- Experiment operator
Connections 26
- Experiment operator → Runtime runner: starts run
- Runtime runner → Input source: configures source
- Runtime runner → Decision pipeline: runs pipeline
- Runtime runner → Telemetry collector: starts collector
- Input source → Decision pipeline: supplies frames
- Decision pipeline → QEC workload: runs workload
- Decision pipeline → PID controller: runs workload
- Decision pipeline → Output sink: sends output
- Decision pipeline → SPSC handoff: records telemetry
- Decision pipeline → Shared memory: writes records
- Telemetry collector → SPSC handoff: reads records
- Telemetry collector → Batch encoding: encodes batches
- Telemetry collector → UDP transport: sends datagrams
- SHM collector → Shared memory: reads records
- SHM collector → Batch encoding: encodes batches
- SHM collector → UDP transport: sends datagrams
- UDP transport → Telemetry receiver: delivers datagrams
- Telemetry receiver → Diagnostic capture: requests capture
- Diagnostic service → Diagnostic capture: captures state
- Experiment orchestration → Runtime runner: runs configurations
- Experiment orchestration → Telemetry receiver: starts receiver
- Experiment orchestration → SHM collector: starts collector
- Experiment orchestration → Result analysis: analyzes results
- Fault qualification → Runtime runner: injects symptoms
- Fault qualification → Telemetry receiver: checks detection
- Fault qualification → Diagnostic service: checks capture