How jamestrichmond/small-cheap-experiments works
This is a refund-only runtime telemetry spike: it probes an agent before a refund, compares its stated intent with the action, and appends a hash-chained replay record. The agent remains responsible for invoking the refund tool; the runtime’s pre-tool hook returns a decision, but the README emphasizes logging rather than enforcing a hard block. The graph covers the sampled hook-to-storage flow and documented replay purpose; it does not imply a dashboard or external hash backup, and the Diet bridge is only a stub.
Main entry points: hooks.py Refund hooks · store.py Replay store · models.py Runtime types · probes.py Probe sampler
Components 12
Probe and decision
- Probe sampler
src/probe_runtime/probes.py - Held-out probes
probes/holdout - Refund hooks
src/probe_runtime/hooks.py - Decision gate
src/probe_runtime/gate.py - Runtime types
src/probe_runtime/models.py
Scoring and replay
- Behavior scoring
src/probe_runtime/score.py - Diet escalation stub
src/probe_runtime/diet_bridge.py - Replay store
src/probe_runtime/store.py - Hash-chain integrity
src/probe_runtime/hash_chain.py
External participants
- AI agent
- Refund tool
- Risk analyst
Connections 14
- AI agent → Refund hooks: requests pre-check
src/probe_runtime/hooks.py - Refund hooks → Probe sampler: samples probe
src/probe_runtime/hooks.py - Probe sampler → Held-out probes: loads probes
src/probe_runtime/probes.py - Refund hooks → Decision gate: gets decision
src/probe_runtime/hooks.py - Refund hooks → Runtime types: parses response
src/probe_runtime/hooks.py - Decision gate → Runtime types: reads parsed type
src/probe_runtime/gate.py - Refund hooks → AI agent: returns probe and decision
src/probe_runtime/hooks.py - AI agent → Refund tool: invokes refund
- AI agent → Refund hooks: reports tool result
src/probe_runtime/hooks.py - Refund hooks → Behavior scoring: scores frames
src/probe_runtime/hooks.py - Behavior scoring → Runtime types: uses parsed types
src/probe_runtime/score.py - Refund hooks → Replay store: appends record
src/probe_runtime/hooks.py - Replay store → Hash-chain integrity: seals and verifies
src/probe_runtime/store.py - Risk analyst → Replay store: reviews replay