How sabertooth-123/llm_instrumentation_tracing_replay works
Transformer Inspector loads Hugging Face causal language models, repeatedly runs inference, and captures layer-level activity for interactive inspection. The main path is CLI configuration → model loading and hook registration → inference and analysis → bounded snapshot storage → terminal or browser views. Analysis includes activation metrics and anomalies, attention, residual-stream and token-prediction features, and optional head ablation; the web mode also supports comparing two models. The README documents these modes, but the web page source and some runtime code were not sampled, so uncertain frontend and inference-call details are omitted.
Main entry points: app.py Terminal app · server.py Web server and API · main.py CLI entry point · interceptor.py Forward-hook interceptor
Components 17
Startup and models
- CLI entry point
transformer_inspector/main.py - Model loader
transformer_inspector/model/loader.py - Causal language model
Capture and analysis
- Forward-hook interceptor
transformer_inspector/hooks/interceptor.py - Metric calculations
transformer_inspector/hooks/metrics.py - Snapshot ring buffer
transformer_inspector/buffer/ring_buffer.py
Web dashboard
- Web server and API
transformer_inspector/web/server.py - Dashboard page
transformer_inspector/web/static/index.html
Terminal dashboard
- Terminal app
transformer_inspector/tui/app.py - Layer topology
transformer_inspector/tui/widgets/topology.py - Snapshot stream
transformer_inspector/tui/widgets/stream.py - Attention view
transformer_inspector/tui/widgets/attention.py - Layer metrics view
transformer_inspector/tui/widgets/metrics_panel.py - Anomaly ledger
transformer_inspector/tui/widgets/anomaly.py
Other components
- Operator
- Browser user
- Hugging Face models
Connections 21
- Operator → CLI entry point: starts with options
- CLI entry point → Model loader: loads model
- Model loader → Hugging Face models: fetches model
- Model loader → Causal language model: returns model
- CLI entry point → Forward-hook interceptor: registers hooks
- Forward-hook interceptor → Causal language model: hooks forward pass
- Forward-hook interceptor → Metric calculations: computes statistics
- Forward-hook interceptor → Snapshot ring buffer: stores snapshots
- CLI entry point → Web server and API: starts web mode
- CLI entry point → Terminal app: starts terminal mode
- Terminal app → Snapshot ring buffer: reads snapshots
- Terminal app → Layer topology: renders topology
- Terminal app → Snapshot stream: refreshes events
- Terminal app → Attention view: loads attention
- Terminal app → Layer metrics view: loads metrics
- Terminal app → Anomaly ledger: adds warnings
- Web server and API → Snapshot ring buffer: reads snapshots
- Web server and API → Forward-hook interceptor: reads analysis state
- Web server and API → Dashboard page: serves page
- Browser user → Web server and API: requests updates
- Web server and API → Browser user: streams dashboard data