How safety-research/circuit-tracer works
High-level breakdown of the file structure
• Top-level:
• pyproject.toml, README.md, LICENSE, .gitignore
• circuit_tracer/ (core package)
• demos/ (Jupyter notebooks + helper scripts for tutorials)
• tests/ (unit tests for attribution, graph-creation, utils)
• circuit_tracer/:
• __main__.py: CLI entrypoint (console_scripts)
• attribution.py: orchestration of the 3-step pipeline (attribution → prune → graph files)
• replacement_model.py, graph.py: core data structures and algorithms for constructing the raw and pruned graph
• configs/*.yaml: predefined transcoder sets (Gemma, Llama, custom)
• transcoder/: code for loading and applying transcoder models & activation functions
• utils/: various helpers (disk offloading, HF model loading, graph file creation)
• frontend/: web server + data models for serving static assets + JSON endpoints
– local_server.py: minimal HTTP server to serve HTML/JS/CSS and JSON graph data
– assets/: HTML, CSS, JS modules for interactive visualization (attribution graphs, feature-view, etc.)
Key insights from README
• The workflow is a “3-step” pipeline:
1. Attribution: compute direct effects between features, error nodes, tokens, logits
2. Graph file creation: prune nodes/edges, emit JSON for visualization
3. Local server: serve static frontend + JSON to browser UI
• Supports three usage modes:
• Notebook/API
• CLI only (attribute → save .pt raw graph)
• CLI + --server (attribute → JSON → launch web UI)
Main components
a. User interfaces
1. Command-Line Interface (“circuit-tracer attribute … [--server]”)
2. Python API / Notebook (demos/*.ipynb import circuit_tracer.*)
3. Browser UI (HTML/JS/CSS assets)
b. Backend services
1. CLI entrypoint (__main__.py)
2. Attribution Engine (attribution.py + transcoder modules)
3. Graph Model & Pruner (graph.py, utils/create_graph_files.py)
4. Configuration Loader (configs/*.yaml)
5. HF Model Loader (utils/hf_utils.py)
6. Local Web Server (frontend/local_server.py)
c. Data stores / file artifacts
1. Raw attribution graph (.pt)
2. Pruned JSON graph files (graph_file_dir/*.json)
3. Config YAML files
4. Static assets (frontend/assets/*.html, .js, .css)
d. External dependencies
1. Hugging Face model hub (download pretrained models & transcoders)
2. Flask or built-in HTTP server library (for local_server)
3. Browser (renders the frontend)
Relationships and data flows
• CLI / Python API → loads config & HF models → runs Attribution Engine → produces Raw Graph (.pt)
→ Graph Pruner → exports JSON files
→ (if --server) Local Server reads JSON + static assets
→ serves HTTP endpoints → Browser UI fetches index.html, JS, CSS, then graph JSON → renders interactive graphs
• In notebook mode: Python API returns in-memory graph objects for further analysis or intervention
• External → HF utilities fetch pretrained transcoders & model weights; disk_offload for large feature sets
Architectural patterns and design principles
• Three-tier separation:
1. Presentation layer (Browser UI static assets)
2. Application layer (Python backend: attribution + server)
3. Data layer (graph files, config YAML, model weights)
• Modular design: separate modules for transcoder logic, graph construction, utilities, and frontend
• Configuration-driven: YAML presets for different model/transcoder sets
Technology stack
• Python 3.8+
• PyTorch / Transformer model loading (via HF utils)
• YAML for config files
• Built-in HTTP server or Flask for local_server.py
• JavaScript, HTML, CSS (vanilla + small utility scripts) for the frontend visuals
• Jupyter notebooks & demos for interactive tutorials
Diagramming recommendations
• Config → Attribution Engine → Raw Graph → Graph Pruner → JSON Graph → Local Server → Browser
• Blue: Backend processing modules
• Green: Data/artifact stores (YAML, .pt, JSON)
• Orange: Web layer (local_server, static assets, browser)