# colbyrobbins/xai-project-2 architecture

> Architecture diagram of the GitHub repository colbyrobbins/xai-project-2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/colbyrobbins/xai-project-2
- Repository: https://github.com/colbyrobbins/xai-project-2

## Overview

VLM-SteerHarness is runtime middleware for multimodal model inference with latent-risk monitoring, conditional activation steering, and governance traces. The principal workflow accepts an image/text request, runs a diagnostic model pass, probes captured activations, applies any planned intervention during generation, and returns output plus an audit record. Separate branches support interactive control, concept-vector calibration, and quantitative evaluation. README lifecycle claims fill some gaps; source excerpts are partial, so uncertain implementation links are omitted or dashed.

Main entry points: [`src/harness/core.py`](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/core.py) (Harness Runtime), [`src/steering/calibration.py`](https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/calibration.py) (Calibration Pipeline), [`src/harness/gatekeeper.py`](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/gatekeeper.py) (Decision Gatekeeper), [`src/eval/benchmark.py`](https://github.com/colbyrobbins/xai-project-2/blob/main/src/eval/benchmark.py) (Benchmark Logic)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Inference Runtime"]
  node_harness["Harness Runtime<br/>[core.py]"]
  node_model["VLM Wrapper<br/>[wrapper.py]"]
  node_config["Runtime Config<br/>[config.py]"]
  node_hooks["Activation Hooks<br/>[hook.py]"]
end

subgraph group_xai["Observability &amp; Policy"]
  node_gatekeeper["Decision Gatekeeper<br/>[gatekeeper.py]"]
  node_probes["Latent Probes<br/>[probes.py]"]
  node_neurons["Neuron Profiler<br/>[neurons.py]"]
  node_telemetry["Audit Telemetry<br/>[telemetry.py]"]
end

subgraph group_steering["Concept Steering"]
  node_vectors["Concept Vectors<br/>[vectors.py]"]
  node_extractor["Vector Extractor<br/>[extractor.py]"]
  node_calibration["Calibration Pipeline<br/>[calibration.py]"]
  node_hookdata["Calibration Pairs"]
  node_storedvectors["Calibrated Vectors"]
end

subgraph group_product["Control &amp; Evaluation"]
  node_dashboard["Control Center<br/>[app.py]"]
  node_benchrunner["Benchmark Runner<br/>[run_benchmarks.py]"]
  node_benchmark["Benchmark Logic<br/>[benchmark.py]"]
  node_metrics["Evaluation Metrics<br/>[metrics.py]"]
  node_results["Benchmark Results"]
end

node_user(("Request User"))

node_user -->|"submits request"| node_harness
node_harness -->|"uses settings"| node_config
node_harness -->|"runs inference"| node_model
node_harness -->|"registers hooks"| node_hooks
node_harness -->|"requests decisions"| node_gatekeeper
node_gatekeeper -->|"evaluates activations"| node_probes
node_gatekeeper -->|"retrieves concepts"| node_vectors
node_hooks -->|"injects direction"| node_vectors
node_harness -->|"profiles circuits"| node_neurons
node_harness -->|"records trace"| node_telemetry
node_probes -->|"produces probe records"| node_telemetry
node_neurons -->|"produces attributions"| node_telemetry
node_calibration -->|"uses algorithms"| node_extractor
node_calibration -->|"creates vectors"| node_vectors
node_calibration -->|"extracts activations"| node_model
node_hookdata -.->|"calibrates from pairs"| node_calibration
node_storedvectors -.->|"registers concepts"| node_harness
node_user -.->|"operates controls"| node_dashboard
node_dashboard -.->|"submits inference"| node_harness
node_benchrunner -->|"runs benchmarks"| node_benchmark
node_benchmark -.->|"computes measures"| node_metrics
node_benchmark -.->|"records results"| node_results

click node_harness "https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/core.py"
click node_model "https://github.com/colbyrobbins/xai-project-2/blob/main/src/models/wrapper.py"
click node_config "https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/config.py"
click node_hooks "https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/hook.py"
click node_gatekeeper "https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/gatekeeper.py"
click node_probes "https://github.com/colbyrobbins/xai-project-2/blob/main/src/attribution/probes.py"
click node_neurons "https://github.com/colbyrobbins/xai-project-2/blob/main/src/attribution/neurons.py"
click node_telemetry "https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/telemetry.py"
click node_vectors "https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/vectors.py"
click node_extractor "https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/extractor.py"
click node_calibration "https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/calibration.py"
click node_hookdata "https://github.com/colbyrobbins/xai-project-2/tree/main/data/calibration"
click node_storedvectors "https://github.com/colbyrobbins/xai-project-2/tree/main/data/concepts"
click node_dashboard "https://github.com/colbyrobbins/xai-project-2/blob/main/app/app.py"
click node_benchrunner "https://github.com/colbyrobbins/xai-project-2/blob/main/eval/run_benchmarks.py"
click node_benchmark "https://github.com/colbyrobbins/xai-project-2/blob/main/src/eval/benchmark.py"
click node_metrics "https://github.com/colbyrobbins/xai-project-2/blob/main/src/eval/metrics.py"
click node_results "https://github.com/colbyrobbins/xai-project-2/blob/main/eval/benchmark_results.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_harness,node_model,node_config,node_hooks,node_user toneBlue
class node_gatekeeper,node_probes,node_neurons,node_telemetry toneAmber
class node_vectors,node_extractor,node_calibration,node_hookdata,node_storedvectors toneMint
class node_dashboard,node_benchrunner,node_benchmark,node_metrics,node_results toneRose
```

## Components

### Inference Runtime

- [Harness Runtime](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/core.py): `src/harness/core.py`
- [VLM Wrapper](https://github.com/colbyrobbins/xai-project-2/blob/main/src/models/wrapper.py): `src/models/wrapper.py`
- [Runtime Config](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/config.py): `src/harness/config.py`
- [Activation Hooks](https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/hook.py): `src/steering/hook.py`

### Observability & Policy

- [Decision Gatekeeper](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/gatekeeper.py): `src/harness/gatekeeper.py`
- [Latent Probes](https://github.com/colbyrobbins/xai-project-2/blob/main/src/attribution/probes.py): `src/attribution/probes.py`
- [Neuron Profiler](https://github.com/colbyrobbins/xai-project-2/blob/main/src/attribution/neurons.py): `src/attribution/neurons.py`
- [Audit Telemetry](https://github.com/colbyrobbins/xai-project-2/blob/main/src/harness/telemetry.py): `src/harness/telemetry.py`

### Concept Steering

- [Concept Vectors](https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/vectors.py): `src/steering/vectors.py`
- [Vector Extractor](https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/extractor.py): `src/steering/extractor.py`
- [Calibration Pipeline](https://github.com/colbyrobbins/xai-project-2/blob/main/src/steering/calibration.py): `src/steering/calibration.py`
- [Calibration Pairs](https://github.com/colbyrobbins/xai-project-2/tree/main/data/calibration): `data/calibration`
- [Calibrated Vectors](https://github.com/colbyrobbins/xai-project-2/tree/main/data/concepts): `data/concepts`

### Control & Evaluation

- [Control Center](https://github.com/colbyrobbins/xai-project-2/blob/main/app/app.py): `app/app.py`
- [Benchmark Runner](https://github.com/colbyrobbins/xai-project-2/blob/main/eval/run_benchmarks.py): `eval/run_benchmarks.py`
- [Benchmark Logic](https://github.com/colbyrobbins/xai-project-2/blob/main/src/eval/benchmark.py): `src/eval/benchmark.py`
- [Evaluation Metrics](https://github.com/colbyrobbins/xai-project-2/blob/main/src/eval/metrics.py): `src/eval/metrics.py`
- [Benchmark Results](https://github.com/colbyrobbins/xai-project-2/blob/main/eval/benchmark_results.json): `eval/benchmark_results.json`

### Other components

- **Request User**

## Connections

- Request User → Harness Runtime: submits request (evidence: `README.md`)
- Harness Runtime → Runtime Config: uses settings (evidence: `src/harness/core.py`)
- Harness Runtime → VLM Wrapper: runs inference (evidence: `src/harness/core.py`)
- Harness Runtime → Activation Hooks: registers hooks (evidence: `src/harness/core.py`)
- Harness Runtime → Decision Gatekeeper: requests decisions (evidence: `src/harness/core.py`)
- Decision Gatekeeper → Latent Probes: evaluates activations (evidence: `src/harness/gatekeeper.py`)
- Decision Gatekeeper → Concept Vectors: retrieves concepts (evidence: `src/harness/gatekeeper.py`)
- Activation Hooks → Concept Vectors: injects direction (evidence: `src/steering/hook.py`)
- Harness Runtime → Neuron Profiler: profiles circuits (evidence: `src/harness/core.py`)
- Harness Runtime → Audit Telemetry: records trace (evidence: `README.md`)
- Latent Probes → Audit Telemetry: produces probe records (evidence: `src/attribution/probes.py`)
- Neuron Profiler → Audit Telemetry: produces attributions (evidence: `src/attribution/neurons.py`)
- Calibration Pipeline → Vector Extractor: uses algorithms (evidence: `src/steering/calibration.py`)
- Calibration Pipeline → Concept Vectors: creates vectors (evidence: `src/steering/calibration.py`)
- Calibration Pipeline → VLM Wrapper: extracts activations (evidence: `src/steering/calibration.py`)
- Calibration Pairs → Calibration Pipeline: calibrates from pairs
- Calibrated Vectors → Harness Runtime: registers concepts
- Request User → Control Center: operates controls
- Control Center → Harness Runtime: submits inference
- Benchmark Runner → Benchmark Logic: runs benchmarks (evidence: `eval/run_benchmarks.py`)
- Benchmark Logic → Evaluation Metrics: computes measures
- Benchmark Logic → Benchmark Results: records results