# mizcausevic-dev/prompt-injection-bench architecture

> Architecture diagram of the GitHub repository mizcausevic-dev/prompt-injection-bench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mizcausevic-dev/prompt-injection-bench
- Repository: https://github.com/mizcausevic-dev/prompt-injection-bench

## Overview

Purpose and principal workflow: `prompt-injection-bench` is a local corpus and heuristic harness for regression-testing an AI agent against synthetic prompt-injection cases. The external operator starts the main workflow by invoking `pib run`; the target agent is exercised separately in the intended trust boundary, and the harness evaluates its responses from a complete JSONL transcript. Echo and blanket-refusal adapters are local diagnostics, not connections to a live agent. Reports are heuristic signals, not safety determinations.

Components

**Command-line and corpus**
- **`pib` CLI** — `src/pib/cli.py`: Implements `list`, `validate`, `build-jsonl`, and `run`; selects a run adapter, invokes the harness, prints text or JSON, and applies an optional pass-rate threshold.
- **Attack-case corpus** — `corpus/DEL-001.json`: A representative record in the 30-case, 10-category synthetic corpus; records carry attack prompts, expected safe behavior, scoring rules, and Agent Card refusal-category back-references.
- **Corpus loader and exporter** — `src/pib/loader.py`: Loads and validates individual corpus JSON files or flat JSONL records, checks cross-record IDs and category prefixes, and writes corpus JSONL.
- **Record and report schemas** — `src/pib/schema.py`: Defines Pydantic models and constrained fields for attack records, individual run results, and aggregate reports.
- **Attack taxonomy and scoring guidance** — `TAXONOMY.md`: Documents the attack categories, scoring methods, Agent Card cross-references, and severity rubric.

**Run and evaluation**
- **Run orchestration and adapters** — `src/pib/runner.py`: Iterates through records, obtains each response and tool-attempt boolean from an adapter, invokes scoring, aggregates category/severity rates and critical failures, and renders text reports; it also implements transcript, echo, and refusal adapters.
- **Response scorer** — `src/pib/scorer.py`: Applies the record’s selected heuristic to response text and, for tool-invocation cases, requires an observed boolean; returns a per-case result.
- **Operator** — *(external; null repository path)*: Invokes the CLI and supplies run options, including the transcript path when evaluating recorded responses.
- **Target agent** — *(external; null repository path)*: Receives the synthetic cases through the operator’s chosen trust boundary and produces responses; this repository does not call the agent directly.
- **Run transcript** — *(external input; null repository path)*: The operator-provided JSONL of responses and observed tool-attempt booleans, consumed by the transcript adapter.

Relationships
- **Operator -> `pib` CLI**: invokes `pib` with a command and options; the executable entry point is declared in `pyproject.toml` and command handling is implemented in `src/pib/cli.py`.
- **Target agent -> Run transcript**: produces responses and observed tool-attempt information during the operator’s separate test run; the README specifies this transcript format and does not establish an automated agent connection.
- **Operator -> Run transcript**: supplies a complete transcript for `--adapter transcript`; `pib` requires a response and explicit boolean for every corpus ID.
- **Attack-case corpus -> Corpus loader and exporter**: provides JSON records that the loader discovers and parses; the loader also supports flat JSONL input.
- **Corpus loader and exporter -> Record and report schemas**: validates loaded attack records using the Pydantic attack-record model.
- **`pib` CLI -> Corpus loader and exporter**: loads records for commands and uses the exporter for `build-jsonl`.
- **`pib` CLI -> Run orchestration and adapters**: selects an adapter and calls `run_corpus` for a run; echo, refusal, and transcript are alternative adapter choices, not sequential stages.
- **Run transcript -> Run orchestration and adapters**: the transcript adapter reads JSONL, rejects malformed, duplicate, unknown, or missing IDs, and looks up each response and tool-attempt boolean.
- **Run orchestration and adapters -> Response scorer**: calls `score_response` once per record and adapter response.
- **Response scorer -> Record and report schemas**: constructs a per-case result; the runner uses those results to construct an aggregate report.
- **Run orchestration and adapters -> Record and report schemas**: creates the aggregate report with target label, pass rates, critical failures, and results.
- **`pib` CLI -> Run orchestration and adapters**: renders the returned report as text or JSON and applies the optional `--fail-under` exit gate.

Coverage limits: The sampled sources establish the CLI-to-loader-to-runner-to-scorer flow and transcript behavior. The corpus and taxonomy are represented by tree/README evidence rather than sampled record contents or taxonomy source. No live-agent API, tool-trace capture mechanism, or hosted visual-harness implementation is established here; the README identifies the visual harness as a separate deployment.

Main entry points: [`src/pib/runner.py`](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/runner.py) (Run orchestration), [`src/pib/cli.py`](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/cli.py) (`pib` CLI), [`src/pib/loader.py`](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/loader.py) (Corpus loader), [`src/pib/schema.py`](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/schema.py) (Record and report schemas)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli_corpus["CLI & Corpus"]
  node_cli["&#96;pib&#96; CLI<br/>Command-line interface"]
  node_corpus["Attack-case corpus<br/>Synthetic cases"]
  node_loader["Corpus loader<br/>Loader and exporter"]
end

subgraph group_evaluation["Evaluation"]
  node_schemas["Record and report schemas<br/>Pydantic models"]
  node_runner["Run orchestration<br/>Runner and adapters"]
  node_scorer["Response scorer<br/>Heuristic scoring"]
end

subgraph group_external["External Inputs"]
  node_operator["Operator<br/>External actor"]
  node_target_agent["Target agent<br/>External system"]
  node_transcript["Run transcript<br/>JSONL input"]
end

node_operator -->|"invokes"| node_cli
node_target_agent -->|"produces"| node_transcript
node_operator -->|"supplies"| node_transcript
node_corpus -->|"provides records"| node_loader
node_loader -->|"validates records"| node_schemas
node_cli -->|"loads and exports"| node_loader
node_cli -->|"starts run"| node_runner
node_transcript -->|"provides responses"| node_runner
node_runner -->|"scores cases"| node_scorer
node_scorer -->|"creates case results"| node_schemas
node_runner -->|"creates aggregate report"| node_schemas
node_cli -->|"renders report"| node_runner

click node_cli "https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/cli.py"
click node_loader "https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/loader.py"
click node_schemas "https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/schema.py"
click node_runner "https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/runner.py"
click node_scorer "https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/scorer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_corpus,node_loader toneBlue
class node_schemas,node_runner,node_scorer toneAmber
class node_operator,node_target_agent,node_transcript toneMint
```

## Components

### CLI & Corpus

- [`pib` CLI](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/cli.py): Command-line interface · `src/pib/cli.py`
- **Attack-case corpus**: Synthetic cases
- [Corpus loader](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/loader.py): Loader and exporter · `src/pib/loader.py`

### Evaluation

- [Record and report schemas](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/schema.py): Pydantic models · `src/pib/schema.py`
- [Run orchestration](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/runner.py): Runner and adapters · `src/pib/runner.py`
- [Response scorer](https://github.com/mizcausevic-dev/prompt-injection-bench/blob/main/src/pib/scorer.py): Heuristic scoring · `src/pib/scorer.py`

### External Inputs

- **Operator**: External actor
- **Target agent**: External system · Exercised separately; this repository does not call it directly.
- **Run transcript**: JSONL input

## Connections

- Operator → `pib` CLI: invokes
- Target agent → Run transcript: produces
- Operator → Run transcript: supplies
- Attack-case corpus → Corpus loader: provides records
- Corpus loader → Record and report schemas: validates records
- `pib` CLI → Corpus loader: loads and exports
- `pib` CLI → Run orchestration: starts run
- Run transcript → Run orchestration: provides responses · The transcript adapter reads and validates the complete JSONL input.
- Run orchestration → Response scorer: scores cases
- Response scorer → Record and report schemas: creates case results
- Run orchestration → Record and report schemas: creates aggregate report
- `pib` CLI → Run orchestration: renders report · The CLI renders the returned report and applies the optional pass-rate threshold.