# giskard-ai/giskard architecture

> Architecture diagram of the GitHub repository giskard-ai/giskard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/giskard-ai/giskard
- Repository: https://github.com/giskard-ai/giskard

## Overview

#### Purpose and package layout
Giskard v3 is a Python 3.12+ monorepo for evaluating, red-teaming, and generating tests for LLMs, agents, RAG systems, and arbitrary sync/async targets. The root distribution exposes focused namespace packages under `giskard.*`; `giskard_pypi_shim/__init__.py` supports the aggregate PyPI package. Core user-facing APIs are `giskard.checks` and optional `giskard.scan`.

#### Evaluation execution: Checks
`giskard-checks` is the primary evaluation runtime. A caller builds a `Scenario` of target interactions and checks, runs it asynchronously, and receives structured results/traces; `Suite` batches scenarios.
- `libs/giskard-checks/src/giskard/checks/core/scenario.py`
- `libs/giskard-checks/src/giskard/checks/scenarios/runner.py`
- `libs/giskard-checks/src/giskard/checks/scenarios/suite.py`

The target boundary is deliberately generic: user-provided sync/async callables consume inputs and return outputs, optionally with trace data. Interaction/trace abstractions normalize those executions before checks inspect them.
- `libs/giskard-checks/src/giskard/checks/core/interaction/interaction.py`
- `libs/giskard-checks/src/giskard/checks/core/interaction/trace.py`

#### Checks, judges, and reporting
Checks implement assertions over extracted interaction data, from deterministic built-ins to LLM-based judges. Result, extraction, testcase, and input-generation contracts isolate check authors from target invocation details.
- `libs/giskard-checks/src/giskard/checks/core/check.py`
- `libs/giskard-checks/src/giskard/checks/core/result.py`
- `libs/giskard-checks/src/giskard/checks/core/extraction.py`

Built-ins include composition/comparison, JSON/text/semantic checks, Rego policy, and LLM judges for groundedness, relevance, conformity, contradiction, and toxicity. Results export to JUnit or Hub-oriented formats.
- `libs/giskard-checks/src/giskard/checks/builtin/composition.py`
- `libs/giskard-checks/src/giskard/checks/judges/judge.py`
- `libs/giskard-checks/src/giskard/checks/export/junit.py`

#### Scan: automated red teaming and quality evaluation
`giskard-scan` composes generated scenarios into vulnerability and knowledge-base quality scans. Its public scan orchestration, catalog/registry, vulnerability types, and quality paths are the high-level entry boundary.
- `libs/giskard-scan/src/giskard/scan/__init__.py`
- `libs/giskard-scan/src/giskard/scan/vulnerability.py`
- `libs/giskard-scan/src/giskard/scan/quality.py`

Generators create adversarial, prompt-injection, jailbreak, and RAG-oriented test inputs, often using prompt templates and LLM generation. Generated suites flow into the Checks scenario runtime rather than requiring a separate target protocol.
- `libs/giskard-scan/src/giskard/scan/generators/base.py`
- `libs/giskard-scan/src/giskard/scan/generators/adversarial.py`
- `libs/giskard-scan/src/giskard/scan/generators/knowledge_base/base.py`

#### LLM and workflow substrate
`giskard-llm` provides provider-neutral chat/response/embedding types, routing, retries, and provider-specific payload translators. It integrates with OpenAI, Anthropic, Google, Azure OpenAI, and Azure AI SDKs, selected through optional dependencies and credentials.
- `libs/giskard-llm/src/giskard/llm/routing.py`
- `libs/giskard-llm/src/giskard/llm/providers/base.py`
- `libs/giskard-llm/src/giskard/llm/translators/`

`giskard-agents` adds generator adapters, prompt templating, tools, and workflow/pipeline orchestration used by LLM judges and scan generators.
- `libs/giskard-agents/src/giskard/agents/generators/base.py`
- `libs/giskard-agents/src/giskard/agents/workflow.py`
- `libs/giskard-agents/src/giskard/agents/templates/prompts_manager.py`

#### Shared cross-cutting services and integrations
`giskard-core` centralizes shared error/discriminated-type utilities, rate limiting, and opt-out telemetry. Telemetry is aggregated and controlled before import via environment variables.
- `libs/giskard-core/src/giskard/core/telemetry/telemetry.py`
- `libs/giskard-core/src/giskard/core/rate_limiter/base.py`

Scan also adapts external red-team ecosystems—DeepTeam, Garak, and LIDAR—behind integration bridges/adapters, preserving Giskard’s scenario/check model as the internal boundary.
- `libs/giskard-scan/src/giskard/scan/integrations/deepteam/_bridge.py`
- `libs/giskard-scan/src/giskard/scan/integrations/garak/_bridge.py`
- `libs/giskard-scan/src/giskard/scan/integrations/lidar/_adapter.py`

Main entry points: [`libs/giskard-checks/src/giskard/checks/core/scenario.py`](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/scenario.py) (Scenario), [`libs/giskard-scan/src/giskard/scan/__init__.py`](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/__init__.py) (Scan API and catalog), [`libs/giskard-checks/src/giskard/checks/scenarios/runner.py`](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/scenarios/runner.py) (Scenario runner), [`libs/giskard-checks/src/giskard/checks/core/check.py`](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/check.py) (Check contracts)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_checks["Checks runtime"]
  node_scenario["Scenario<br/>evaluation API<br/>[scenario.py]"]
  node_suite["Suite<br/>scenario batch<br/>[suite.py]"]
  node_runner["Scenario runner<br/>async executor<br/>[runner.py]"]
  node_interaction["Interaction model<br/>execution contract<br/>[interaction.py]"]
  node_trace["Trace model<br/>trace contract<br/>[trace.py]"]
  node_check_contract["Check contracts<br/>check API<br/>[check.py]"]
  node_results["Results and extraction<br/>result contracts<br/>[result.py]"]
  node_builtin_checks["Built-in checks<br/>assertion library<br/>[composition.py]"]
  node_llm_judges["LLM judges<br/>judge checks<br/>[judge.py]"]
  node_junit_export["JUnit export<br/>report exporter<br/>[junit.py]"]
end

subgraph group_scan["Scan"]
  node_scan_api["Scan API and catalog<br/>scan orchestration<br/>[__init__.py]"]
  node_scan_generators["Scan generators<br/>adversarial generation<br/>[adversarial.py]"]
  node_kb_generators["Knowledge-base generators<br/>RAG generation<br/>[base.py]"]
  node_external_bridges["External red-team bridges<br/>integration adapters<br/>[_bridge.py]"]
end

subgraph group_llm["LLM workflow substrate"]
  node_agent_workflows["Agent workflows<br/>workflow orchestration<br/>[workflow.py]"]
  node_prompt_templates["Prompt templates<br/>template manager<br/>[prompts_manager.py]"]
  node_llm_routing["LLM routing<br/>provider router<br/>[routing.py]"]
  node_provider_adapters{{"Provider adapters<br/>LLM provider boundary<br/>[base.py]"}}
end

subgraph group_shared["Shared services"]
  node_telemetry["Telemetry<br/>[telemetry.py]"]
  node_rate_limiter["Rate limiter<br/>shared control<br/>[base.py]"]
end

node_user_target{{"User target<br/>sync/async callable"}}

node_suite -->|"batches"| node_scenario
node_scenario -->|"runs through"| node_runner
node_runner -->|"invokes"| node_user_target
node_runner -->|"normalizes"| node_interaction
node_interaction -.->|"includes optional"| node_trace
node_runner -->|"evaluates"| node_check_contract
node_check_contract -->|"implemented by"| node_builtin_checks
node_check_contract -->|"implemented by"| node_llm_judges
node_check_contract -->|"produces"| node_results
node_results -->|"exports as"| node_junit_export
node_scan_api -->|"orchestrates"| node_scan_generators
node_scan_api -->|"orchestrates"| node_kb_generators
node_scan_api -->|"uses"| node_external_bridges
node_scan_generators -->|"generates"| node_scenario
node_kb_generators -->|"generates"| node_scenario
node_external_bridges -->|"adapts into"| node_scenario
node_scan_generators -->|"uses"| node_agent_workflows
node_llm_judges -->|"uses"| node_agent_workflows
node_agent_workflows -->|"renders"| node_prompt_templates
node_agent_workflows -->|"requests LLMs through"| node_llm_routing
node_llm_routing -->|"dispatches to"| node_provider_adapters
node_llm_routing -.->|"is constrained by"| node_rate_limiter
node_scan_api -.->|"reports via"| node_telemetry

click node_scenario "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/scenario.py"
click node_suite "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/scenarios/suite.py"
click node_runner "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/scenarios/runner.py"
click node_interaction "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/interaction/interaction.py"
click node_trace "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/interaction/trace.py"
click node_check_contract "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/check.py"
click node_results "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/result.py"
click node_builtin_checks "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/builtin/composition.py"
click node_llm_judges "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/judges/judge.py"
click node_junit_export "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/export/junit.py"
click node_scan_api "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/__init__.py"
click node_scan_generators "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/generators/adversarial.py"
click node_kb_generators "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/generators/knowledge_base/base.py"
click node_external_bridges "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/integrations/deepteam/_bridge.py"
click node_agent_workflows "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-agents/src/giskard/agents/workflow.py"
click node_prompt_templates "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-agents/src/giskard/agents/templates/prompts_manager.py"
click node_llm_routing "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-llm/src/giskard/llm/routing.py"
click node_provider_adapters "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-llm/src/giskard/llm/providers/base.py"
click node_telemetry "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-core/src/giskard/core/telemetry/telemetry.py"
click node_rate_limiter "https://github.com/giskard-ai/giskard/blob/main/libs/giskard-core/src/giskard/core/rate_limiter/base.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scenario,node_suite,node_runner,node_interaction,node_trace,node_check_contract,node_results,node_builtin_checks,node_llm_judges,node_junit_export toneBlue
class node_scan_api,node_scan_generators,node_kb_generators,node_external_bridges toneAmber
class node_agent_workflows,node_prompt_templates,node_llm_routing,node_provider_adapters toneMint
class node_telemetry,node_rate_limiter toneRose
class node_user_target toneNeutral
```

## Components

### Checks runtime

Async scenario evaluation and reporting.

- [Scenario](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/scenario.py): evaluation API · Defines target interactions and checks. · `libs/giskard-checks/src/giskard/checks/core/scenario.py`
- [Suite](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/scenarios/suite.py): scenario batch · Batches scenario execution. · `libs/giskard-checks/src/giskard/checks/scenarios/suite.py`
- [Scenario runner](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/scenarios/runner.py): async executor · Invokes targets and coordinates checks. · `libs/giskard-checks/src/giskard/checks/scenarios/runner.py`
- [Interaction model](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/interaction/interaction.py): execution contract · Normalizes target inputs and outputs. · `libs/giskard-checks/src/giskard/checks/core/interaction/interaction.py`
- [Trace model](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/interaction/trace.py): trace contract · Captures optional execution trace data. · `libs/giskard-checks/src/giskard/checks/core/interaction/trace.py`
- [Check contracts](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/check.py): check API · Isolates check logic from target invocation. · `libs/giskard-checks/src/giskard/checks/core/check.py`
- [Results and extraction](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/core/result.py): result contracts · Structured outcomes and extracted interaction data. · `libs/giskard-checks/src/giskard/checks/core/result.py`
- [Built-in checks](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/builtin/composition.py): assertion library · Composition, text, JSON, semantic, and policy checks. · `libs/giskard-checks/src/giskard/checks/builtin/composition.py`
- [LLM judges](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/judges/judge.py): judge checks · Evaluates groundedness, relevance, safety, and conformity. · `libs/giskard-checks/src/giskard/checks/judges/judge.py`
- [JUnit export](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-checks/src/giskard/checks/export/junit.py): report exporter · Exports evaluation results. · `libs/giskard-checks/src/giskard/checks/export/junit.py`

### Scan

Automated red teaming and quality scans.

- [Scan API and catalog](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/__init__.py): scan orchestration · Entry point for vulnerability and quality scans. · `libs/giskard-scan/src/giskard/scan/__init__.py`
- [Scan generators](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/generators/adversarial.py): adversarial generation · Builds adversarial and prompt-injection scenarios. · `libs/giskard-scan/src/giskard/scan/generators/adversarial.py`
- [Knowledge-base generators](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/generators/knowledge_base/base.py): RAG generation · Creates RAG-oriented quality test inputs. · `libs/giskard-scan/src/giskard/scan/generators/knowledge_base/base.py`
- [External red-team bridges](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-scan/src/giskard/scan/integrations/deepteam/_bridge.py): integration adapters · Adapts DeepTeam, Garak, and LIDAR to internal scenarios. · `libs/giskard-scan/src/giskard/scan/integrations/deepteam/_bridge.py`

### LLM workflow substrate

Provider-neutral LLM execution and agent workflows.

- [Agent workflows](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-agents/src/giskard/agents/workflow.py): workflow orchestration · Composes generators, tools, and pipelines. · `libs/giskard-agents/src/giskard/agents/workflow.py`
- [Prompt templates](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-agents/src/giskard/agents/templates/prompts_manager.py): template manager · Manages prompts used by generators and judges. · `libs/giskard-agents/src/giskard/agents/templates/prompts_manager.py`
- [LLM routing](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-llm/src/giskard/llm/routing.py): provider router · Routes requests with retries to configured providers. · `libs/giskard-llm/src/giskard/llm/routing.py`
- [Provider adapters](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-llm/src/giskard/llm/providers/base.py): LLM provider boundary · Translates neutral requests to provider payloads. · `libs/giskard-llm/src/giskard/llm/providers/base.py`

### Shared services

Cross-cutting runtime controls.

- [Telemetry](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-core/src/giskard/core/telemetry/telemetry.py): opt-out telemetry · Environment-controlled aggregated telemetry. · `libs/giskard-core/src/giskard/core/telemetry/telemetry.py`
- [Rate limiter](https://github.com/giskard-ai/giskard/blob/main/libs/giskard-core/src/giskard/core/rate_limiter/base.py): shared control · Coordinates request rate limits. · `libs/giskard-core/src/giskard/core/rate_limiter/base.py`

### Other components

- **User target**: sync/async callable · Consumes inputs and may return trace data.

## Connections

- Suite → Scenario: batches
- Scenario → Scenario runner: runs through
- Scenario runner → User target: invokes
- Scenario runner → Interaction model: normalizes
- Interaction model → Trace model: includes optional
- Scenario runner → Check contracts: evaluates
- Check contracts → Built-in checks: implemented by
- Check contracts → LLM judges: implemented by
- Check contracts → Results and extraction: produces
- Results and extraction → JUnit export: exports as
- Scan API and catalog → Scan generators: orchestrates
- Scan API and catalog → Knowledge-base generators: orchestrates
- Scan API and catalog → External red-team bridges: uses
- Scan generators → Scenario: generates
- Knowledge-base generators → Scenario: generates
- External red-team bridges → Scenario: adapts into
- Scan generators → Agent workflows: uses
- LLM judges → Agent workflows: uses
- Agent workflows → Prompt templates: renders
- Agent workflows → LLM routing: requests LLMs through
- LLM routing → Provider adapters: dispatches to
- LLM routing → Rate limiter: is constrained by
- Scan API and catalog → Telemetry: reports via
