How nvidia/garak works
garak is an LLM vulnerability scanner: users select a target model and probes, then the toolkit generates adversarial attempts, evaluates responses with detectors, and records reports for later analysis. Its plugin/configuration machinery supports this workflow; language services and intent data add assessment capabilities. A separate React report application presents run data. The graph follows documented behavior and SOURCE INDEX references; unsampled modules are represented by their supported paths, but their internal behavior and unshown connections are not inferred.
Main entry points: cli.py CLI commands · harnesses Harness execution · command.py Run orchestration · base.py Probe plugins
Components 21
CLI and selection
- CLI commands
garak/cli.py - Run configuration
garak/_config.py - Probe selection
garak/_selection.py - Run specs
garak/_spec.py - Plugin registry
garak/_plugins.py
Scan execution
- Run orchestration
garak/command.py - Harness execution
garak/harnesses - Probe plugins
garak/probes/base.py - Model generators
garak/generators/base.py - Attempt records
garak/attempt.py - Prompt buffs
garak/buffs
Assessment services
- Vulnerability detectors
garak/detectors - Result evaluation
garak/evaluators/base.py - Intent service
garak/services/intentservice.py - Language service
garak/services/langservice.py - Language providers
garak/langproviders/base.py
Reports and analysis
- Run reports
garak/report.py - Report analysis
garak/analyze
Report application
- Report viewer
garak-report/src/pages/Report.tsx - Theme app shell
garak-report/src/App.tsx
Other components
- CLI user
Connections 27
- CLI user → CLI commands: chooses scan
README.md - CLI commands → Run configuration: loads settings
garak/cli.py - CLI commands → Probe selection: selects plugins
garak/cli.py - CLI commands → Run specs: parses specs
garak/cli.py - CLI commands → Run orchestration: starts run
garak/cli.py - Run configuration → Run specs: builds legacy spec
garak/_config.py - Run specs → Probe plugins: resolves probe tiers
garak/_spec.py - Run orchestration → Plugin registry: loads plugins
garak/command.py - Run orchestration → Harness execution: dispatches run
garak/command.py - Run orchestration → Intent service: uses intent service
garak/command.py - Run orchestration → Report analysis: uses report digest
garak/command.py - Harness execution → Probe plugins: runs probes
garak/harnesses/base.py - Harness execution → Attempt records: processes attempts
garak/harnesses/base.py - Probe plugins → Attempt records: creates attempts
garak/probes/base.py - Model generators → Attempt records: uses attempts
garak/generators/base.py - Probe plugins → Language service: uses translation service
garak/probes/base.py - Language service → Language providers: loads providers
garak/services/langservice.py - Intent service → Run specs: reads intent specs
garak/services/intentservice.py - Intent service → Vulnerability detectors: checks detector mapping
garak/services/intentservice.py - Result evaluation → Attempt records: evaluates attempts
garak/evaluators/base.py - Result evaluation → Report analysis: uses metrics
garak/evaluators/base.py - Harness execution → Model generators: queries target
- Harness execution → Vulnerability detectors: evaluates with
- Harness execution → Prompt buffs: applies buffs
- CLI commands → Run reports: writes run report
garak/cli.py - Theme app shell → Report viewer: renders viewer
garak-report/src/App.tsx - CLI user → Report viewer: reviews results