How confident-ai/deepteam works
DeepTeam is a local LLM red-teaming framework: an engineer configures a target model and risk scenarios, the framework simulates attacks, submits test cases to the target, and evaluates replies with LLM-as-a-judge metrics to produce scored findings. Its broader product includes guardrails, trace analysis, code scanning, risk-framework mappings, a CLI, and an optional hosted results integration. The sampled sources substantiate attack refinement and vulnerability assessment/evaluation; unsampled subsystem wiring is represented only where documented, so some links are intentionally omitted.
Main entry points: vulnerabilities Vulnerability scenarios · red_teamer.py Red team runner · risk_assessment.py Risk assessment and reports · test_case Red-team test cases
Components 17
Red-team assessment
- Red team runner
deepteam/red_teamer/red_teamer.py - Vulnerability scenarios
deepteam/vulnerabilities - Judge metrics
deepteam/metrics - Red-team test cases
deepteam/test_case - Risk frameworks
deepteam/frameworks
Test generation
- Attack refinement
deepteam/attacks/attack_engine/attack_engine.py - Single- and multi-turn attacks
deepteam/attacks - Attack simulation
deepteam/attacks/attack_simulator/attack_simulator.py
Protection and analysis
- Runtime guardrails
deepteam/guardrails/guardrails.py - Trace scanner
deepteam/trace_scanner/trace_scanner.py - Code scanner
deepteam/code_scanner/code_scanner.py
Interfaces and reporting
- Command-line interface
deepteam/cli/main.py - Platform integration
deepteam/confident/api.py - Risk assessment and reports
deepteam/red_teamer/risk_assessment.py
Other components
- Engineer
- Target LLM system
- Confident AI
Connections 21
- Engineer → Red team runner: configures and runs
- Red team runner → Vulnerability scenarios: selects scenarios
- Vulnerability scenarios → Single- and multi-turn attacks: simulates attacks
- Vulnerability scenarios → Attack simulation: generates test inputs
- Vulnerability scenarios → Attack refinement: refines inputs
- Attack refinement → Attack simulation: generates variations
- Vulnerability scenarios → Red-team test cases: creates test cases
- Vulnerability scenarios → Target LLM system: submits test inputs
- Target LLM system → Red-team test cases: returns responses
- Vulnerability scenarios → Judge metrics: evaluates responses
- Judge metrics → Red-team test cases: adds scores and reasons
- Risk frameworks → Vulnerability scenarios: categorizes risks
- Engineer → Runtime guardrails: configures protection
- Engineer → Trace scanner: submits traces
- Engineer → Code scanner: scans source
- Code scanner → Risk assessment and reports: produces findings
- Engineer → Command-line interface: runs commands
- Command-line interface → Red team runner: configures assessment
- Red team runner → Risk assessment and reports: returns assessment results
- Risk assessment and reports → Platform integration: syncs results
- Platform integration → Confident AI: transfers risk data