How athenasec/project-typhon works
Purpose and execution model
Project Typhon is a proof-of-concept LLM red-team and EU AI Act audit workflow. A CLI executes curated attacks against an HTTP chatbot, uses an LLM judge to classify outcomes, and is structured to persist findings, map them to regulations, and render an HTML report. The current working path is the target agent plus red-team runner/judge; downstream persistence and reporting are planned.
Primary entry points
• The orchestrator is cli.py, invoked as python cli.py run --target <URL> --packs <dir>.
• The mock target is a FastAPI application served from target_agent/main.py via Uvicorn.
• Container-oriented local deployment is represented by docker-compose.yml.
• Provider credentials, target model, and target prompt location are environment-controlled through .env.example.
Target-agent boundary
The target is deliberately isolated as an HTTP-served chatbot, allowing the attack engine to target either the bundled mock or a partner staging endpoint without changing core attack logic.
• target_agent/main.py exposes the FastAPI target behavior.
• target_agent/system_prompt.txt supplies the intentionally weak system prompt.
• llm_client.py is the provider-agnostic model access layer, supporting DeepSeek, OpenAI, and Anthropic configurations.
Red-team execution pipeline
The engine loads attack definitions, sends their prompts to the target URL, gathers responses, and judges whether each attempt exposed a vulnerability.
• redteam_engine/runner.py owns attack-pack execution and target interaction.
• redteam_engine/judge.py performs LLM-as-judge pass/fail assessment.
• redteam_engine/schemas.py defines the typed contracts between packs, runner, and judge.
This separates untrusted target output and attack content from the evaluator’s structured finding decision.
Attack-pack content boundary
Attack scenarios are data-driven YAML assets rather than hard-coded workflows, so categories can be expanded without redesigning the runner.
• redteam_engine/attack_packs/prompt_injection.yaml
• redteam_engine/attack_packs/impersonation.yaml
The packs directory passed to the CLI is the discovery boundary for a run.
Findings, compliance, and reporting path
The intended downstream flow is: judged findings → run/finding storage → AI Act article mapping → audit-style HTML report.
• observability/store.py is the planned SQLite persistence boundary.
• compliance/mapper.py and compliance/article_map.yaml map findings to EU AI Act articles.
• compliance/report_generator.py renders output using compliance/templates/report.html, with generated artifacts intended for reports/.
These components exist in the repository but are identified as not yet implemented in the project status.
Runtime and external dependencies
The runtime is Python 3.14 with uv dependency management (pyproject.toml, uv.lock). The target is FastAPI/Uvicorn; target and judge model calls depend on configured external LLM providers and their API keys. SQLite is the planned local observability store, avoiding a required remote database for the PoC.
Main entry points: runner.py Attack runner · main.py Mock target API · attack_packs Attack-pack directory · judge.py LLM judge
Components 18
Red-team execution
- CLI orchestrator · Runs attacks against a target URL and packs directory.
cli.py - Attack-pack directory · Discovery boundary for data-driven attacks.
redteam_engine/attack_packs - Prompt injection pack
redteam_engine/attack_packs/prompt_injection.yaml - Impersonation pack
redteam_engine/attack_packs/impersonation.yaml - Attack runner · Loads packs, calls the target, and gathers responses.
redteam_engine/runner.py - Typed contracts · Contracts shared by packs, runner, and judge.
redteam_engine/schemas.py - LLM judge · Classifies attack outcomes into findings.
redteam_engine/judge.py
Target boundary
HTTP chatbot can be bundled or remote.
- Mock target API · HTTP-served intentionally weak chatbot.
target_agent/main.py - Weak system prompt
target_agent/system_prompt.txt - Model access layer · Provider-agnostic access for target and judge models.
llm_client.py - Local deployment · Container-oriented local target deployment.
docker-compose.yml
Planned audit output
Present but not yet implemented.
- Findings store · Planned run and finding storage boundary.
observability/store.py - AI Act mapper · Planned finding-to-article mapping.
compliance/mapper.py - AI Act article map
compliance/article_map.yaml - HTML report generator · Planned audit-style report renderer.
compliance/report_generator.py - Report template
compliance/templates/report.html - Generated reports
Other components
- LLM providers · DeepSeek, OpenAI, or Anthropic.
Connections 18
- CLI orchestrator → Attack-pack directory: selects
- Attack-pack directory → Prompt injection pack: contains
- Attack-pack directory → Impersonation pack: contains
- CLI orchestrator → Attack runner: starts
- Attack runner → Typed contracts: uses
- Attack runner → Mock target API: sends attack prompts over HTTP
- Mock target API → Weak system prompt: uses
- Mock target API → Model access layer: requests model output
- LLM judge → Model access layer: requests evaluation
- Model access layer → LLM providers: calls
- Attack runner → LLM judge: submits target responses
- Local deployment → Mock target API: deploys
- LLM judge → Findings store: judged findings
- Findings store → AI Act mapper: stored findings
- AI Act article map → AI Act mapper: article rules
- AI Act mapper → HTML report generator: mapped findings
- Report template → HTML report generator: renders with
- HTML report generator → Generated reports: writes HTML