How google-research/camel-prompt-injection works
Project Type and Purpose
• This is a Python-based research artifact (CaMeL) implementing a defense-by-design framework to defeat prompt injections when interacting with large language models (LLMs).
• It exposes a CLI entrypoint (main.py) orchestrated via the “uv” runner, loads API keys from .env, constructs LLM prompts, runs attacks/evaluations, and applies a multi-stage pipeline of security policies and sandboxed interpretation.
File-Structure Analysis
Top-Level:
• main.py, run_code.py, print_conversation.py – CLI scripts driven by uv
• pyproject.toml, uv.lock, .env.example – build/configuration/runtime lockfile
• analysis.ipynb, count_tokens.py – ad-hoc analysis notebooks/scripts
src/camel/ (Core Library):
• capabilities/ – token counting, readers, sources, caching utilities
• interpreter/ – sandboxed code execution, protocol definitions, result/value models
• pipeline_elements/ – chain‐of‐responsibility filters (AgentDojo, Anthropic filter, privileged LLM wrapper, replay), with submodule security_policies/ for domain-specific policy sets (banking, travel, Slack, workspace)
• Other modules: models.py (supported LLMs), system_prompt_generator.py, security_policy.py, quarantined_llm.py, chat_turn.py, etc.
tests/ – unit tests covering the interpreter, language constructs, pipeline elements, token counting.
README Insights
• CLI arguments drive model choice, defense toggles, evaluation mode.
• External dependencies: OpenAI/Anthropic API, AgentDojo attack harness, “uv” runtime.
• The paper title and warning highlight that this is research code, not a production-grade system.
Main Components
a. CLI Layer (uv runner with main.py / run_code.py)
b. Configuration (.env, API keys)
c. Core Library:
• Prompt Generator
• Pipeline of Elements (filters, policy enforcement, replay)
• Interpreter Sandbox
• Capabilities (token counter, cache)
• Model Interface (OpenAI, Anthropic wrappers, quarantined LLM)
d. External Services: LLM Providers (OpenAI, Anthropic), AgentDojo attack harness
e. Test Suite (separate, can be a side-component)
Component Relationships and Data Flow
• CLI invokes the Prompt Generator → builds system/user prompts.
• Prompts are fed into the Pipeline (chain-of-responsibility): each pipeline element intercepts or transforms the prompt/response.
• After filtering, the request goes to the Model Interface which calls the external LLM API.
• Model’s response may yield code snippets – passed into the Interpreter Sandbox for execution.
• Interpreter outputs results → Security Policy module validates or rejects outputs.
• Final output returned to user or evaluation harness (AgentDojo).
• Capabilities (token counter, cache) are invoked at various points to track usage or reuse prior results.
Architectural Patterns & Principles
• Chain-of-Responsibility (pipeline elements)
• Interpreter Pattern (sandboxed code execution)
• Modular Separation (capabilities, policies, prompt generation, model abstraction)
• Test-Driven Approach (comprehensive unit tests for each module)
Technologies and Frameworks
• Python 3.x, “uv” CLI runner
• ONNX or direct HTTP calls to LLM APIs (OpenAI, Anthropic)
• PyProject/TOML build, Ruff for linting, Pyright for type checks, Pytest for testing
Detailed Capture of Architectural Elements
• Security policies under one composite element with sub-labels for each domain
• Token-count capability hooking into both prompt construction and response processing