How 2501pr0ject/ullr-ai works
Ullr-AI is a security harness for tool-calling agents: the agent routes actions through a deterministic Gateway, which validates and classifies them, obtains signed approval for irreversible actions, records decisions and executes through a separately protected Tools service. Tools state and execution outcomes support reconciliation and evaluation. The README also documents benchmark runs across protected and unprotected conditions with simulated approvers. Relationships below rely on the README and indexed source references; unsampled evaluation internals are represented at directory level.
Main entry points: app.py Gateway API · run.py Tool-calling agent · executor_client.py Executor client · state.py Protected resources
Components 24
Agent and evaluation
- Tool-calling agent
src/ullr_ai/agent/run.py - Agent clients
src/ullr_ai/agent/clients.py - Benchmark runner
src/ullr_ai/bench - Simulated approvers
src/ullr_ai/bench/approvers.py - Evaluation metrics
src/ullr_ai/bench/evaluate.py
Gateway controls
- Gateway API
src/ullr_ai/gateway/app.py - Action schemas
src/ullr_ai/gateway/schemas.py - Canonicalization
src/ullr_ai/gateway/canon.py - Deterministic policy
src/ullr_ai/gateway/policy.py - Session provenance
src/ullr_ai/gateway/provenance.py - Approval checks
src/ullr_ai/gateway/approvals.py - Decision signing
src/ullr_ai/gateway/signing.py - Budget reservations
src/ullr_ai/gateway/budget.py - Executor client
src/ullr_ai/gateway/executor_client.py - Execution reconciliation
src/ullr_ai/gateway/reconcile.py
Tools execution
- Tools service API
src/ullr_ai/tools_service/app.py - Execution handling
src/ullr_ai/tools_service/executions.py - Protected resources
src/ullr_ai/tools_service/state.py
Persistence and evidence
- Gateway records
src/ullr_ai/gateway/db.py - SQLite helpers
src/ullr_ai/storage.py - Audit records
src/ullr_ai/gateway/audit.py
Other components
- Agent runtime
- Approver
- Language model
Connections 25
- Agent runtime → Tool-calling agent: initiates tasks
README.md - Tool-calling agent → Language model: requests responses
README.md - Tool-calling agent → Agent clients: invokes tools
src/ullr_ai/agent/run.py - Agent clients → Gateway API: proposes actions
src/ullr_ai/agent/clients.py - Gateway API → Action schemas: validates actions
src/ullr_ai/gateway/app.py - Action schemas → Canonicalization: hashes actions
src/ullr_ai/gateway/schemas.py - Gateway API → Deterministic policy: decides actions
src/ullr_ai/gateway/app.py - Gateway API → Session provenance: tracks reads
src/ullr_ai/gateway/app.py - Gateway API → Approval checks: checks approvals
src/ullr_ai/gateway/app.py - Approval checks → Decision signing: verifies decisions
src/ullr_ai/gateway/approvals.py - Approver → Gateway API: submits decision
README.md - Approver → Decision signing: signs decision
README.md - Gateway API → Budget reservations: reserves budgets
src/ullr_ai/gateway/app.py - Gateway API → Executor client: dispatches actions
src/ullr_ai/gateway/app.py - Executor client → Tools service API: executes requests
src/ullr_ai/gateway/executor_client.py - Tools service API → Execution handling: handles execution
src/ullr_ai/tools_service/app.py - Execution handling → Protected resources: reads and writes
src/ullr_ai/tools_service/executions.py - Gateway API → Gateway records: records actions
src/ullr_ai/gateway/app.py - Gateway records → SQLite helpers: uses transactions
src/ullr_ai/gateway/db.py - Protected resources → SQLite helpers: uses transactions
src/ullr_ai/tools_service/state.py - Gateway API → Audit records: writes audit events
src/ullr_ai/gateway/app.py - Execution reconciliation → Executor client: checks outcomes
src/ullr_ai/gateway/reconcile.py - Benchmark runner → Tool-calling agent: runs episodes
- Benchmark runner → Simulated approvers: selects conditions
- Evaluation metrics → Protected resources: scores outcomes