How ibm/itbench-ciso-caa-agent works
Project Type and Purpose
• This is a full-stack automation agent tool (CISO Agent) for compliance assessments in Kubernetes/RHEL environments.
• It uses LLMs to generate policies (Kyverno, OPA Rego), orchestrate evidence collection, integrate with GitOps, and deploy/fix policies automatically.
• Built on top of Python, Docker/Podman, and open-source frameworks CrewAI and LangGraph.
File Structure Analysis
• Top level: configuration files (.flake8, .gitignore, Dockerfile, pyproject.toml, requirements-dev.txt), build/deployment artifacts (Dockerfile, agent-harness.yaml), documentation (README.md, langtrace.md, img/)
• src/ciso_agent/ – core code:
• main.py: entrypoint for the agent container
• manager.py: orchestrates the overall workflow
• llm.py: LLM interface (wrapper around CrewAI/LangGraph)
• agents/: concrete agents for various scenarios (kubernetes_kubectl_opa.py, kubernetes_kyverno.py, rhel_playbook_opa.py, etc.)
• tools/: helper modules to generate/run policies and playbooks (generate_*.py, run_*.py, utils.py)
• tests/: unit tests for llm and model abstractions
Key Architecture Insights from README
• The agent runs as a container, mounts an environment (.env) and a workdir containing cluster credentials/inventory.
• It reads a “goal” (natural language scenario), then invokes the LLM to plan steps, generates policy artifacts, deploys them, collects results, iterates until success.
• Supports multiple LLM backends (OpenAI-compatible, IBM watsonx.ai, Azure) via environment configuration.
• Integrates with external systems: Kubernetes (via kubectl), Ansible playbooks (RHEL).
a. Main Components
• User / CLI: invokes the agent container with goal, workdir, .env
• Containerized CISO Agent:
• Entrypoint (main.py)
• Manager (workflow coordinator)
• LLM Interface (llm.py with CrewAI/LangGraph)
• Policy Agents (kubernetes_kubectl_opa, kubernetes_kyverno, rhel_playbook_opa)
• Tools Module (generate_*, run_*, utils)
• External Services/Systems:
• LLM Provider (OpenAI, WatsonX, Azure)
• Kubernetes API Server (kubectl)
• RHEL Host via Ansible (inventory.ansible.ini -> ansible-playbook)
• GitOps Repository (optional, for policy deployment)
b. Relationships & Data Flow
1. User/CLI → Container: passes .env, workdir, goal text
2. main.py → Manager: initializes environment, loads goal
3. Manager ↔ llm.py: sends goal & context, receives plan/next steps
4. Manager → Agents: for each step, calls appropriate agent module
5. Agent → Tools (generate_*.py): produce policy or playbook artifacts
6. Agent → Tools (run_*.py): invoke kubectl/OPA CLI or ansible-playbook against target env
7. Tools ↔ External System: run commands, collect responses (success/failure, logs)
8. Manager loops until goal achieved, then outputs result
c. Architectural Patterns & Principles
• Modular design: clear separation between orchestration (manager), LLM interface, agent implementations, and tool adapters.
• Plugin-style agents: each “agents/” file implements a scenario adapter.
• Environment-agnostic LLM integration: llm.py abstracts provider details via .env.
• Containerized deployment: all components run in a single Docker/Podman container, relying on mounted volumes for input.
d. Key Technologies/Frameworks
• Python 3.11 (pyproject.toml)
• LLM frameworks: CrewAI, LangGraph
• Docker/Podman for containerization
• kubectl, OPA CLI, Ansible for policy deployment & compliance checks
• YAML for scenario harness (agent-harness.yaml)