How antoninolorenzo/ai-ops works
Project Type and Purpose
• AI-OPS is an AI assistant for penetration testing that leverages open-source LLMs to assist with exploit development, vulnerability research, and code analysis.
• The README clearly states that it serves as an assistive tool (not fully autonomous) for ethical hacking. It integrates with tools like Ollama and supports web search as well as PoC exploit generation.
Analyzing the File Structure
• Top-Level Components:
• The main executable (ai_ops_cli.py), which provides the command-line interface.
• Dockerfile and related configuration files for containerized deployment (API layer).
• The "src" directory holds the core application code, divided into several submodules:
▪ src/agent – containing the AI assistant agent logic.
▪ src/core – subdivided further into libraries for handling LLM interactions (src/core/llm), memory management (src/core/memory), knowledge storage (src/core/knowledge), and tools (src/core/tools).
▪ src/routers – likely for routing and monitoring requests (e.g., tailored endpoints for sessions and monitoring).
▪ src/utils – miscellaneous helper functions including logging and monitoring utilities.
• The "evaluation" directory houses modules for testing or analyzing the architecture (default architectures, tests, visualization tools).
• The "benchmarks" and "test" directories provide additional context for performance measurement and unit testing.
• Architectural Choices:
• A layered/module design is used combining the agent functionality, core processing (LLMs, memory, knowledge, tools), and routing/API support.
• The separation between "src/agent", "src/core", "src/routers", and "evaluation" indicates that AI-OPS separates concern areas such as the decision-making logic (agent), underlying support (LLM and memory), and external interfacing (API/CLI).
a. Main Components:
• Command-Line Interface (CLI): ai_ops_cli.py, which acts as the user’s entry point.
• API Server/Container: Defined by Dockerfile and src/api.py that exposes endpoints.
• AI Agent Module: Located in src/agent (and its subdirectories, including architectures and prompt management).
• Core Processing Modules:
• LLM Component: In src/core/llm (with providers such as Ollama).
• Memory Module: In src/core/memory for session or conversation history.
• Knowledge Base: In src/core/knowledge for storing exploit data or vulnerability records.
• Tools: In src/core/tools, including modules like web_search, terminal, and exploit_db.
• Router & Monitoring Modules: In src/routers and src/utils/monitoring to manage traffic flows and monitor state.
• Evaluation/Benchmarking: The evaluation and benchmarks directories, which are used for testing and validating the architectures.
• External Dependencies: Supporting tools like Ollama for LLM inference and Docker for API deployment.
b. Relationships and Interactions:
• The CLI interacts with the API server which in turn triggers the AI agent.
• The AI agent orchestrates operations by interfacing with the LLM module, memory and knowledge modules, and tool modules.
• Routers manage session handling and monitoring for both API and agent components.
• The evaluation modules assist with validating the architecture and generating visualizations (e.g., in evaluation/commons/visualization.py).
c. Architectural Patterns and Principles:
• Layered Architecture: Separation between presentation (CLI/API) and business logic (agent and core modules).
• Modular Design: Clearly separated modules for LLM operations, memory management, and domain-specific tools.
• Extensibility: The architecture is designed to allow plugging in different LLMs and test modules (see supported models in README).
• Use of Containers: Dockerfile indicates that the API component is containerized for easier deployment.
d. Relevant Technologies/Frameworks:
• Python (≥ 3.11) forms the backbone.
• Docker for containerized API deployment.
• Integration with Ollama for open-source LLM inference.
• Libraries/tools for unit testing (as seen in test/ directories) and code quality (pylint, etc.).