How martel-innovate/horse-genai-ckb works
Major Components
• Centralized Database: Represented by the PostgreSQL database, which is set up using Docker Compose (with configuration files such as .env.example, docker-compose.yml, and the init.sql script). This component stores the attack data and enriched mitigation information.
• Data Acquisition & Scraping (Dynamic Attack Resolver): This includes the WebScraper modules (see src/scrape_and_generate.py, src/scrape_attacks.py, src/scrape_mitigations.py, src/scraper.py) that extract attack patterns and mitigation strategies from CAPEC.
• ETL Process: The component (src/extract_transform_load.py) that converts the scraped and AI-generated data into a structured format, making it ready for integration into the central database.
• Mitigation Generator: Comprising the mitigation_generator.py and generate_from_csv.py modules, this component uses generative AI (via LLMs such as Llama 3.1, Claude 3.5 Sonnet, DeepSeek R1 8B, Granite 3.0 8B, Llama 2 13B/7B, and ALIENTELLIGENCE-cyberaisecurity) to generate and enhance mitigation responses.
• Evaluation Module: Although not a runtime component, the evaluation and notebook directories (evaluation/LLMs and evaluation/notebooks) provide CSV results and Jupyter visualizations detailing the performance of various models.
• External Services & Integrations: Incorporate the connection to external LLM providers (Ollama for local model serving and proprietary APIs for models like Claude) and the RESTful API service mentioned in the README for external communication.
Relationships and Data Flow
• Data originates at the CAPEC website and flows into the WebScraper modules, which output raw data (stored into CSV files found in the ‘output’ folder).
• This scraped data is then processed by the ETL script to format it appropriately.
• The formatted data is sent to the Mitigation Generator, which invokes external generative AI models. The generated mitigation responses are saved back into the ‘output’ folder (each file named by model) for traceability.
• The Centralized Database then receives the processed data (as handled via ETL) to maintain the enhanced Common Knowledge Base.
• The Evaluation component separately processes the outputs from various LLMs, providing performance assessments that inform the accuracy and quality of the mitigations.
Architectural Patterns and Technologies
• Modular Design: The project cleanly separates functions into components (Scraping, ETL, AI Generation, and Evaluation).
• Pipeline/Workflow Pattern: Data flows sequentially from scraping to generation, evaluation, and finally into the database.
• Containerization: The use of Docker Compose emphasizes containerized deployment for the PostgreSQL database.
• Integration with External APIs: Interaction with various generative AI models through both local (Ollama for Llama models) and remote (Claude) APIs.
• Extensibility: The modular approach allows you to plug in additional models or modify components (like swapping ETL processes or adding new evaluation criteria) without affecting the entire system.