How mariomc14/devsecops-adversary-llm works
Project Type and Purpose
• This is a research‐oriented security tool combining:
• An “Attack Tree Generation” framework that uses LLMs (QwQ-32B, GPT-4) to produce .DOT attack trees from structured prompts.
• An “SCE Experiment” suite that automates a DevSecOps security scenario in AWS, validating an attacker’s ability to misuse permissions via a CI/CD pipeline (ChaosXploit framework).
• It is not a classic full‐stack app but a hybrid open‐source research toolkit with two main subsystems:
1. LLM-driven attack tree generation (local Python scripts + prompts + artifacts)
2. AWS CI/CD experiment orchestration (automation scripts, knowledge database, AWS topology)
File Structure Analysis
A. Attack_Tree_Generation/
• Prompts.txt, AttackProcedures.pdf, Flow_Diagram.png
• Output PDFs (gpt4.pdf, qwq.pdf)
→ Core component: prompt design → LLM invocation → .DOT attack-tree artifacts
B. SCE_Experiment/
• KnowledgeDB/AttackTrees/*.dot → stored attack tree definitions
• Runner/
• actions/actions.py (defines exploit steps)
• probes/probes.py (verifies system state)
• run.sh + SCE_Experiment.json (experiment entrypoint & config)
• requirements.txt
→ Core component: orchestration script → virtualenv + dependency install → run experiment against AWS topology
README Insights
• The AWS topology: IAM, CodeCommit/GitHub, CodePipeline, CodeBuild, CodeDeploy, S3, CloudFormation, EC2 Spot Instances, RDS, GuardDuty, CloudWatch, (optional CDN/WAF/API Gateway/NLB)
• Quick‐start guidance for the Runner
• Security scenario details (misconfigured IAM, Spot instance abuse)
1. Attack Tree Generation Module
• Prompts Manager (Prompts.txt)
• LLM Engine (QwQ-32B / GPT-4 service)
• Output Formatter (.DOT generator, cosmetic rules)
• Artifact Store (qwq.pdf, gpt4.pdf, AttackProcedures.pdf)
2. SCE Experiment Module
• KnowledgeDB (AttackTrees/*.dot)
• Experiment Runner
– actions.py (exploit steps)
– probes.py (state checks)
– run.sh & config JSON
• Virtual Environment / Dependency Installer
3. AWS CI/CD Pipeline (external services):
• Developers (IAM role)
• CodeCommit / GitHub
• CodePipeline
– CodeBuild
– CodeDeploy
• S3 (artifacts storage)
• CloudFormation (IaC)
• EC2 Spot Instances
• RDS
• GuardDuty & CloudWatch
• Optional: API Gateway, WAF, CloudFront, NLB
• Prompts Manager → LLM Engine (input prompts) → Output Formatter → Artifact Store
• Runner → virtualenv → actions.py invokes AWS CLI → AWS services (SpotInstance, IAM:PassRole)
• probes.py polls CloudWatch/GuardDuty → reports results back to Runner
• KnowledgeDB feeds .dot definitions into Runner (to compare with generated attack trees)
c. Architectural Patterns & Principles:
• Modular separation: “Generation” vs “Experiment” subsystems
• Pipeline/orchestration pattern: Runner script automates environment setup → execution → verification
• IaC pattern: CloudFormation driven by CodePipeline
• Data‐driven workflows: Prompts → LLM → .DOT → PDF artifacts; Experiment config JSON → run.sh → AWS actions & probes
d. Technologies / Frameworks:
• Python 3.12 (scripts: hypGenerator.py, actions.py, probes.py)
• Bash (run.sh)
• LLM APIs (QwQ-32B, GPT-4)
• Graphviz (.DOT format)
• AWS CLI & services
• Virtualenv / requirements.txt