How yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning works
Project type and purpose
• This is an open-source toolkit for Autonomous Penetration Testing using Reinforcement Learning.
• Core purpose: ingest vulnerability data (NVD, Nessus, Nmap), build simulated network scenarios with CyberBattleSim, and train RL agents (via OpenAI Gym) to execute penetration-testing campaigns autonomously.
File structure
Top-level:
• CyberBattleSim/ ← the simulation engine (Git submodule)
• Modifications/ ← all custom code, data pipelines, and notebooks
• Test/ ← supplementary docs (e.g., OpenAI Gym guidance)
• README.md, .gitmodules, documentation files
Under Modifications/:
• Demo/ ← example notebooks (runme.ipynb, runme2.ipynb)
• Nessusextract.py, nmapExtract.py, nvd2sqlite3.py ← data ingestion scripts
• ctf simulation scripts (cyberbattle_myctf.py, my_ctf.py)
• Demo data (nmapScan.xml, VA_Network_Scan.nessus, nvdtest.db)
README for additional insights
• Confirms use of CyberBattleSim as a library.
• All custom logic lives under /Modifications.
• Emphasis on keeping documentation and submodule initialization.
Main components and interactions
a. Data Sources (external feeds)
– NVD JSON data
– Nessus scan exports (.nessus XML)
– Nmap scan exports (.xml)
b. Data Processing Pipeline
– nvd2sqlite3.py: loads NVD into sqlite
– Nessusextract.py: parses Nessus XML → sqlite / in-memory structures
– nmapExtract.py: parses Nmap XML → host/service metadata
c. Database (sqlite)
– Central storage for vulnerabilities, hosts, services
d. Simulation Engine
– CyberBattleSim library: builds network scenarios, attack graph, defines actions & rewards
– cyberbattle_myctf.py / my_ctf.py: glue code to instantiate environments using DB + sim
e. RL Agent & Training Environment
– OpenAI Gym interface around CBS environments
– Notebooks (runme.ipynb, runme2.ipynb): define agent, training loop, policy updates
f. Reporting / Evaluation
– Metrics logged in notebooks or exported for analysis
Relationships & Data Flow:
1. Raw scan files & NVD data → Data Processing Scripts → sqlite DB
2. sqlite DB + CyberBattleSim → Gym-compatible environment
3. RL Agent (in notebooks) ↔ Environment (observations, actions, rewards)
4. Training logs & metrics → Analysis / documentation outputs
Architectural patterns & design principles
• Modular pipeline: ingestion, storage, simulation, training are decoupled
• Adapter pattern: each extractor adapts a different XML/JSON format into the common DB schema
• Notebook-driven orchestration for rapid prototyping
Key elements to capture
• All three extractors as separate modules
• The sqlite database as the integration hub
• The OpenAI Gym agent/training loop in notebooks
• Data artifacts (scan XMLs, demo DB) feeding into the pipeline