# yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning architecture

> Architecture diagram of the GitHub repository yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning
- Repository: https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning

## Overview

##### Project type and purpose
- This is an open-source toolkit for Autonomous Penetration Testing using Reinforcement Learning.
- Core purpose: ingest vulnerability data (NVD, Nessus, Nmap), build simulated network scenarios with CyberBattleSim, and train RL agents (via OpenAI Gym) to execute penetration-testing campaigns autonomously.

##### File structure
Top-level:
- CyberBattleSim/           ← the simulation engine (Git submodule)
- Modifications/            ← all custom code, data pipelines, and notebooks
- Test/                     ← supplementary docs (e.g., OpenAI Gym guidance)
- README.md, .gitmodules, documentation files

Under Modifications/:
- Demo/                     ← example notebooks (runme.ipynb, runme2.ipynb)
- Nessusextract.py, nmapExtract.py, nvd2sqlite3.py ← data ingestion scripts
- ctf simulation scripts (cyberbattle_myctf.py, my_ctf.py)
- Demo data (nmapScan.xml, VA_Network_Scan.nessus, nvdtest.db)

##### README for additional insights
- Confirms use of CyberBattleSim as a library.
- All custom logic lives under /Modifications.
- Emphasis on keeping documentation and submodule initialization.

##### Main components and interactions
a. Data Sources (external feeds)
– NVD JSON data
– Nessus scan exports (.nessus XML)
– Nmap scan exports (.xml)
b. Data Processing Pipeline
– nvd2sqlite3.py: loads NVD into sqlite
– Nessusextract.py: parses Nessus XML → sqlite / in-memory structures
– nmapExtract.py: parses Nmap XML → host/service metadata
c. Database (sqlite)
– Central storage for vulnerabilities, hosts, services
d. Simulation Engine
– CyberBattleSim library: builds network scenarios, attack graph, defines actions & rewards
– cyberbattle_myctf.py / my_ctf.py: glue code to instantiate environments using DB + sim
e. RL Agent & Training Environment
– OpenAI Gym interface around CBS environments
– Notebooks (runme.ipynb, runme2.ipynb): define agent, training loop, policy updates
f. Reporting / Evaluation
– Metrics logged in notebooks or exported for analysis

Relationships & Data Flow:
1. Raw scan files & NVD data → Data Processing Scripts → sqlite DB
2. sqlite DB + CyberBattleSim → Gym-compatible environment
3. RL Agent (in notebooks) ↔ Environment (observations, actions, rewards)
4. Training logs & metrics → Analysis / documentation outputs

##### Architectural patterns & design principles
- Modular pipeline: ingestion, storage, simulation, training are decoupled
- Adapter pattern: each extractor adapts a different XML/JSON format into the common DB schema
- Notebook-driven orchestration for rapid prototyping

##### Key elements to capture
- All three extractors as separate modules
- The sqlite database as the integration hub
- The OpenAI Gym agent/training loop in notebooks
- Data artifacts (scan XMLs, demo DB) feeding into the pipeline

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Sources"
        NVDFile(("NVD JSON Feed")):::external
        NessusFile(("Nessus Scan XML")):::external
        NmapFile(("Nmap Scan XML")):::external
    end

    subgraph "Data Ingestion"
        NVDExtractor["NVD Extractor\n(nvd2sqlite3.py)"]:::dataIngestion
        NessusExtractor["Nessus Extractor\n(Nessusextract.py)"]:::dataIngestion
        NmapExtractor["Nmap Extractor\n(nmapExtract.py)"]:::dataIngestion
    end

    SQLiteDB["sqlite DB\n(nvdtest.db)"]:::database

    subgraph "Simulation Engine"
        CyberBattleSim["CyberBattleSim\n(sim engine)"]:::simulation
        CTFGlue1[/"cyberbattle_myctf.py"/]:::simulation
        CTFGlue2[/"my_ctf.py"/]:::simulation
    end

    subgraph "RL Training"
        GymEnv["OpenAI Gym Env Wrapper"]:::simulation
        Notebook1["runme.ipynb"]:::rl
        Notebook2["runme2.ipynb"]:::rl
    end

    subgraph "Documentation"
        OpenAI_Gym_Doc["OpenAI Gym Guidance\n(OpenAI-Gym.md)"]:::documentation
        README["README.md"]:::documentation
    end

    NVDFile --> NVDExtractor
    NessusFile --> NessusExtractor
    NmapFile --> NmapExtractor

    NVDExtractor --> SQLiteDB
    NessusExtractor --> SQLiteDB
    NmapExtractor --> SQLiteDB

    SQLiteDB --> CyberBattleSim
    SQLiteDB --> CTFGlue1
    SQLiteDB --> CTFGlue2

    CyberBattleSim --> CTFGlue1
    CyberBattleSim --> CTFGlue2

    CTFGlue1 --> GymEnv
    CTFGlue2 --> GymEnv

    GymEnv --> Notebook1
    GymEnv --> Notebook2

    Notebook1 --> OpenAI_Gym_Doc
    Notebook2 --> OpenAI_Gym_Doc

    click NVDExtractor "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/nvd2sqlite3.py"
    click NessusExtractor "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/Nessusextract.py"
    click NmapExtractor "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/nmapExtract.py"
    click SQLiteDB "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/nvdtest.db"
    click CyberBattleSim "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/tree/main/CyberBattleSim/"
    click CTFGlue1 "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/cyberbattle_myctf.py"
    click CTFGlue2 "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/my_ctf.py"
    click Notebook1 "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/runme.ipynb"
    click Notebook2 "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Modifications/Demo/runme2.ipynb"
    click OpenAI_Gym_Doc "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/Test/OpenAI-Gym.md"
    click README "https://github.com/yeeticusyey/autonomous-penetration-testing-using-reinforcement-learning/blob/main/README.md"

    classDef dataIngestion fill:#adebad,stroke:#333,stroke-width:1px
    classDef database fill:#add8e6,stroke:#333,stroke-width:1px
    classDef simulation fill:#ffcc99,stroke:#333,stroke-width:1px
    classDef rl fill:#d8bfd8,stroke:#333,stroke-width:1px
    classDef external fill:#cccccc,stroke:#333,stroke-width:1px
    classDef documentation fill:#f0e68c,stroke:#333,stroke-width:1px
```
