# jnanasaj/web-copilot architecture

> Architecture diagram of the GitHub repository jnanasaj/web-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jnanasaj/web-copilot
- Repository: https://github.com/jnanasaj/web-copilot

## Overview

##### Project Type and Purpose
- WebCopilot is an open-source command-line automation tool for subdomain enumeration and vulnerability scanning. It orchestrates multiple third-party tools in a pipeline to discover subdomains, filter live hosts, extract assets (titles, screenshots, endpoints), and then run vulnerability scanners on filtered parameters. Results are printed to the console and saved to a specified directory.

##### File Structure Analysis
- Top-level files:
- LICENSE, README.md – project metadata
- install.sh – installer script for dependencies and setup
- webcopilot – the main executable script (likely a shell or Python script orchestrating the workflow)
- No separate “src” or “lib” folder: the core logic lives in the “webcopilot” script.
- No frontend/backend split—this is a single‐purpose CLI tool.
- External dependencies are invoked at runtime by the script.

##### Key Components
a. User Interface
- CLI Entry Point (“webcopilot” script) – accepts target domain and output directory.
b. Orchestration Engine
- Core workflow logic that calls each stage in sequence, handles intermediate files and logging.
c. Enumeration Stage
- Passive enumeration tools: assetfinder, sublister, subfinder, amass, findomain, hackertarget, riddler, crt.sh
- Aggregator/merger – collects and deduplicates subdomain lists.
d. Active Enumeration
- gobuster with SecLists DNS wordlists
- amass in brute-force mode
- dnsx for live host filtering.
e. Asset Extraction
- httpx and aquatone for HTTP probing, title extraction, screenshots
- subjack for takeover detection.
f. Endpoint Crawling & Parameter Filtering
- waybackurls, gauplus for URL collection
- gf patterns to isolate parameters for XSS, SQLi, SSRF, open redirect, RCE.
g. Vulnerability Scanning
- Tools per vulnerability: dalfox (XSS), kxss, openredirex, nuclei, sqlmap, etc.
- Parallel execution manager (if used) or sequential invocation.
h. Output Management
- Result formatter (console output)
- Directory structure for saving raw tool outputs, consolidated reports, logs.

##### Relationships and Data Flow
- Orchestration Engine → Enumeration Stage (passes domain, receives file1.txt)
- Enumeration → Active Enumeration (file1.txt → file2.txt)
- Active Enumeration → Asset Extraction (live_hosts.txt → assets.json/screenshots/)
- Asset Extraction → Endpoint Crawling (assets.json → urls.txt)
- Endpoint Crawling → Parameter Filtering (urls.txt → param_lists/)
- Parameter Filtering → Vulnerability Scanning (param_lists/ → vuln_reports/)
- Vulnerability Scanning → Output Management (vuln_reports/ → final_report, console)

##### Architectural Patterns and Principles
- Pipeline/chain-of-responsibility: each stage takes input, processes, outputs to the next.
- Loose coupling: each tool invocation is a separate process; new tools can be added at any stage.
- Single responsibility: each stage handles one concern (enumeration vs. scanning vs. reporting).

##### Technologies and Extensibility
- Shell scripting (bash/zsh) or Python for orchestration
- External CLI tools: assetfinder, subfinder, amass, gobuster, dnsx, httpx, aquatone, waybackurls, gauplus, gf, dalfox, nuclei, etc.
- Output folders on filesystem; optional JSON, CSV or plain text.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Project Metadata"
        Doc["README.md"]:::metadata
        Lic["LICENSE"]:::metadata
    end
    subgraph "Setup"
        SetupScript["install.sh"]:::orchestration
    end
    subgraph "Pipeline"
        UI["webcopilot"]:::orchestration
        Orch["Workflow Engine"]:::orchestration
        subgraph "Enumeration Layer"
            Passive["Passive Enumeration\n(assetfinder, subfinder, amass, etc.)"]:::enumeration
            Agg["Aggregator/Merger"]:::enumeration
        end
        subgraph "Active Enumeration Layer"
            Gob["gobuster + wordlists"]:::enumeration
            AmassBr["amass (brute-force)"]:::enumeration
            DNSX["dnsx (live filter)"]:::enumeration
        end
        subgraph "Asset Extraction Layer"
            HTTPX["httpx (HTTP probing)"]:::extraction
            Aqua["aquatone (screenshots)"]:::extraction
            Subjack["subjack (takeover)"]:::extraction
        end
        subgraph "Crawling & Filtering Layer"
            Wayback["waybackurls"]:::extraction
            Gau["gauplus"]:::extraction
            GF["gf patterns"]:::extraction
        end
        subgraph "Vulnerability Scanning Layer"
            Dalfox["dalfox (XSS)"]:::scanning
            KXSS["kxss"]:::scanning
            OpenRedir["openredirex"]:::scanning
            Nuclei["nuclei"]:::scanning
            SQLmap["sqlmap"]:::scanning
        end
        subgraph "Output Management"
            Reports["Reports Directory\n(raw & consolidated)"]:::output
            Console["Console Output"]:::output
        end
    end

    Doc --> SetupScript
    Lic --> SetupScript
    SetupScript --> UI
    UI -->|"domain, output_dir"| Orch
    Orch -->|"subdomains.txt"| Passive
    Passive --> Agg
    Agg -->|"deduped.txt"| Gob
    Gob --> AmassBr
    AmassBr --> DNSX
    DNSX -->|"live_hosts.txt"| HTTPX
    HTTPX --> Aqua
    Aqua --> Subjack
    Subjack --> Wayback
    Wayback --> Gau
    Gau --> GF
    GF --> Dalfox
    GF --> KXSS
    GF --> OpenRedir
    GF --> Nuclei
    GF --> SQLmap
    Dalfox --> Reports
    KXSS --> Reports
    OpenRedir --> Reports
    Nuclei --> Reports
    SQLmap --> Reports
    Reports --> Console

    click UI "https://github.com/jnanasaj/web-copilot/tree/main/webcopilot"
    click SetupScript "https://github.com/jnanasaj/web-copilot/blob/main/install.sh"
    click Doc "https://github.com/jnanasaj/web-copilot/blob/main/README.md"
    click Lic "https://github.com/jnanasaj/web-copilot/tree/main/LICENSE"

    classDef orchestration fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef enumeration fill:#90EE90,stroke:#333,stroke-width:1px
    classDef extraction fill:#FFFFE0,stroke:#333,stroke-width:1px
    classDef scanning fill:#FFB6C1,stroke:#333,stroke-width:1px
    classDef output fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef metadata fill:#E0E0E0,stroke:#333,stroke-width:1px
```
