# t0sche/cvss-bt architecture

> Architecture diagram of the GitHub repository t0sche/cvss-bt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/t0sche/cvss-bt
- Repository: https://github.com/t0sche/cvss-bt

## Overview

##### Project Type and Purpose
- This is a Python-based data-enrichment pipeline (an open-source tool) that runs daily, pulls CVSS Base scores from the NVD, fetches supplemental threat intelligence (EPSS, CISA KEV, VulnCheck KEV, Metasploit, Nuclei, ExploitDB, PoC-in-GitHub), computes an enriched CVSS Temporal metric (Exploit Code Maturity/Exploitability “E”), and outputs a consolidated CSV (`cvss-bt.csv`).

##### File-Structure Analysis
- Top level:
- `.github/workflows/epss.yml` & `cvss-bt.yml` – CI/CD or scheduler definitions (GitHub Actions).
- `code/` – core Python logic:
– `process_nvd.py` – fetches new EPSS scores, checks NVD for updates.
– `enrich_nvd.py` – applies the temporal metric logic and produces enriched records.
– `requirements.txt` – dependencies.
– `last_run.txt` – state marker for incremental fetches.
- `cvss-bt.csv` – final daily output.
- `test.sh` – local end-to-end test harness.

##### README Insights
- Describes the enrichment logic (mapping threat-intel sources to CVSS “E” values).
- Emphasizes daily schedule, data sources, score-version handling and thresholds.
- Lists external integrations (NVD API, EPSS API, CISA KEV, etc.).

a. Main Components
1. Scheduler / Orchestrator (GitHub Actions)
2. Ingestion Layer
- EPSS Fetcher
- NVD Fetcher
- Threat-Intel Adapters (CISA KEV, VulnCheck, Metasploit, Nuclei, ExploitDB, PoC-in-GitHub)
3. Processing Layer
- process_nvd.py (merge EPSS + NVD)
- enrich_nvd.py (calculate Temporal metric “E”)
4. Persistence
- State Store: last_run.txt
- Output Store: cvss-bt.csv
5. Test Harness (test.sh for local verification)
6. External Services/APIs

b. Relationships & Data Flow
- Scheduler → runs `process_nvd.py` → fetch EPSS → query NVD (if new EPSS) → merge base scores
- `process_nvd.py` → writes intermediate state / updates `last_run.txt`
- Scheduler → runs `enrich_nvd.py` → calls threat-intel adapters → computes “E” value per CVE
- `enrich_nvd.py` → writes final `cvss-bt.csv`
- test.sh simulates the above locally

c. Architectural Patterns & Principles
- Batch ETL pattern: clear separation of ingestion vs. transformation vs. storage
- Single-responsibility scripts for maintainability
- Idempotent scheduling (uses `last_run.txt` to avoid duplicate work)
- Config-as-code for workflows (GitHub Actions YAML)

d. Key Technologies & Libraries
- Python 3.x + `requests` / HTTP client libraries
- GitHub Actions for orchestration
- CSV file I/O
- External REST APIs (NVD, EPSS, various OSINT feeds)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration" 
        direction TB
        Scheduler1["GitHub Actions: cvss-bt.yml"]:::orchestration
        Scheduler2["GitHub Actions: epss.yml"]:::orchestration
    end

    %% Ingestion & Initial Processing Layer
    subgraph "Ingestion & Initial Processing"
        direction TB
        ProcessNVD["process_nvd.py"]:::processing
    end

    %% Processing & Enrichment Layer
    subgraph "Processing & Enrichment"
        direction TB
        Enrich["enrich_nvd.py"]:::processing
        subgraph "Threat-Intel Adapters"
            direction TB
            CISA["CISA KEV"]:::external
            VulnCheck["VulnCheck KEV"]:::external
            Metasploit["Metasploit"]:::external
            Nuclei["Nuclei"]:::external
            ExploitDB["ExploitDB"]:::external
            PoCGit["PoC-in-GitHub"]:::external
        end
    end

    %% External Services Layer
    subgraph "External Services"
        direction TB
        EPSSAPI["EPSS API"]:::external
        NVDAPI["NVD API"]:::external
    end

    %% Storage Layer
    subgraph "Storage"
        direction TB
        StateStore["last_run.txt"]:::storage
        OutputStore["cvss-bt.csv"]:::storage
    end

    %% Auxiliary Files
    subgraph "Auxiliary"
        direction TB
        Test["test.sh"]:::aux
        Req["requirements.txt"]:::aux
        README["README.md"]:::aux
        LICENSE["LICENSE"]:::aux
        GITIGNORE[".gitignore"]:::aux
    end

    %% Data Flows
    Scheduler1 -->|triggers process| ProcessNVD
    Scheduler2 -->|triggers process| ProcessNVD
    ProcessNVD -->|writes state| StateStore
    Scheduler1 -->|triggers enrich| Enrich
    Scheduler2 -->|triggers enrich| Enrich

    ProcessNVD -->|fetch EPSS scores JSON| EPSSAPI
    ProcessNVD -->|fetch CVE list| NVDAPI
    EPSSAPI --> ProcessNVD
    NVDAPI --> ProcessNVD

    Enrich -->|fetch intel| CISA
    Enrich -->|fetch intel| VulnCheck
    Enrich -->|fetch intel| Metasploit
    Enrich -->|fetch intel| Nuclei
    Enrich -->|fetch intel| ExploitDB
    Enrich -->|fetch intel| PoCGit
    Enrich -->|writes enriched CSV| OutputStore

    Test --> ProcessNVD
    Test --> Enrich

    Req --> ProcessNVD
    Req --> Enrich
    README --> ProcessNVD
    README --> Enrich

    %% Click Events
    click Scheduler1 "https://github.com/t0sche/cvss-bt/blob/main/.github/workflows/cvss-bt.yml"
    click Scheduler2 "https://github.com/t0sche/cvss-bt/blob/main/.github/workflows/epss.yml"
    click ProcessNVD "https://github.com/t0sche/cvss-bt/blob/main/code/process_nvd.py"
    click Enrich "https://github.com/t0sche/cvss-bt/blob/main/code/enrich_nvd.py"
    click StateStore "https://github.com/t0sche/cvss-bt/blob/main/code/last_run.txt"
    click OutputStore "https://github.com/t0sche/cvss-bt/blob/main/cvss-bt.csv"
    click Req "https://github.com/t0sche/cvss-bt/blob/main/code/requirements.txt"
    click Test "https://github.com/t0sche/cvss-bt/blob/main/test.sh"
    click README "https://github.com/t0sche/cvss-bt/blob/main/README.md"
    click LICENSE "https://github.com/t0sche/cvss-bt/tree/main/LICENSE"
    click GITIGNORE "https://github.com/t0sche/cvss-bt/blob/main/.gitignore"

    %% Styles
    classDef orchestration fill:#ddeeff,stroke:#3399ff,color:#000;
    classDef processing fill:#ddffdd,stroke:#33aa33,color:#000;
    classDef external fill:#ffeedd,stroke:#ff9933,color:#000;
    classDef storage fill:#eeeeee,stroke:#888888,color:#000;
    classDef aux fill:#f5f5f5,stroke:#999999,color:#000;
```
