How t0sche/cvss-bt works
Project Type and Purpose
• This is a Python-based data-enrichment pipeline (an open-source tool) that runs daily, pulls CVSS Base scores from the NVD, fetches supplemental threat intelligence (EPSS, CISA KEV, VulnCheck KEV, Metasploit, Nuclei, ExploitDB, PoC-in-GitHub), computes an enriched CVSS Temporal metric (Exploit Code Maturity/Exploitability “E”), and outputs a consolidated CSV (cvss-bt.csv).
File-Structure Analysis
• Top level:
• .github/workflows/epss.yml & cvss-bt.yml – CI/CD or scheduler definitions (GitHub Actions).
• code/ – core Python logic:
– process_nvd.py – fetches new EPSS scores, checks NVD for updates.
– enrich_nvd.py – applies the temporal metric logic and produces enriched records.
– requirements.txt – dependencies.
– last_run.txt – state marker for incremental fetches.
• cvss-bt.csv – final daily output.
• test.sh – local end-to-end test harness.
README Insights
• Describes the enrichment logic (mapping threat-intel sources to CVSS “E” values).
• Emphasizes daily schedule, data sources, score-version handling and thresholds.
• Lists external integrations (NVD API, EPSS API, CISA KEV, etc.).
a. Main Components
1. Scheduler / Orchestrator (GitHub Actions)
2. Ingestion Layer
• EPSS Fetcher
• NVD Fetcher
• Threat-Intel Adapters (CISA KEV, VulnCheck, Metasploit, Nuclei, ExploitDB, PoC-in-GitHub)
3. Processing Layer
• process_nvd.py (merge EPSS + NVD)
• enrich_nvd.py (calculate Temporal metric “E”)
4. Persistence
• State Store: last_run.txt
• Output Store: cvss-bt.csv
5. Test Harness (test.sh for local verification)
6. External Services/APIs
b. Relationships & Data Flow
• Scheduler → runs process_nvd.py → fetch EPSS → query NVD (if new EPSS) → merge base scores
• process_nvd.py → writes intermediate state / updates last_run.txt
• Scheduler → runs enrich_nvd.py → calls threat-intel adapters → computes “E” value per CVE
• enrich_nvd.py → writes final cvss-bt.csv
• test.sh simulates the above locally
c. Architectural Patterns & Principles
• Batch ETL pattern: clear separation of ingestion vs. transformation vs. storage
• Single-responsibility scripts for maintainability
• Idempotent scheduling (uses last_run.txt to avoid duplicate work)
• Config-as-code for workflows (GitHub Actions YAML)
d. Key Technologies & Libraries
• Python 3.x + requests / HTTP client libraries
• GitHub Actions for orchestration
• CSV file I/O
• External REST APIs (NVD, EPSS, various OSINT feeds)