How aquasecurity/trivy-db works
Project Type and Purpose
• This is a Go-based CLI tool *and* library (“trivy-db”) for building and manipulating a vulnerability database (SQLite).
• The CLI (cmd/trivy-db/main.go) drives the build process; the library packages (under pkg/) encapsulate fetching vulnerability data from various upstream sources, database construction, and utility functions.
High-Level Components
a. CLI Entry Point
• Component: trivy-db binary (built from cmd/trivy-db)
• Responsibilities: parse commands (build, etc.), kick off data fetch, invoke DB build/compact/compress routines.
b. Core Library
• pkg/app.go / pkg/build.go: orchestration logic
• pkg/db: abstraction over SQLite operations (create tables, insert advisories/vulns)
• pkg/vulnsrc: a family of fetcher modules (e.g., pkg/vulnsrc/nvd, pkg/vulnsrc/redhat-oval, etc.)
• Each implements a common interface to download and parse vendor-specific advisory feeds
• pkg/vulndb: staging/load test fixtures and DB migrations
• pkg/metadata: manage metadata.json (version, update interval)
• pkg/utils: helper routines (file I/O, progress bars, string/int utilities)
• pkg/log: centralized logging interface
c. External Services
• Upstream vulnerability data sources (NVD API, GitHub Advisory, Linux distro OVAL or JSON feeds)
• Container registry (GitHub Container Registry) for distribution
d. CI/CD Pipeline
• GitHub Actions workflows under .github/workflows/cron.yml: schedule builds every 6 h
• Makefile scripts orchestrate fetch→build→compact→compress steps
• ORAS CLI pushes artifacts (db.tar.gz) to GHCR
Interactions and Data Flow
1. Scheduler (GitHub Actions Cron) → triggers → 2. Makefile
2. Makefile → invokes → CLI binary (trivy-db build)
3. CLI → calls → pkg/app → enumerates all pkg/vulnsrc/* modules
4. Each Fetcher Module → fetch data from an External Source → parse into unified vulnerability structs
5. Aggregated data → passed to pkg/db → writes to a local SQLite database file (trivy.db) and metadata file
6. pkg/db-compact (Makefile) → optimize/compact the SQLite file
7. pkg/db-compress (Makefile) → gzip/tar the final DB artifact
8. ORAS CLI → pushes the compressed DB to GHCR
9. Clients (e.g., Trivy scanner) → pull the DB via ORAS or Trivy CLI
Architectural Patterns and Design Principles
• Modular Fetchers: each vendor or feed parser (pkg/vulnsrc/*) implements a shared interface for consistency.
• Separation of Concerns:
• Fetching/parsing (vulnsrc)
• DB operations (pkg/db)
• Metadata management (pkg/metadata)
• CLI orchestration (pkg/app)
• Dependency Injection / Interfaces: low-coupling between fetch modules and DB writer (e.g., inject parsed records into the builder).
Critical Details to Capture
• The many independent pkg/vulnsrc fetchers feeding into a single DB builder.
• The SQLite database as the core product of the build.
• The periodic CI/CD workflow that automates fetch→build→publish.
• How client tools (Trivy) consume the published DB artifact.