How datadog/guarddog works
Project Overview and Purpose
• Recognize that GuardDog is a CLI-based security scanning tool. Its purpose is to detect malicious packages by running a variety of heuristics on both package metadata and source code.
• It targets multiple ecosystems (PyPI, npm, Go, GitHub Actions) and leverages Semgrep and Yara rules to identify potential threats.
Identifying the Main Components from the File Tree and README
• Command Line Interface (CLI) and Entry Point:
• Main entry in guarddog/__main__.py and cli.py.
• This component handles user commands, parsing arguments, and dispatching actions.
• Analyzers:
• Located under guarddog/analyzer, split into two primary areas:
▪ Source Code Heuristics (guarddog/analyzer/sourcecode) – uses Semgrep/Yara rules.
▪ Package Metadata Heuristics (guarddog/analyzer/metadata) – specific checks for PyPI, npm, Go, and GitHub Actions.
• Scanners:
• Under guarddog/scanners, which includes scanners tailored for each ecosystem (e.g., pypi_package_scanner.py, npm_package_scanner.py, etc.). These modules are responsible for obtaining the package data (either locally or remotely) and initiating scans.
• Reporters:
• For instance, guarddog/reporters/sarif.py generates output (SARIF, JSON, etc.).
• External Integrations and Interactions:
• Components that interact with external package registries (using HTTP calls, downloads) for fetching packages.
• Custom Rules integration found under guarddog/analyzer/sourcecode and configuration for importing additional Semgrep/Yara rules.
• Utilities & Configuration:
• Located in guarddog/utils (e.g., config, package_info, archives) managing common tasks.
• Testing, Documentation, and Deployment:
• Test directory structure (tests) along with scripts and Docker-related files that aid in CI/CD and containerized deployments.
Relationships and Data Flows
• The CLI is the user-facing interface. When a scan or verify command is issued:
• The CLI parses input and delegates to a corresponding scanner module.
• The Scanner module retrieves the package (locally or from a remote location) and passes the package data to the Analyzer.
• Depending on the package type, the Analyzer then uses specific heuristics (either under sourcecode or metadata) to analyze the package.
• After scanning:
• The Reporter component formats the analysis results (e.g., as JSON or SARIF).
• Optional external interactions:
• Downloading packages from external registries (PyPI, npm, GitHub, etc.) and integration with CI (as shown in the GitHub Action YAML snippet in the README).
Architectural Patterns and Design Principles
• Modular Design: Each component (CLI, scanners, analyzers, reporters) is distinctly separated, facilitating modularity and ease of extension (for example, adding new heuristics or supporting new ecosystems).
• Separation of Concerns: Different directories (analyzer vs. scanners vs. reporters) imply a clean separation where each is responsible for a specific aspect of the scanning process.
• Pipeline Flow: User command → Package Fetch → Analysis (branching into sourcecode/metadata heuristics) → Reporting output.