How trailofbits/it-depends works
Project type and purpose
• This is an open-source CLI tool (written in Python) that builds dependency graphs and SBOMs for arbitrary code repositories or package specs.
• It supports multiple ecosystems (Python, JavaScript, Rust, Go, C/C++), resolves native dependencies (via Ubuntu’s apt metadata or Docker), and audits vulnerabilities against OSV.
File structure
• Top-level CLI and packaging:
– it_depends/__main__.py & it_depends/cli.py → entrypoint and argument parsing
– pyproject.toml, setup.py → build and packaging config
• Core modules under it_depends/:
– Language handlers: pip.py, npm.py, cargo.py, go.py, autotools.py, cmake.py
– Native/OS handlers: native.py, docker.py, ubuntu/apt.py, ubuntu/docker.py
– Resolution engine: resolver.py, dependencies.py
– Persistence/cache: db.py
– SBOM export: sbom.py, html.py
– Graph generation: graphs.py
– Audit integration: audit.py
– VCS scanning: vcs.py
• Tests in test/ validate each component; CI workflows automate lint, test, and publish.
README details
• Confirms multi-language support and reliance on external systems (npm, pip, cargo, apt-file, Docker, OSV).
• Mentions output formats: JSON, Graphviz/Dot, vis.js/HTML.
• Notes caching via local DB and parallelized tasks.
Main components and their interactions
– CLI Frontend
– Core Engine
• Command Parser (cli.py)
• Dependency Resolver (resolver.py)
• Language Adapters (pip, npm, cargo, go, autotools, cmake)
• Native/OS Adapter (native, docker, ubuntu)
• Database Cache (db.py)
– Audit Module (audit.py → OSV API)
– Exporters
• Graph Generator (graphs.py → Dot/vis.js)
• SBOM Generator (sbom.py → JSON/SPDX-in-dev)
• HTML Reporter (html.py)
– External Services
• Package Registries (PyPI, npm, crates.io, GOPROXY)
• OS Package Metadata (apt-file via Docker)
• Vulnerability DB (OSV)
1. User invokes CLI → passes flags to Command Parser
2. Parser invokes Core Engine → kicks off Resolution
3. Resolver calls appropriate Language Adapter(s) + Native Adapter → fetches manifest/deps
4. Resolver writes/reads cache in Database Cache
5. Full dependency graph flows into Exporters (Graph + SBOM + HTML)
6. If --audit passed → Graph/SBOM results flow into Audit Module → calls OSV → augments results
7. Final output → stdout/files
c. Architectural patterns & principles
– Plugin/Adapter pattern for each ecosystem (loose coupling)
– Single-process modular design, parallel task execution (concurrency with worker pools)
– Cache-aside pattern for immutable package metadata
d. Technologies & frameworks
– Python3 & pip
– SQLite (or similar) via db.py for caching
– Docker for native dependency resolution on non-Ubuntu hosts
– Graphviz/Dot or vis.js for visualization
– OSV REST API for auditing