How perplexityai/bumblebee works
1. Bumblebee is a single-purpose Go CLI that runs as a static binary on macOS and Linux. Its architecture is intentionally read-only: it inspects local metadata files, never invokes package managers, and never reads source code or performs network lookups during a scan.
2. The executable entrypoint lives under cmd/bumblebee. It exposes a small command surface: scan for inventory collection, roots for previewing resolved scan roots, version for build provenance, and selftest for an embedded end-to-end smoke test.
3. The scan model is one-shot. Bumblebee performs a single filesystem pass and exits; scheduling is external. Scan behavior is shaped by profile (baseline, project, deep) plus optional flags such as --root, --ecosystem, --exposure-catalog, and --findings-only.
4. Root resolution is a distinct boundary. roots.go and internal/walk determine which directories are eligible based on profile and root kind. baseline and project use curated global/user/project locations; deep is the only mode allowed to walk arbitrary explicit roots, including broad roots like $HOME.
5. Filesystem traversal is centralized in internal/walk. This layer is responsible for enumerating candidate metadata files and directory keys, while keeping the scanner focused on interpretation rather than discovery. The read surface is limited to inventory sources listed in the docs: lockfiles, installed package metadata, extension manifests, and supported MCP JSON configs.
6. Ecosystem-specific parsing is isolated in internal/ecosystem/*. Each subpackage handles one family of metadata and maps it into the shared model: npm/pnpm/Yarn/Bun, PyPI, Go modules, RubyGems, Composer, MCP configs, editor extensions, and browser extensions. This separation is the main parsing boundary in the repo.
7. Normalization and shared data modeling live in internal/model and internal/normalize. These packages canonicalize package names, versions, ecosystems, source types, and confidence levels so that records are comparable across scans. Stable record_id values are content-addressed from canonical identity fields, which is important for deduplication and current-state tracking downstream.
8. Endpoint identity is captured separately in internal/endpoint. Every emitted record carries host-level context such as hostname, OS, architecture, username, UID, and optional device ID. This makes records suitable for fleet correlation without needing an external device inventory service.
9. Exposure matching is an optional second pass in internal/exposure. When given one or more catalog files, Bumblebee performs exact (ecosystem, package name, version) matching against the scanned inventory and emits finding records with severity, catalog identifiers, and evidence. Catalog loading supports multiple JSON files in a directory, but only if they share the same supported schema version.
10. Output is split between normal records and diagnostics. internal/output and sink.go handle NDJSON emission, with package/finding/summary records going to the main output stream and diagnostics going to stderr. The run always ends with a scan_summary record so downstream receivers can decide whether to promote the run to current state.
11. The transport boundary is intentionally outside the scanner core. The docs describe file and HTTPS sinks plus a receiver-side current-state model, which means Bumblebee’s job is to produce structured NDJSON and not to manage persistence or fleet ingestion itself.
12. selftest is a built-in validation path, not a production scan mode. It runs against embedded fixtures under cmd/bumblebee/selftest/fixtures, uses fake package names, and verifies that the installed binary can still detect the expected metadata without touching the network.
13. The repository also contains versioned schemas and threat-intel inputs. docs/schema/v0.1.0 defines the wire format for package, finding, exposure catalog, diagnostics, and summary records, while threat_intel/ holds maintained catalog JSON files that can be fed into scans as prebuilt advisory inputs.
Main entry points: walk.go Walk · model.go Model · scanner.go Scanner · main.go Main
Components 27
CLI
Executable entrypoint and user-facing commands
- Main · Bootstraps the command surface and dispatches subcommands
cmd/bumblebee/main.go - Roots · Resolves scan roots for preview and scan planning
cmd/bumblebee/roots.go - Selftest · Runs fixture-based end-to-end validation against the installed binary
cmd/bumblebee/selftest.go - Version · Emits build and version provenance
cmd/bumblebee/version.go
Scan core
Root resolution, traversal, parsing, normalization, and findings
- Scanner · Coordinates one-shot inventory collection and optional exposure matching
internal/scanner/scanner.go - Walk · Enumerates eligible directories and candidate metadata files
internal/walk/walk.go - Dir keys · Platform-specific directory key logic used during traversal
internal/walk/dirkey_unix.go - Model · Defines shared package, finding, summary, and record structures
internal/model/model.go - Normalize · Canonicalizes names, versions, ecosystems, and confidence levels
internal/normalize/normalize.go - Endpoint · Captures host-level context attached to every emitted record
internal/endpoint/endpoint.go - Exposure · Matches scanned inventory against advisory catalogs to produce findings
internal/exposure/exposure.go - npm · Parses npm and related JavaScript package metadata
internal/ecosystem/npm/npm.go - pnpm · Parses pnpm lock and package metadata
internal/ecosystem/pnpm/pnpm.go - Yarn · Parses Yarn lock and package metadata
internal/ecosystem/yarn/yarn.go - Bun · Parses Bun package metadata
internal/ecosystem/bun/bun.go - PyPI · Parses installed Python package metadata
internal/ecosystem/pypi/pypi.go - Go mod · Parses Go module metadata
internal/ecosystem/gomod/gomod.go - RubyGems · Parses installed Ruby gem metadata
internal/ecosystem/rubygems/rubygems.go - Composer · Parses PHP Composer metadata
internal/ecosystem/composer/composer.go - MCP · Parses supported MCP JSON configs
internal/ecosystem/mcp/mcp.go - Editor ext · Parses editor extension manifests
internal/ecosystem/editorext/editorext.go - Browser ext · Parses browser extension manifests
internal/ecosystem/browserext/browserext.go
Emission
Record shaping and output sinks
Schemas & intel
Wire schemas and maintained threat-intel inputs
- Schemas · Versioned JSON schemas for package, finding, exposure catalog, diagnostic, and summary records
docs/schema/v0.1.0 - Threat intel · Maintained advisory catalog JSON inputs usable during scans
threat_intel
Connections 40
- Main → Roots: scan roots · CLI delegates root preview and root selection to the resolver
- Main → Scanner: scan · CLI dispatches the one-shot scan pipeline
- Main → Selftest: selftest · CLI exposes the embedded validation path
- Main → Version: version · CLI exposes build provenance
- Roots → Walk: eligibility · Root resolution feeds the traversal boundary
- Scanner → Walk: discover · Scanner asks walk to enumerate candidate metadata sources
- Scanner → Model: records · Scanner produces shared inventory and finding records
- Scanner → Normalize: canonicalize · Scanner normalizes identities before emission and matching
- Scanner → Endpoint: host context · Scanner attaches endpoint identity to emitted records
- Scanner → Exposure: optional match · Scanner performs a second-pass advisory match when catalogs are provided
- Walk → Dir keys: dir keys · Traversal uses platform-specific directory key logic
- Walk → npm: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → pnpm: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Yarn: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Bun: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → PyPI: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Go mod: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → RubyGems: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Composer: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → MCP: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Editor ext: metadata · Traversal surfaces metadata for ecosystem parsers
- Walk → Browser ext: metadata · Traversal surfaces metadata for ecosystem parsers
- npm → Model: map · Parser maps ecosystem-specific metadata into shared records
- pnpm → Model: map · Parser maps ecosystem-specific metadata into shared records
- Yarn → Model: map · Parser maps ecosystem-specific metadata into shared records
- Bun → Model: map · Parser maps ecosystem-specific metadata into shared records
- PyPI → Model: map · Parser maps ecosystem-specific metadata into shared records
- Go mod → Model: map · Parser maps ecosystem-specific metadata into shared records
- RubyGems → Model: map · Parser maps ecosystem-specific metadata into shared records
- Composer → Model: map · Parser maps ecosystem-specific metadata into shared records
- MCP → Model: map · Parser maps ecosystem-specific metadata into shared records
- Editor ext → Model: map · Parser maps ecosystem-specific metadata into shared records
- Browser ext → Model: map · Parser maps ecosystem-specific metadata into shared records
- Normalize → Model: canonical ids · Normalization supports content-addressed record identity and deduplication
- Exposure → Model: findings · Exposure matching consumes inventory records and emits findings
- Exposure → Threat intel: catalogs · Matcher consumes maintained threat-intel catalog JSON inputs
- Output → Sink: emit · Output layer routes records and diagnostics to the CLI sink boundary
- Output → Schemas: format · Emission follows the versioned wire schemas
- Sink → HTTP sink: transport · Sink boundary aligns with file and HTTPS transport adapters
- Selftest → Schemas: fixtures · Selftest validates output shape against the embedded schema expectations