How safedep/vet works
Project type and purpose
• This is a monolithic Go-based open-source CLI application (“vet”) with complementary server (“MCP Server”) and agent components.
• Purpose: enterprise-grade software supply-chain security—dependency scanning, malicious package detection, policy enforcement, SBOM support, reporting, and AI-driven query/analysis.
File structure to extract architectural layers
• Entrypoints (cmd/, main.go): CLI commands (vet scan, vet query, vet server mcp, vet agent …).
• Core engine (pkg/ + internal/ + connect.go): modules for reading inputs, parsing/manipulating SBOMs and code, policy evaluation, vulnerability & malware analysis, reporting, storage abstractions, telemetry.
• Persistence (ent/ + pkg/storage/ent_sqlite.go): SQLite via Ent ORM plus optional graph backends.
• API definitions (api/, gen/): Protocol Buffers & OpenAPI specs for inter-component & cloud communications.
• Server (mcp/): implements MCP server for streaming scan/query results over SSE or stdio.
• Agents (agent/): AI agents that consume MCP server streams and interact with LLMs.
• External integration: SafeDep Cloud (malware-analysis API), GitHub/GitLab, container registries, OSV, Scorecard, telemetry endpoints.
a. Main components
• User interfaces
• CLI Frontend (cmd/…)
• Agent Frontend (agent/)
• Core Engine
• Readers (pkg/readers)
• Parsers (pkg/parser)
• Analyzer & Policy Engine (pkg/analyzer, pkg/policy)
• Vulnerability/Malware Scanner (pkg/scanner + pkg/cloud)
• Reporter (pkg/reporter)
• Persistence
• Ent/SQLite (ent/, pkg/storage)
• MCP Server (mcp/)
• External Services
• SafeDep Cloud API
• GitHub/GitLab APIs
• Container registries
• OSV, Scorecard
• Telemetry endpoint
b. Relationships & data flows
1. User → CLI: user invokes vet scan or vet server mcp.
2. CLI → Readers: reads directory, manifest, SBOM, image.
3. Readers → Parser: transforms raw inputs into internal model.
4. Parser → Analyzer/Policy: computes usage evidence, applies CEL filters.
5. Analyzer → Scanner: calls local vulnerability DB or SafeDep Cloud for malware analysis.
6. Scanner → Persistence: stores scan data if query-mode enabled (--json-dump-dir) or into Ent/SQLite for MCP server.
7. Persistence → MCP Server: streams results on SSE or stdio.
8. Reporter → User: outputs console, JSON, SARIF, CSV, Markdown, SBOMs.
9. Agent → MCP Server: subscribes to SSE stream, calls LLM via agent/llm.go to generate reports or suggestions.
10. Telemetry: Core engine emits anonymized usage events to analytics endpoint.
c. Architectural patterns & design principles
• Layered architecture: clear separation of UI (CLI/agent), core engine (reader/parser/analyzer/scanner/reporter), persistence, server, external integrations.
• Plug-in style extensibility: readers, parsers, reporters, scanners as interchangeable modules.
• Policy as code: CEL expressions evaluated within policy engine.
• Streaming server: MCP server uses SSE/stdio for bi-directional streaming.
• ORM abstraction: Ent for relational storage; optional graph backends.
d. Key technologies & libraries
• Go modules; Cobra/Viper for CLI
• Protocol Buffers (gRPC) for API specs
• Ent ORM + SQLite
• CEL (Common Expression Language) for policies
• SafeDep Cloud HTTP API
• LLM integrations for AI agents
Detail
• Stage: input, normalization, analysis, storage, reporting, streaming, AI-driven queries.
• Don’t over-aggregate: show Readers vs. Parsers vs. Analyzers vs. Scanners vs. Reporters separately.
• Both local (SQLite/Ent) and remote (SafeDep Cloud, GitHub) interactions.
• Dual-mode operation: synchronous CLI pipelines vs. asynchronous MCP server streams.