# bitbomdev/minefield architecture

> Architecture diagram of the GitHub repository bitbomdev/minefield, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bitbomdev/minefield
- Repository: https://github.com/bitbomdev/minefield

## Overview

##### Project Overview
- Type & Purpose: Minefield is a command line and server-based tool designed for rapid, offline processing and analysis of SBOMs (Software Bill Of Materials). It makes heavy use of roaring Bitmaps for graphing dependencies. The project is optimized for speed, security (air-gapped operation), scalability, and customizability.
- Key Use Cases: Ingesting SBOM files, caching and indexing package dependency data, running complex queries (including leaderboard, custom queries, and dependency intersections), and visualizing query results.

##### Analyzing the File Structure
- Top-Level Directories/Files:
- "cmd": Contains all main command entrypoints such as “server”, “ingest”, “cache”, “query”, “leaderboard”, and “llm”.
- "api": Contains API definitions (proto files are here), signaling that the system exposes a programmatic (RPC/HTTP) interface.
- "pkg": Contains the core business logic modules:
· "graph": Implements the graph engine including parsing, caching, storage—for managing dependencies using roaring Bitmaps.
· "storages": Provides different storage implementations (e.g., Redis, SQL) that the system can use.
· "tools": Contains helper components for various ingestion methods (e.g., for sbom, scorecard, vulnerability analysis).
· "weightedNACD": Contains algorithms and risk calculation tools.
· "utils": Provides miscellaneous utility functions.
- "testdata" & "e2e": Test folders to validate ingestion and query functionality.
- Supporting files: Dockerfile, docker-compose.yml, Makefile, and buf configuration files indicate containerization and proto generation for API.

##### Identifying Main Components and Their Relationships
- Client/CLI Interface:
- Entry Point (main.go) that passes commands to various subcommands (server, ingest, cache, query, leaderboard).
- Commands are organized under the "cmd" directory.
- API Server:
- cmd/server: Hosts the API server which could be invoked to expose functionalities programmatically.
- Uses generated API code (under "api", "gen/api") and proto definitions.
- Ingestion Module:
- cmd/ingest: Handles the ingestion functionality. This is subdivided further for different types of data (OSV, SBOM, and Scorecard).
- This module accepts SBOM files or directories and processes them before feeding into the graph and storage subsystems.
- Caching Mechanism:
- cmd/cache: Provides commands for caching processed data.
- Works closely with the graph module (pkg/graph) and storage implementations (pkg/storages which support SQL and Redis).
- Query Module:
- cmd/query: Contains various query functions including custom queries, glob searches, metadata retrieval, etc.
- Interacts with the graph engine to return dependency relationships, intersect queries, and more (utilizing pkg/graph logic).
- Leaderboard Module:
- cmd/leaderboard: Generates ranked lists of packages based on dependency counts.
- Offers different flavors of leaderboard results (custom queries, key-based).
- Internal Libraries and Tools:
- cmd/helpers: Provides common utility functions used by different commands.
- pkg/tools: Contains ingestion helpers, SBOM parsing, vulnerability analysis, and risk calculation algorithms.
- Data Storage & External Dependencies:
- pkg/storages: Abstracts different storage mechanisms (Redis, SQL).
- The README and architecture indicate the system operates in air-gapped mode, emphasizing local processing without internet dependency.

##### Architectural Patterns & Design Principles
- Modularity: Code is highly modular with clearly separated commands (each implemented as a submodule under “cmd”) and internal libraries (under “pkg”), making the system extensible.
- Layered Architecture: There’s a clear separation between user commands (CLI/API), business logic (graphing, ingestion, caching), and persistence (storage modules).
- Dependency Injection: Likely practiced in components such as “cmd/server/wire.go” (using wire_gen.go) to hook different dependencies.
- Air-Gapped Philosophy: All components are designed to function without external network calls, emphasizing local and secure data processing.
- Use of Protocol Buffers: Protobuf definitions under “api” and generated API code in “gen/api” indicate service communication might be based on RPC or gRPC.

- Additional Annotations: Optionally include notes such as “Air-Gapped Operation” near all modules to reinforce the secure, offline design philosophy.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Interface Layer
    subgraph "Interface Layer"
        CLI["CLI/Command Interface"]:::interface
        API["API Server"]:::interface
    end

    %% Processing Layer
    subgraph "Processing Layer"
        INJ["Ingestion Module"]:::processing
        TOOLS["Internal Libraries & Tools"]:::processing
        GRAPH["Graph Processing Engine"]:::processing
        CACHE["Caching Mechanism"]:::processing
    end

    %% Query & Data Persistence Layer
    subgraph "Query & Data Persistence Layer"
        QUERY["Query Module"]:::query
        LB["Leaderboard Module"]:::query
        STORAGE["Data Storage Layer"]:::storage
    end

    %% Data Flow
    CLI -->|"invokes"| INJ
    CLI -->|"triggers"| QUERY
    CLI -->|"triggers"| LB
    API -->|"interfaces"| GRAPH
    INJ -->|"processes_SBOM"| GRAPH
    GRAPH -->|"utilizes"| CACHE
    CACHE -->|"persists_to"| STORAGE
    GRAPH -->|"supports"| QUERY
    GRAPH -->|"supports"| LB
    TOOLS -->|"assists"| INJ
    TOOLS -->|"enhances"| GRAPH

    %% Styles
    classDef interface fill:#FFDDC1,stroke:#BF5700,stroke-width:2px;
    classDef processing fill:#C1FFD7,stroke:#2E8B57,stroke-width:2px;
    classDef query fill:#C1D4FF,stroke:#1E90FF,stroke-width:2px;
    classDef storage fill:#FFFACD,stroke:#DAA520,stroke-width:2px;

    %% Click Events
    click CLI "https://github.com/bitbomdev/minefield/blob/main/main.go"
    click CLI "https://github.com/bitbomdev/minefield/blob/main/cmd/root/root.go"
    click API "https://github.com/bitbomdev/minefield/blob/main/cmd/server/server.go"
    click API "https://github.com/bitbomdev/minefield/blob/main/cmd/server/wire.go"
    click API "https://github.com/bitbomdev/minefield/tree/main/api"
    click API "https://github.com/bitbomdev/minefield/tree/main/gen/api"
    click INJ "https://github.com/bitbomdev/minefield/tree/main/cmd/ingest"
    click INJ "https://github.com/bitbomdev/minefield/tree/main/cmd/ingest/osv"
    click INJ "https://github.com/bitbomdev/minefield/tree/main/cmd/ingest/sbom"
    click INJ "https://github.com/bitbomdev/minefield/tree/main/cmd/ingest/scorecard"
    click GRAPH "https://github.com/bitbomdev/minefield/tree/main/pkg/graph"
    click CACHE "https://github.com/bitbomdev/minefield/tree/main/cmd/cache"
    click CACHE "https://github.com/bitbomdev/minefield/blob/main/pkg/graph/cache.go"
    click QUERY "https://github.com/bitbomdev/minefield/tree/main/cmd/query"
    click LB "https://github.com/bitbomdev/minefield/tree/main/cmd/leaderboard"
    click LB "https://github.com/bitbomdev/minefield/tree/main/cmd/leaderboard/keys"
    click LB "https://github.com/bitbomdev/minefield/tree/main/cmd/leaderboard/custom"
    click TOOLS "https://github.com/bitbomdev/minefield/tree/main/cmd/helpers"
    click TOOLS "https://github.com/bitbomdev/minefield/tree/main/pkg/tools"
    click TOOLS "https://github.com/bitbomdev/minefield/tree/main/pkg/tools/weightedNACD"
    click STORAGE "https://github.com/bitbomdev/minefield/tree/main/pkg/storages"
```
