# apuslabs/hb-os architecture

> Architecture diagram of the GitHub repository apuslabs/hb-os, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apuslabs/hb-os
- Repository: https://github.com/apuslabs/hb-os

## Overview

##### Project type and purpose
- HyperBEAM OS is a command-line automation tool (written mostly in Python with some Rust components) that orchestrates the full lifecycle of building, configuring, attesting, and running SEV-SNP–enabled virtual machines via QEMU. It replaces Makefile tasks with a more readable, script-based pipeline.

##### File structure
- Top-level “run” script: the CLI entry point.
- `src/`: Python modules implementing each stage of the pipeline (environment setup, dependency installation, initramfs build, VM image creation, guest content, etc.).
- `config/`: static JSONC/TOML templates and a Python config loader (`config.py`).
- `scripts/` & `launch.sh`: supplemental shell or Python scripts invoked as part of certain targets (e.g., `node_api.py`, post-start hooks).
- `tools/`: two Rust projects (`attestation_server` and `digest_calc`) that are built and invoked by the Python code.
- `resources/`: Dockerfiles, systemd service unit files, initramfs templates used when building content images.
- `examples/`: sample VM configuration files showing how to invoke QEMU or template new VMs.
- `run` orchestrates all of the above via named targets (init, setup_host, build_base_image, build_guest_image, start, package_release, etc.).

##### README for additional insights
- README describes a staged pipeline:
- init (environment + binary builds)
- setup_host (install SNP releases on the host)
- build_base_image (kernel unpack + initramfs)
- build_guest_image (Docker → guest content + dm-verity)
- start (launch via QEMU)
- package/download/start_release (release artifact lifecycle)
- clean
- It calls out external dependencies (QEMU, Docker, SNP firmware) and the two Rust tools.

a. Main components (suggest grouping into logical layers):
- CLI Layer: `run` script & `launch.sh`
- Configuration Layer: `config/config.py`, JSONC/TOML templates, `examples/`
- Orchestration Layer (Python): modules in `src/` (e.g., `dependencies.py`, `build_initramfs.py`, `create_new_vm.py`, `setup_guest.py`, etc.)
- Tooling Layer:
- Rust binaries: `attestation_server` & `digest_calc` (in `tools/`)
- Shell scripts: `scripts/*.sh` & `scripts/node_api.py`
- Build Artifacts: kernel package, initramfs image, VM disk images, DM-verity hash tree, release tarballs
- Execution Layer: QEMU (+ SNP hardware via KVM) and Docker for building guest content
- External Services: SEV-SNP firmware, host OS packages, Docker Daemon, QEMU hypervisor

b. Relationships and interactions
- The CLI invokes Python orchestration modules in sequence based on the target.
- Python modules load configuration, call shell scripts, build or invoke Rust binaries, produce images and config files.
- The Docker daemon is called from `build_content.py` (via `content.Dockerfile`) to assemble guest rootfs and generate DM-verity.
- QEMU is launched via `launch.sh` (or direct Python call) with the generated VM configuration.
- `attestation_server` binaries are built early in the init stage, then used to perform attestation tasks.
- `digest_calc` is used by Python to generate measurement inputs for the guest.
- Packaging steps collect all artifacts into a release folder and bundle them with a tar.gz.

c. Important architectural patterns
- Modular “target” pattern: each CLI target is its own Python module under `src/`.
- Pipeline staging: linear data flow from raw kernel → initramfs → base image → guest image → QEMU.
- Side-tool integration: Rust-based tools are first-class citizens in the build pipeline.
- Configuration-as-code: all parameters come from `config/config.py` + JSONC/TOML templates.

d. Relevant technologies/frameworks/libraries
- Python 3 scripts for orchestration
- Rust (Cargo) for `attestation_server` and `digest_calc`
- Docker (via content.Dockerfile)
- QEMU/KVM for VM execution
- SEV-SNP hardware features via Linux KVM module
- Systemd service files in `resources/`

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        CLI_run["run"]:::python
        CLI_launch["launch.sh"]:::python
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        Config_loader["config/config.py"]:::config
        Config_jsonc["server.jsonc"]:::config
        Config_template["template-user-data"]:::config
        Config_vm_toml["vm-config-template.toml"]:::config
        Examples_jsonc["examples/servers/*.jsonc"]:::config
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer (Python Modules)"
        dep["dependencies.py"]:::python
        build_initramfs["build_initramfs.py"]:::python
        build_content["build_content.py"]:::python
        create_vm["create_new_vm.py"]:::python
        create_config["create_vm_config.py"]:::python
        setup_guest["setup_guest.py"]:::python
    end

    %% Tooling Layer
    subgraph "Tooling Layer"
        attestation["attestation_server"]:::rust
        digest_calc["digest_calc"]:::rust
        shell_base["base_setup.sh"]:::python
        shell_init["init.sh"]:::python
        post_start["post_start.py"]:::python
        node_api["node_api.py"]:::python
    end

    %% Resources
    subgraph "Resources"
        df_init["initramfs.Dockerfile"]:::config
        df_content["content.Dockerfile"]:::config
        unit_cu_srv["cu.service"]:::config
        unit_cu_env["cu.env"]:::config
        unit_hb_srv["hyperbeam.service"]:::config
        unit_hb_conf["config.flat"]:::config
    end

    %% External Services
    subgraph "External Services"
        docker["Docker Daemon"]:::external
        qemu["QEMU/KVM"]:::external
        firmware["SEV-SNP firmware"]:::external
    end

    %% Build Artifacts
    subgraph "Build Artifacts"
        kernel_pkg["Kernel package"]:::artifact
        initramfs_img["Initramfs image"]:::artifact
        base_img["Base VM image"]:::artifact
        guest_img["Guest VM image"]:::artifact
        dmverity["DM-verity hash tree"]:::artifact
        release["Release tarball"]:::artifact
    end

    %% Connections
    CLI_run --> dep
    CLI_run --> build_initramfs
    CLI_run --> build_content
    CLI_run --> create_vm
    CLI_run --> create_config
    CLI_run --> setup_guest
    CLI_run --> attestation
    CLI_run --> digest_calc
    CLI_run --> release

    CLI_launch --> qemu

    dep --> shell_base
    dep --> shell_init
    dep --> digest_calc

    build_initramfs --> df_init
    build_initramfs --> initramfs_img

    build_content --> df_content
    build_content --> docker
    docker --> guest_img
    docker --> dmverity

    create_vm --> base_img
    create_config --> base_img

    setup_guest --> post_start
    setup_guest --> node_api

    %% Config loading
    dep --> Config_loader
    build_initramfs --> Config_loader
    build_content --> Config_loader
    create_vm --> Config_loader
    create_config --> Config_loader
    setup_guest --> Config_loader

    Config_loader --> Config_jsonc
    Config_loader --> Config_template
    Config_loader --> Config_vm_toml
    Config_loader --> Examples_jsonc

    %% QEMU execution
    qemu --> firmware
    qemu --> kernel_pkg
    qemu --> initramfs_img
    qemu --> base_img
    qemu --> guest_img
    qemu --> dmverity

    %% Packaging
    kernel_pkg --> release
    initramfs_img --> release
    base_img --> release
    guest_img --> release
    dmverity --> release

    %% Click Events
    click CLI_run "https://github.com/apuslabs/hb-os/tree/main//run"
    click CLI_launch "https://github.com/apuslabs/hb-os/blob/main//launch.sh"
    click Config_loader "https://github.com/apuslabs/hb-os/blob/main//config/config.py"
    click Config_jsonc "https://github.com/apuslabs/hb-os/blob/main//config/server.jsonc"
    click Config_template "https://github.com/apuslabs/hb-os/tree/main//config/template-user-data"
    click Config_vm_toml "https://github.com/apuslabs/hb-os/blob/main//examples/vm-config-template.toml"
    click Examples_jsonc "https://github.com/apuslabs/hb-os/tree/main//examples/servers/"
    click dep "https://github.com/apuslabs/hb-os/blob/main//src/dependencies.py"
    click build_initramfs "https://github.com/apuslabs/hb-os/blob/main//src/build_initramfs.py"
    click build_content "https://github.com/apuslabs/hb-os/blob/main//src/build_content.py"
    click create_vm "https://github.com/apuslabs/hb-os/blob/main//src/create_new_vm.py"
    click create_config "https://github.com/apuslabs/hb-os/blob/main//src/create_vm_config.py"
    click setup_guest "https://github.com/apuslabs/hb-os/blob/main//src/setup_guest.py"
    click attestation "https://github.com/apuslabs/hb-os/tree/main//tools/attestation_server/"
    click digest_calc "https://github.com/apuslabs/hb-os/tree/main//tools/digest_calc/"
    click shell_base "https://github.com/apuslabs/hb-os/blob/main//scripts/base_setup.sh"
    click shell_init "https://github.com/apuslabs/hb-os/blob/main//scripts/init.sh"
    click post_start "https://github.com/apuslabs/hb-os/blob/main//scripts/post_start.py"
    click node_api "https://github.com/apuslabs/hb-os/blob/main//scripts/node_api.py"
    click df_init "https://github.com/apuslabs/hb-os/blob/main//resources/initramfs.Dockerfile"
    click df_content "https://github.com/apuslabs/hb-os/blob/main//resources/content.Dockerfile"
    click unit_cu_srv "https://github.com/apuslabs/hb-os/blob/main//resources/cu/cu.service"
    click unit_cu_env "https://github.com/apuslabs/hb-os/blob/main//resources/cu/cu.env"
    click unit_hb_srv "https://github.com/apuslabs/hb-os/blob/main//resources/hyperbeam/hyperbeam.service"
    click unit_hb_conf "https://github.com/apuslabs/hb-os/blob/main//resources/hyperbeam/config.flat"

    %% Styles
    classDef python fill:#BBDEFB,stroke:#1976D2,color:#0D47A1
    classDef rust fill:#FFCC80,stroke:#EF6C00,color:#E65100
    classDef external fill:#C8E6C9,stroke:#388E3C,color:#1B5E20
    classDef config fill:#CFD8DC,stroke:#455A64,color:#263238
    classDef artifact fill:#E1BEE7,stroke:#8E24AA,color:#4A148C
```
