How apuslabs/hb-os works
Project type and purpose
• HyperBEAM OS is a command-line automation tool (written mostly in Python with some Rust components) that orchestrates the full lifecycle of building, configuring, attesting, and running SEV-SNP–enabled virtual machines via QEMU. It replaces Makefile tasks with a more readable, script-based pipeline.
File structure
• Top-level “run” script: the CLI entry point.
• src/: Python modules implementing each stage of the pipeline (environment setup, dependency installation, initramfs build, VM image creation, guest content, etc.).
• config/: static JSONC/TOML templates and a Python config loader (config.py).
• scripts/ & launch.sh: supplemental shell or Python scripts invoked as part of certain targets (e.g., node_api.py, post-start hooks).
• tools/: two Rust projects (attestation_server and digest_calc) that are built and invoked by the Python code.
• resources/: Dockerfiles, systemd service unit files, initramfs templates used when building content images.
• examples/: sample VM configuration files showing how to invoke QEMU or template new VMs.
• run orchestrates all of the above via named targets (init, setup_host, build_base_image, build_guest_image, start, package_release, etc.).
README for additional insights
• README describes a staged pipeline:
• init (environment + binary builds)
• setup_host (install SNP releases on the host)
• build_base_image (kernel unpack + initramfs)
• build_guest_image (Docker → guest content + dm-verity)
• start (launch via QEMU)
• package/download/start_release (release artifact lifecycle)
• clean
• It calls out external dependencies (QEMU, Docker, SNP firmware) and the two Rust tools.
a. Main components (suggest grouping into logical layers):
• CLI Layer: run script & launch.sh
• Configuration Layer: config/config.py, JSONC/TOML templates, examples/
• Orchestration Layer (Python): modules in src/ (e.g., dependencies.py, build_initramfs.py, create_new_vm.py, setup_guest.py, etc.)
• Tooling Layer:
• Rust binaries: attestation_server & digest_calc (in tools/)
• Shell scripts: scripts/*.sh & scripts/node_api.py
• Build Artifacts: kernel package, initramfs image, VM disk images, DM-verity hash tree, release tarballs
• Execution Layer: QEMU (+ SNP hardware via KVM) and Docker for building guest content
• External Services: SEV-SNP firmware, host OS packages, Docker Daemon, QEMU hypervisor
b. Relationships and interactions
• The CLI invokes Python orchestration modules in sequence based on the target.
• Python modules load configuration, call shell scripts, build or invoke Rust binaries, produce images and config files.
• The Docker daemon is called from build_content.py (via content.Dockerfile) to assemble guest rootfs and generate DM-verity.
• QEMU is launched via launch.sh (or direct Python call) with the generated VM configuration.
• attestation_server binaries are built early in the init stage, then used to perform attestation tasks.
• digest_calc is used by Python to generate measurement inputs for the guest.
• Packaging steps collect all artifacts into a release folder and bundle them with a tar.gz.
c. Important architectural patterns
• Modular “target” pattern: each CLI target is its own Python module under src/.
• Pipeline staging: linear data flow from raw kernel → initramfs → base image → guest image → QEMU.
• Side-tool integration: Rust-based tools are first-class citizens in the build pipeline.
• Configuration-as-code: all parameters come from config/config.py + JSONC/TOML templates.
d. Relevant technologies/frameworks/libraries
• Python 3 scripts for orchestration
• Rust (Cargo) for attestation_server and digest_calc
• Docker (via content.Dockerfile)
• QEMU/KVM for VM execution
• SEV-SNP hardware features via Linux KVM module
• Systemd service files in resources/