How beam-cloud/beta9 works
Main Components
a) Client Layer:
• Python SDK (sdk/src/beta9)
• CLI Interface (sdk/src/beta9/cli)
• HTTP/gRPC APIs
b) Control Plane:
• Gateway Service (cmd/gateway)
• Scheduler Service (pkg/scheduler)
• Authentication System (pkg/auth)
c) Data Plane:
• Worker Nodes (cmd/worker)
• Container Runtime (pkg/worker)
• Storage System (pkg/storage)
d) Infrastructure Components:
• PostgreSQL Database (for persistent storage)
• Redis (for caching and queues)
• Kubernetes (as orchestration layer)
Key Relationships to Highlight
a) Control Flow:
• Client → Gateway → Scheduler → Worker nodes
• Authentication flow through all components
• Task scheduling and distribution paths
b) Data Flow:
• Storage system interactions
• Container image distribution
• Task queue processing
Architectural Patterns to Emphasize
a) Microservices Architecture:
• Clear separation between gateway, scheduler, and worker services
• Service communication via gRPC (evident from .proto files)
b) Event-Driven System:
• Event processing flows (pkg/repository/events.go)
• Queue-based communication patterns
Specific Implementation Details
a) Worker Management:
• GPU resource allocation
• Autoscaling mechanisms
• Container lifecycle management
b) Security Layer:
• WireGuard encryption
• Tailscale integration
• Authentication/Authorization flow
Component Grouping
Related components into logical zones:
• Client Layer |
• [Python SDK] [CLI] [HTTP/gRPC APIs] |
• Control Plane |
• [Gateway Service] ←→ [Scheduler Service] ←→ [Authentication] |
• Data Plane |
• [Worker Nodes] [Container Runtime] [Storage System] |
• Infrastructure Layer |
• [PostgreSQL] [Redis] [Kubernetes] |
Technical Details to Note
• Protocol details (gRPC/HTTP) between components
• Persistent storage locations
• Security boundaries and encryption points
• Scaling boundaries and load balancing points