# beam-cloud/beta9 architecture

> Architecture diagram of the GitHub repository beam-cloud/beta9, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/beam-cloud/beta9
- Repository: https://github.com/beam-cloud/beta9

## Overview

##### Main Components

a) Client Layer:
- Python SDK (sdk/src/beta9)
- CLI Interface (sdk/src/beta9/cli)
- HTTP/gRPC APIs

b) Control Plane:
- Gateway Service (cmd/gateway)
- Scheduler Service (pkg/scheduler)
- Authentication System (pkg/auth)

c) Data Plane:
- Worker Nodes (cmd/worker)
- Container Runtime (pkg/worker)
- Storage System (pkg/storage)

d) Infrastructure Components:
- PostgreSQL Database (for persistent storage)
- Redis (for caching and queues)
- Kubernetes (as orchestration layer)

##### Key Relationships to Highlight

a) Control Flow:
- Client → Gateway → Scheduler → Worker nodes
- Authentication flow through all components
- Task scheduling and distribution paths

b) Data Flow:
- Storage system interactions
- Container image distribution
- Task queue processing

##### Architectural Patterns to Emphasize

a) Microservices Architecture:
- Clear separation between gateway, scheduler, and worker services
- Service communication via gRPC (evident from .proto files)

b) Event-Driven System:
- Event processing flows (pkg/repository/events.go)
- Queue-based communication patterns

##### Specific Implementation Details

a) Worker Management:
- GPU resource allocation
- Autoscaling mechanisms
- Container lifecycle management

b) Security Layer:
- WireGuard encryption
- Tailscale integration
- Authentication/Authorization flow

##### Component Grouping

Related components into logical zones:

- Client Layer                                         |
- [Python SDK] [CLI] [HTTP/gRPC APIs]                                                   |

- Control Plane                                          |
- [Gateway Service] ←→ [Scheduler Service] ←→ [Authentication]                          |

- Data Plane                                            |
- [Worker Nodes] [Container Runtime] [Storage System]                                    |

- Infrastructure Layer                                       |
- [PostgreSQL] [Redis] [Kubernetes]                                                     |

##### Technical Details to Note

- Protocol details (gRPC/HTTP) between components
- Persistent storage locations
- Security boundaries and encryption points
- Scaling boundaries and load balancing points

## Diagram (Mermaid)

```mermaid
graph TB
    %% Style Definitions
    classDef client fill:#FFE5CC,stroke:#FF9933
    classDef control fill:#CCE5FF,stroke:#3399FF
    classDef data fill:#E5FFCC,stroke:#66CC33
    classDef infra fill:#FFE6E6,stroke:#FF6666
    classDef security fill:#E6CCE6,stroke:#993399
    classDef queue fill:#FFE6CC,stroke:#FF9966

    subgraph ClientLayer["Client Layer"]
        SDK["Python SDK"]:::client
        CLI["CLI Interface"]:::client
        API["API Layer"]:::client
    end

    subgraph ControlPlane["Control Plane"]
        Gateway["Gateway Service"]:::control
        Scheduler["Scheduler Service"]:::control
        Auth["Authentication System"]:::security
        TaskQueue["Task Queue System"]:::queue
        Repo["Repository Layer"]:::control
    end

    subgraph DataPlane["Data Plane"]
        Worker["Worker Nodes"]:::data
        Container["Container Runtime"]:::data
        Storage["Storage System"]:::data
        Network["Network Layer"]:::data
    end

    subgraph Infrastructure["Infrastructure Layer"]
        Postgres[(PostgreSQL)]:::infra
        Redis[(Redis)]:::infra
        K8s["Kubernetes"]:::infra
    end

    %% Relationships
    SDK --> Gateway
    CLI --> Gateway
    API --> Gateway
    Gateway --> Auth
    Gateway --> Scheduler
    Scheduler --> TaskQueue
    TaskQueue --> Worker
    Worker --> Container
    Worker --> Storage
    Worker --> Network
    Scheduler --> Repo
    Repo --> Postgres
    TaskQueue --> Redis
    Worker --> K8s

    %% Click Events
    click SDK "https://github.com/beam-cloud/beta9/tree/main/sdk/src/beta9/"
    click CLI "https://github.com/beam-cloud/beta9/tree/main/sdk/src/beta9/cli/"
    click API "https://github.com/beam-cloud/beta9/tree/main/pkg/api/v1/"
    click Gateway "https://github.com/beam-cloud/beta9/tree/main/cmd/gateway/"
    click Scheduler "https://github.com/beam-cloud/beta9/tree/main/pkg/scheduler/"
    click Auth "https://github.com/beam-cloud/beta9/tree/main/pkg/auth/"
    click Worker "https://github.com/beam-cloud/beta9/tree/main/cmd/worker/"
    click Container "https://github.com/beam-cloud/beta9/tree/main/pkg/abstractions/container/"
    click Storage "https://github.com/beam-cloud/beta9/tree/main/pkg/storage/"
    click Postgres "https://github.com/beam-cloud/beta9/tree/main/manifests/kustomize/components/postgresql/"
    click Redis "https://github.com/beam-cloud/beta9/tree/main/manifests/kustomize/components/redis/"
    click K8s "https://github.com/beam-cloud/beta9/tree/main/manifests/kustomize/base/"
    click TaskQueue "https://github.com/beam-cloud/beta9/tree/main/pkg/abstractions/taskqueue/"
    click Repo "https://github.com/beam-cloud/beta9/tree/main/pkg/repository/"
    click Network "https://github.com/beam-cloud/beta9/tree/main/pkg/network/"

    %% Legend
    subgraph Legend
        Client["Client Components"]:::client
        ControlComp["Control Components"]:::control
        DataComp["Data Components"]:::data
        InfraComp["Infrastructure"]:::infra
        SecurityComp["Security Components"]:::security
        QueueComp["Queue Components"]:::queue
    end
```
