# burla-cloud/burla architecture

> Architecture diagram of the GitHub repository burla-cloud/burla, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/burla-cloud/burla
- Repository: https://github.com/burla-cloud/burla

## Overview

##### Main Components

a) Client Layer:
- Client library (burla package) that users interact with
- Main `remote_parallel_map` function
- Authentication component (_auth.py)

b) Main Service (Cluster Controller):
- Central orchestration service
- Job management endpoints
- Cluster state management
- Dashboard interface (static files)

c) Node Service:
- Node management and health monitoring
- Worker allocation

d) Worker Service:
- UDF (User-Defined Function) execution
- Function runtime environment

##### Component Relationships

- Client → Main Service: Job submission, authentication, results retrieval
- Main Service → Node Service: Node health checks, worker allocation
- Node Service → Worker Service: Function execution, result collection
- Worker Service → Main Service: Job status updates, results reporting

##### Architectural Patterns to Highlight

- Microservices Architecture: Show clear separation between services
- Master-Worker Pattern: Illustrate the hierarchy from cluster controller to workers
- RESTful Communication: Indicate API endpoints between services

**[Client Library Layer]**
- ↓
**[Main Service (Cluster Controller)]**
- ↓
**[Node Service Layer]**
- ↓
**[Worker Service Layer]**

##### Additional Elements

- A monitoring/dashboard component
- Multi-tenant nature of the cluster
- Single-tenant worker nodes
- Scalability aspect with multiple worker instances

##### Technical Details

- Python as the primary language
- Docker containers for service isolation
- REST APIs for inter-service communication
- Authentication flow
- Job lifecycle

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and classes
    classDef client fill:#b3d1ff,stroke:#333,stroke-width:2px
    classDef main fill:#90EE90,stroke:#333,stroke-width:2px
    classDef node fill:#FFE5B4,stroke:#333,stroke-width:2px
    classDef worker fill:#FFA07A,stroke:#333,stroke-width:2px
    classDef container fill:#E6E6FA,stroke:#333,stroke-width:2px

    %% Client Layer
    subgraph ClientLayer["Client Layer"]
        CL["Client Library"]
        RPM["Remote Parallel Map"]
        Auth["Authentication Component"]
    end
    
    %% Main Service Layer
    subgraph MainService["Main Service (Cluster Controller)"]
        CO["Central Orchestration"]
        JM["Job Management"]
        CSM["Cluster State Management"]
        DI["Dashboard Interface"]
    end
    
    %% Node Service Layer
    subgraph NodeService["Node Service"]
        NM["Node Management"]
        WA["Worker Allocation"]
    end
    
    %% Worker Service Layer
    subgraph WorkerService["Worker Service"]
        UDF["UDF Execution"]
        FRE["Function Runtime"]
        WE["Worker Endpoints"]
    end
    
    %% Container Infrastructure
    subgraph Containers["Container Infrastructure"]
        MSC["Main Service Container"]
        NSC["Node Service Container"]
        WSC["Worker Service Container"]
    end

    %% Relationships
    CL --> CO
    RPM --> JM
    Auth --> CO
    CO --> NM
    JM --> WA
    CSM --> NM
    NM --> UDF
    WA --> WE
    UDF --> JM
    WE --> CSM

    %% Component Classifications
    class CL,RPM,Auth client
    class CO,JM,CSM,DI main
    class NM,WA node
    class UDF,FRE,WE worker
    class MSC,NSC,WSC container

    %% Click Events
    click CL "https://github.com/burla-cloud/burla/blob/main/client/src/burla/__init__.py"
    click RPM "https://github.com/burla-cloud/burla/blob/main/client/src/burla/_remote_parallel_map.py"
    click Auth "https://github.com/burla-cloud/burla/blob/main/client/src/burla/_auth.py"
    click CO "https://github.com/burla-cloud/burla/blob/main/main_service/src/main_service/cluster.py"
    click JM "https://github.com/burla-cloud/burla/blob/main/main_service/src/main_service/endpoints/jobs.py"
    click CSM "https://github.com/burla-cloud/burla/blob/main/main_service/src/main_service/endpoints/cluster.py"
    click DI "https://github.com/burla-cloud/burla/blob/main/main_service/src/main_service/static/dashboard.html"
    click NM "https://github.com/burla-cloud/burla/blob/main/node_service/src/node_service/endpoints.py"
    click WA "https://github.com/burla-cloud/burla/blob/main/node_service/src/node_service/worker.py"
    click UDF "https://github.com/burla-cloud/burla/blob/main/worker_service/src/worker_service/udf_executor.py"
    click FRE "https://github.com/burla-cloud/burla/blob/main/worker_service/Dockerfile.base"
    click WE "https://github.com/burla-cloud/burla/blob/main/worker_service/src/worker_service/endpoints.py"
    click MSC "https://github.com/burla-cloud/burla/tree/main/main_service/Dockerfile"
    click NSC "https://github.com/burla-cloud/burla/tree/main/node_service/Dockerfile"
    click WSC "https://github.com/burla-cloud/burla/tree/main/worker_service/Dockerfile"

    %% Legend
    subgraph Legend
        L1["Client Components"]:::client
        L2["Main Service Components"]:::main
        L3["Node Service Components"]:::node
        L4["Worker Service Components"]:::worker
        L5["Container Components"]:::container
    end
```
