# pkuhpc/openscow architecture

> Architecture diagram of the GitHub repository pkuhpc/openscow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pkuhpc/openscow
- Repository: https://github.com/pkuhpc/openscow

## Overview

##### Main components
- Frontend/Web Clients:
– apps/portal-web and apps/mis-web (user interfaces and dashboard components)
– Other UI modules in apps/notification and components within libraries (e.g., libs/web)
- Backend Services:
– apps/portal-server for portal business logic and interaction with clusters
– apps/mis-server for managing MIS operations (billing, user models, etc.)
– apps/auth for authentication and user management services
– apps/audit-server for logging and auditing operations
– apps/gateway for API/proxy functions
– Additional utilities such as apps/cli (command-line deployment and management)
- Domain-Specific Services:
– apps/ai for AI compute management with specific configurations and deployment scripts.
– Integration with scheduling systems (e.g. via libs/scheduler-adapter, libs/scowd, and others).
- Supporting Libraries:
– A series of libraries under libs (auth, config, decimal, hook, libconfig, operation-log, protos, server, ssh, web, utils, etc.) that encapsulate shared logic, API definitions, data models, and error handling.
- Infrastructure and Deployment:
– Docker and dev container configurations, Vagrant deployment scripts (deploy/vagrant, dev/vagrant) and configuration files for clusters, authentication, audit, etc.
– Config management for external integrations (clusters’ slurm config, AI configurations, etc.)

##### Relationships and interactions
- A central audit mechanism is provided by apps/audit-server, receiving logs and operation events from other services.
- The gateway and file proxy services (found in apps/gateway and portal-server’s clusterops) mediate communications between external client requests and the underlying compute clusters (managed by slurm or other schedulers).
- Data persistence is managed behind the scenes (via migrations in apps/ai, apps/audit-server, and mis-server) so include a central database component that many services interact with.
- The deployment and configuration files (seen in deploy/*, docs/*, and dev/* directories) indicate a well-organized orchestration, so show that the entire system sits on top of a deployment infrastructure that uses Docker, Vagrant, and Kubernetes (if applicable).

##### Architectural patterns and design principles
- Modular/Microservices: Each “app” under the apps directory acts as an independent microservice or web application.
- Separation of Concerns: The presentation layer (web UIs), business logic (portal-server, mis-server, auth, audit-server), and configuration/integration internals (various libs, protos, config files) are clearly separated.
- API-First: Multiple APIs (rest/trpc/gRPC) are used to communicate between services and the front ends.
- Layered Architecture: Presentation (web UIs) → API Gateway → Services (business logic and cluster adapters) → Database/External Systems.
- Extensibility: The use of libraries (under libs) and clearly separated configuration (apps/*/config) allows plug-and-play updates or customization, supporting both open-source and enterprise editions.

##### Relevant technologies and frameworks
- Frontend: Next.js framework (as seen in next.config files in the portal-web and mis-web apps), React components, and integration for multilingual support (i18n).
- Backend: Node.js with TypeScript, TRPC endpoints, MikroORM for database interactions, and possibly a RESTful architecture.
- DevOps/Deployment: Docker, Vagrant, and CI workflows (.github/workflows) for build/testing/deployment.
- Communication: gRPC or TRPC endpoints for inter-service communications, plus custom protocol buffers defined in the protos directories.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        portalUI["Portal UI (portal-web)"]:::frontend
        misUI["MIS UI (mis-web)"]:::frontend
        notificationUI["Notification UI (notification)"]:::frontend
        sharedUI["Shared Web Components (libs/web)"]:::frontend
    end

    %% Backend Services
    subgraph "Backend Services"
        portalService["Portal Service (portal-server)"]:::backend
        misService["MIS Service (mis-server)"]:::backend
        authService["Authentication Service (auth)"]:::backend
        auditService["Audit Service (audit-server)"]:::backend
        apiGateway["API Gateway (gateway)"]:::backend
        cliManagement["CLI Management (cli)"]:::backend
    end

    %% Domain-Specific Services
    subgraph "Domain Services"
        aiService["AI Compute Management (ai)"]:::domain
        schedulerIntegration["Scheduler/Cluster Integration (scheduler-adapter & scowd)"]:::domain
    end

    %% Supporting Libraries
    subgraph "Supporting Libraries"
        authLib["Authentication Library (auth)"]:::library
        configLib["Configuration Management (config & libconfig)"]:::library
        decimalLib["Decimal Computation (decimal)"]:::library
        hookLib["Hooks/Extensions (hook)"]:::library
        operationLog["Operation Log (operation-log)"]:::library
        protosLib["Protocol Definitions (protos)"]:::library
        serverHelper["Server Helpers (server)"]:::library
        sshUtils["SSH Utilities (ssh)"]:::library
        generalUtils["General Utilities (utils)"]:::library
    end

    %% Infrastructure/Deployment
    subgraph "Infrastructure/Deployment"
        devContainers["Dev Containers (.devcontainer & docker)"]:::infrastructure
        vagrantScripts["Vagrant Scripts (deploy/vagrant & dev/vagrant)"]:::infrastructure
    end

    %% External Systems
    centralDatabase["Central Database"]:::external
    hpcCluster["HPC Cluster (Slurm)"]:::external

    %% Frontend to Backend Routes
    portalUI -->|"calls"| apiGateway
    misUI -->|"calls"| apiGateway
    notificationUI -->|"calls"| apiGateway
    sharedUI -->|"supports"| portalUI
    sharedUI -->|"supports"| misUI

    %% API Gateway Routing
    apiGateway -->|"routes"| portalService
    apiGateway -->|"routes"| misService
    apiGateway -->|"routes"| authService
    apiGateway -->|"proxies"| hpcCluster

    %% Backend Service Interactions
    portalService -->|"validates"| authService
    misService -->|"validates"| authService
    portalService -->|"logs"| auditService
    misService -->|"logs"| auditService

    %% Database Interactions
    portalService -->|"updates"| centralDatabase
    misService -->|"updates"| centralDatabase
    authService -->|"updates"| centralDatabase
    auditService -->|"updates"| centralDatabase
    aiService -->|"updates"| centralDatabase

    %% HPC and Cluster Management
    portalService -->|"manages"| hpcCluster
    misService -->|"manages"| hpcCluster

    %% Domain-Specific Interaction
    aiService -->|"integrates"| schedulerIntegration

    %% Backend Services Utilizing Libraries (representative connections)
    portalService -->|"uses"| configLib
    authService -->|"uses"| authLib

    %% Infrastructure Orchestration (deployment tools)
    devContainers ---|"deploys"| apiGateway
    vagrantScripts ---|"deploys"| centralDatabase
    vagrantScripts ---|"deploys"| hpcCluster

    %% Click Events for Frontend/Web Clients
    click portalUI "https://github.com/pkuhpc/openscow/tree/master/apps/portal-web"
    click misUI "https://github.com/pkuhpc/openscow/tree/master/apps/mis-web"
    click notificationUI "https://github.com/pkuhpc/openscow/tree/master/apps/notification"
    click sharedUI "https://github.com/pkuhpc/openscow/tree/master/libs/web"

    %% Click Events for Backend Services
    click portalService "https://github.com/pkuhpc/openscow/tree/master/apps/portal-server"
    click misService "https://github.com/pkuhpc/openscow/tree/master/apps/mis-server"
    click authService "https://github.com/pkuhpc/openscow/tree/master/apps/auth"
    click auditService "https://github.com/pkuhpc/openscow/tree/master/apps/audit-server"
    click apiGateway "https://github.com/pkuhpc/openscow/tree/master/apps/gateway"
    click cliManagement "https://github.com/pkuhpc/openscow/tree/master/apps/cli"

    %% Click Events for Domain-Specific Services
    click aiService "https://github.com/pkuhpc/openscow/tree/master/apps/ai"
    click schedulerIntegration "https://github.com/pkuhpc/openscow/tree/master/libs/scheduler-adapter"
    click schedulerIntegration "https://github.com/pkuhpc/openscow/tree/master/libs/scowd"

    %% Click Events for Supporting Libraries
    click authLib "https://github.com/pkuhpc/openscow/tree/master/libs/auth"
    click configLib "https://github.com/pkuhpc/openscow/tree/master/libs/config"
    click configLib "https://github.com/pkuhpc/openscow/tree/master/libs/libconfig"
    click decimalLib "https://github.com/pkuhpc/openscow/tree/master/libs/decimal"
    click hookLib "https://github.com/pkuhpc/openscow/tree/master/libs/hook"
    click operationLog "https://github.com/pkuhpc/openscow/tree/master/libs/operation-log"
    click protosLib "https://github.com/pkuhpc/openscow/tree/master/libs/protos"
    click serverHelper "https://github.com/pkuhpc/openscow/tree/master/libs/server"
    click sshUtils "https://github.com/pkuhpc/openscow/tree/master/libs/ssh"
    click generalUtils "https://github.com/pkuhpc/openscow/tree/master/libs/utils"

    %% Click Events for Infrastructure/Deployment
    click devContainers "https://github.com/pkuhpc/openscow/blob/master/.devcontainer"
    click devContainers "https://github.com/pkuhpc/openscow/tree/master/docker/"
    click vagrantScripts "https://github.com/pkuhpc/openscow/tree/master/deploy/vagrant"
    click vagrantScripts "https://github.com/pkuhpc/openscow/tree/master/dev/vagrant"

    %% Class Styles
    classDef frontend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef backend fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef domain fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef library fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef infrastructure fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
```
