# bartongroup/slivka architecture

> Architecture diagram of the GitHub repository bartongroup/slivka, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bartongroup/slivka
- Repository: https://github.com/bartongroup/slivka

## Overview

##### Project type and purpose
- Slivka is a Python-based server application and CLI tool for quickly scaffolding and running REST APIs that wrap command-line programs on compute clusters.
- It consists of three primary runtime roles: a CLI front end, a Flask HTTP server, and a background scheduler that submits jobs to external queuing systems (e.g., Univa Grid Engine, Slurm, LSF) or a local queue.

##### File structure
- Top-level files: setup.py, Makefile, conda recipe, environment.yml, pytest.ini, README, CHANGELOG, LICENSE.
- slivka/ package:
- cli.py & __main__.py – entrypoints for the command-line interface.
- conf/ – JSON schemas and loader for project and service settings.
- db/ – MongoDB document models, helper functions, repository classes.
- local_queue/ – single-machine queue server & client.
- migrations/ – migration engine and scripts to update on-disk or database layouts.
- project_template/ – directory of files & YAML templates used by the CLI to scaffold new Slivka services.
- scheduler/ – core scheduler, service monitor, factory plus pluggable runners (shell, slurm, lsf, grid_engine, slivka_queue).
- server/ – Flask app: api_views, media_views, form definitions and validation, WSGI entrypoint.
- utils/ – common utilities (daemonization, env loader, retry logic, expression parsing).
- sphinx-docs/: documentation source.
- test/: unit and integration tests covering each module.

##### README for additional insights
- Slivka is built on Flask (REST API) and MongoDB (persistent job metadata & logs).
- Primary use case: easy configuration and extensibility for running CLI programs on clusters.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Slivka Core Components
    subgraph "Slivka Core"
        direction TB
        CLI["CLI Front End"]:::core
        Config["Configuration Loader"]:::core
        Server["Flask HTTP Server"]:::core
        Scheduler["Scheduler Core"]:::core
        Migrations["Migration Tool"]:::core
    end

    %% Database
    DB["MongoDB"]:::db

    %% Runner Plugins
    subgraph "Runner Plugins"
        direction TB
        Runners["Runner Plugins"]:::plugin
    end

    %% External Execution Targets
    subgraph "Job Execution Targets"
        direction TB
        Cluster["External Clusters"]:::external
        LocalQueue["Local Queue Service"]:::external
    end

    %% Core Flows
    CLI -->|"loads schemas"| Config
    Config -->|"start-server / run-scheduler / migrate"| CLI
    CLI -->|"start-server"| Server
    CLI -->|"run-scheduler"| Scheduler
    CLI -->|"migrate"| Migrations

    Server -->|"read/write jobs"| DB
    DB -->|"job metadata"| Server

    Scheduler -->|"pull pending jobs"| DB
    DB -->|"pending jobs"| Scheduler

    Scheduler -->|"delegate job"| Runners
    Runners -->|"submit jobs"| Cluster
    Runners -->|"submit jobs"| LocalQueue

    Cluster -->|"status updates"| Runners
    LocalQueue -->|"status updates"| Runners
    Runners -->|"update status"| DB

    %% Migration interactions (dashed)
    Migrations -.->|"read/write"| DB
    Migrations -.->|"update templates"| Config

    %% Click Events
    click CLI "https://github.com/bartongroup/slivka/blob/master/slivka/cli.py"
    click CLI "https://github.com/bartongroup/slivka/blob/master/slivka/__main__.py"
    click Config "https://github.com/bartongroup/slivka/tree/master/slivka/conf/"
    click Server "https://github.com/bartongroup/slivka/tree/master/slivka/server/"
    click DB "https://github.com/bartongroup/slivka/tree/master/slivka/db/"
    click Scheduler "https://github.com/bartongroup/slivka/blob/master/slivka/scheduler/scheduler.py"
    click Runners "https://github.com/bartongroup/slivka/tree/master/slivka/scheduler/runners/"
    click LocalQueue "https://github.com/bartongroup/slivka/tree/master/slivka/local_queue/"
    click Migrations "https://github.com/bartongroup/slivka/tree/master/slivka/migrations/"

    %% Styling
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef db fill:#90EE90,stroke:#333,stroke-width:1px
    classDef plugin fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
    class CLI,Config,Server,Scheduler,Migrations core
    class DB db
    class Runners plugin
    class Cluster,LocalQueue external
```
