How bartongroup/slivka works
Project type and purpose
• Slivka is a Python-based server application and CLI tool for quickly scaffolding and running REST APIs that wrap command-line programs on compute clusters.
• It consists of three primary runtime roles: a CLI front end, a Flask HTTP server, and a background scheduler that submits jobs to external queuing systems (e.g., Univa Grid Engine, Slurm, LSF) or a local queue.
File structure
• Top-level files: setup.py, Makefile, conda recipe, environment.yml, pytest.ini, README, CHANGELOG, LICENSE.
• slivka/ package:
• cli.py & __main__.py – entrypoints for the command-line interface.
• conf/ – JSON schemas and loader for project and service settings.
• db/ – MongoDB document models, helper functions, repository classes.
• local_queue/ – single-machine queue server & client.
• migrations/ – migration engine and scripts to update on-disk or database layouts.
• project_template/ – directory of files & YAML templates used by the CLI to scaffold new Slivka services.
• scheduler/ – core scheduler, service monitor, factory plus pluggable runners (shell, slurm, lsf, grid_engine, slivka_queue).
• server/ – Flask app: api_views, media_views, form definitions and validation, WSGI entrypoint.
• utils/ – common utilities (daemonization, env loader, retry logic, expression parsing).
• sphinx-docs/: documentation source.
• test/: unit and integration tests covering each module.
README for additional insights
• Slivka is built on Flask (REST API) and MongoDB (persistent job metadata & logs).
• Primary use case: easy configuration and extensibility for running CLI programs on clusters.