How spcl/serverless-benchmarks works
Main Components
• Benchmark Applications & Experiments – As evidenced by the “benchmarks” directory that holds various microbenchmarks, webapps benchmarks, multimedia tests, scientific experiments, and utilities. These benchmarks are the payloads (or tests) that are deployed and measured.
• Platform-Specific Modules – Located in the “sebs” directory (with subdirectories for aws, azure, gcp, openwhisk, local). These modules encapsulate the provider-specific logic for deploying functions, container management, resource configuration, triggers, and storage interfaces.
• Deployment & Build Tools – The presence of “install.py”, “scripts/”, and “tools/” indicates automated build/deployment workflows. This layer handles the orchestration of the experiments and benchmark deployment.
• Configuration & Documentation – “config” folder (with example JSON files) provides system configurations, while the “docs” folder contains design and usage guides.
• Docker & Containerization – Multiple “dockerfiles” directories for each cloud provider and language (nodejs, python) show that containerized function packaging is a core element.
• Testing & CI – The “tests” directory and integration with CI pipelines (CircleCI, GitHub actions) confirm continuous integration and regression testing as integral parts.
Relationship and Block Interactions
• Orchestration Layer: At the top, include a central orchestration module (triggered via the install script and deployment APIs) that uses configuration files (from “config”) to decide which platform to deploy.
• Function Deployment Pipeline: Within each platform module, show the flow through container building (using Dockerfiles), packaging (scripts and tools), deployment, and later invocation. This may further reference the “dockerfiles” and “tools/build_docker_images.py”.
• Benchmark Engine: Include a section representing the benchmark engine which runs experiments. This engine (supported by processes in “sebs/experiments” and “scripts/”) provides workload stimuli to the deployed FaaS functions.
Important Architectural Patterns and Design Principles
• Modularity – The clear separation between provider-specific modules and the central orchestration layer demonstrates a plug-in architecture that favors extensibility.
• Containerization – Use of Docker for building function containers is a key element that decouples the benchmark runtime environment from each cloud’s underlying infrastructure.
• Automation – Automated deployment and measurements are core to the design. Emphasize the automation flow starting from input (configuration) to function deployment, automatic triggering, and collection of benchmarks.
• CI/CD Integration – CD pipelines are supported as shown by CircleCI and GitHub config files. They continuously build, test, and deploy the system’s components.
Relevant Technologies, Frameworks, and Libraries
• Docker for containerization and deployment.
• Python (with virtual environment and pip) as the primary scripting and orchestration language.
• Node.js components (noticeable in benchmarks and providers) for non-Python function implementations.
• Cloud provider SDKs/APIs (for AWS, Azure, GCP, OpenWhisk) wrapped in the sebs provider modules.
• CI/CD tooling such as CircleCI and GitHub Actions for automated testing and integration.
Elements
• Supporting modules like testing, configuration, and logging.
• Sub-components (such as container build pipelines, CI/CD flows, and results aggregation).