How portworx/torpedo works
Project Type & Purpose
• Torpedo is a test suite designed to qualify external storage providers (including CSI-compliant and legacy drivers) for stateful container environments. It simulates various failure scenarios (e.g., node, network, scheduler, and volume failures) in distributed container orchestration systems.
• It interacts with container orchestrators such as Kubernetes, Mesos, Docker, and even legacy DVDI systems to verify that storage integrations are robust and resilient under production conditions.
High-Level Architectural Components
• Test Orchestrator Layer: This is the core of Torpedo, which drives all testing scenarios. It includes the main test suite (found in the “tests” directory), scripts, and helper modules.
• API Server: Under “apiServer”, this component (and its submodules like “taas”) acts as a control plane, likely managing and coordinating test tasks or interacting with external storage providers.
• Driver Modules: The “drivers” directory contains a wide array of storage driver implementations and utility modules. These drivers are grouped by target (e.g., “api”, “applications”, “backup”, “monitor”, “node”, “scheduler”, “volume”, etc.) and cover different storage technologies (like Portworx, pure storage arrays, cloud providers like AWS, Azure, GCP, etc.).
• Utility & Helper Packages: The “pkg” directory includes utility components for logging, network operations, error handling, snapshot management, etc.
• Deployment & CI/CD: Files like Dockerfiles, Jenkinsfile, Travis CI, and deployment scripts (under “deployments” and “scripts”) indicate how the application can be built, deployed, and integrated into continuous integration pipelines.
• Testing & Simulation: The “tests” directory provides a comprehensive suite for running various scenarios (backup tests, basic functionality, longevity, PDS etc.) to simulate real failure cases in a cluster environment.
Relationships & Interactions
• The Test Orchestrator interacts with the API Server to trigger storage-related test cases.
• The API Server, in turn, may call into the various drivers to simulate storage operations (like create, mount, unmount, and delete volumes) under different failure modes.
• The Driver Modules are the adapters between the test logic and the actual storage provider (whether via CSI or legacy DVDI). They encapsulate interactions with underlying cloud storage services, container orchestrators, or local storage solutions.
• Utility packages support both the test orchestrator and drivers by providing standardized logging, error handling, configuration parsing, and network operations.
• Deployment scripts and CI configuration files bridge development, integration, and production environments by automating build, test, and deployment procedures.
Architectural Patterns & Design Principles
• Modularity and Separation of Concerns: The clear distinction between testing (tests/), driver implementation (drivers/), and utilities (pkg/) suggests a modular design where components can be independently developed or extended.
• Adapter/Strategy Pattern: Within the “drivers” directory, multiple implementations provide a strategy for different storage providers. This pattern allows the test suite to be adaptable across a range of storage solutions.
• Continuous Integration/Deployment: The presence of files for Jenkins, Travis, Docker, and Makefiles shows an emphasis on automation and repeatability in testing and deployment.
• External Integration: Support for CSI and legacy volume interfaces indicates the system is designed to interact with multiple external services and technologies.