How dasharo/open-source-firmware-validation works
And Label Main Components
• Test Orchestration Layer:
• The Robot Framework tests are the heart of the system. Mark areas such as “dasharo-compatibility”, “dasharo-security”, “dasharo-performance”, “dasharo-stability”, and “self-tests” as distinct test modules.
• “keys-and-keywords”, “lib” (which contains shared libraries and helper scripts) and “platform-configs” (which encapsulates per-platform configuration details).
• External Device Control and Power Management:
• External hardware components referenced in the README: DUT (Device Under Test), Sonoff WiFi smart plugs for power control, RTE relay systems, serial connectivity (via ser2net), and PiKVM for display/keyboard emulation.
• Software Integration Layer:
• That these APIs/libraries are integrated through the “requirements.txt” and are used by underlying scripts in “osfv_cli” and the “lib” directory.
• Supporting Utilities and Scripts:
• “scripts/” that contain CI workflows, regression and run wrappers, configuration parsers, and documentation generators.
• “os-config” and “pikvm-rest-api” as support components.
• From the test runner (Robot Commands, scripts/run.sh, and scripts/regression.sh) to each module (the various dasharo-* directories).
• From each test module to common libraries (e.g., “lib/”) that encapsulate hardware interactions (bios, power, network, flash, etc).
• Demonstrate configuration flow: Platform-specific configurations are read from “platform-configs” and are used to drive test conditions.
Architectural Patterns and Design Principles
• Modular Design:
• That the test system is organized by functional concerns (compatibility, security, performance, stability) – each functioning as separate modules that feed into the overall validation pipeline.
• Reusability and Extensibility:
• How common libraries in “lib/” (for instance, for network operations, flash operations, secure boot management) are reused across modules.
• Integration with External Services:
• Model the external API services (the previously standalone sonoff and RTE APIs now unified into common libraries via requirements) and how they integrate with the test environment.
• CI/CD and Automation:
• GitHub Workflows that are used to run “smoke” tests, QEMU tests, and documentation generation (e.g., scripts in “.github/workflows/”), indicating automation pipelines.
Technologies and Frameworks
• Robot Framework:
• Since many test files are in .robot format, include a “Robot Framework Test Runner” component.
• Scripting Languages:
• Python scripts and command-line wrappers (e.g., get_results_from_logs.py, scripts/lib/robot.sh, etc.) are part of the integration layer.
• External APIs:
• External libraries and previously independent repositories for controlling power devices (Sonoff) and test setups (RTE, PiKVM).
• Related test modules (all dasharo-* directories) together in a “Test Modules” group.
• Have a separate section for “Platform Configurations” from the “platform-configs” directory.
• Isolate the “lib” and “scripts” directories as shared infrastructure components.
• Annotations (labels) for connection logic:
• How generated documentation is produced from “scripts/create-docs.sh” and exported to “docs/”.