How pauldouane/fluxum works
Main Components
• CLI Entry Point: Represented by src/main.rs, which is the starting point and uses Tokio.
• Configuration Manager: Handled by src/config.rs, responsible for reading and parsing the YAML configuration file.
• Task Manager/Job Executor: Found in src/job.rs. This component is central to orchestrating the execution of the tasks described in the YAML file.
• Logger: Indicated by src/logger.rs. This module handles logging of process outputs and events.
• Error Handling: Represented by src/error.rs which manages errors across the application.
• Utilities: Provided via src/utils.rs for common utility functions that may help with parsing or execution.
• Dockerfile & Workflow File: Although not part of the runtime code, these files indicate deployment and continuous integration/delivery (CI/CD) usage, so you might consider including a “Deployment/CI Pipeline” block that interacts with the CLI.
Relationships & Data Flow
• The CLI, upon invocation, starts in main.rs (running on the Tokio async runtime).
• It then calls the Configuration Manager to load and parse the YAML file (workflow.yaml, or standard config file), extracting the tasks and their order.
• The Task Manager uses the output of the Configuration Manager to determine which tasks to run. It will dispatch tasks based on sequential, parallel (fork and join), optional, or fail-able configurations.
• Logging is performed by the Logger, which receives outputs from tasks as well as lifecycle events.
• The Error Handler module ensures that any operational or runtime errors are caught and managed appropriately, affecting the lifecycle of the job pipeline.
Important Architectural Patterns
• Modularity: Different domain concerns (configuration, execution, logging, errors, utilities) are separated into distinct modules.
• Asynchronous Processing: The use of Tokio dictates that many operations, especially task execution, are designed to be non-blocking.
• Convention over Configuration: The design supports default behaviors based on certain file names and locations, minimizing the need for explicit user configuration.
• Extensibility: Although current features focus on shell execution, the design anticipates enhancements with additional runners for different types of tasks.
Relevant Technologies & Frameworks
• Rust and its ecosystem (Cargo.toml for dependencies).
• Tokio crate for asynchronous processing.
• YAML for configuration.
• Potential use of external process libraries & shell command execution tools.
Elements
• CLI Entry (main.rs)
• Configuration Manager (config.rs)
• Task Executor (job.rs)
• Logger (logger.rs)
• Error Handler (error.rs)
• Utilities (utils.rs)
• External Task Runners (Shell Runner; placeholders for future Python, Go, Docker runners)
• From CLI Entry to Configuration Manager for loading the YAML.
• From Configuration Manager to Task Executor for task scheduling.
• From Task Executor to External Task Runners and Logger.
• Error flows from Task Executor to Error Handler.
• Use another style (e.g., squares) to represent external systems (like shell processes).