How dagu-org/dagu works
Main Components
• Backend Engine: Represent the core “engine” (the binary started from cmd/main.go) which includes several subcomponents:
• Workflow Parser & Executor: The modules under internal/digraph and internal/config that read DAG YAML files, build dependency graphs, and manage execution flows.
• Task Executors: Components in internal/digraph/executor (for Docker, SSH, HTTP, jq, mail, etc.) that provide different execution mechanisms.
• Scheduler: The scheduling logic (see internal/scheduler and scheduler jobs) that determines when tasks should run based on cron schedules.
• Persistence & Logging: File-based storage components (internal/persistence/jsondb, internal/persistence/local, internal/logger) managing state, logs, and output.
• Agent & Command Interface: Modules for handling CLI commands (cmd and internal/cmd) and interactions with the agent (internal/agent) that monitors workflow state.
• Web UI: The frontend served from the “ui” directory, built with TypeScript/React. It handles rendering of workflows, live updates of logs/status (using routes defined in internal/frontend/server and assets under ui/src).
• API Layer: The REST API (generated in internal/frontend/gen/restapi) provides endpoints that the UI consumes to show real-time status, logs, and configuration details.
• Supporting Tools & Configuration: Tools such as the Dockerfile, docker-compose.yaml, and various configuration files support deployment and containerization, while docs and examples help users understand the system.
Relationships and Interactions
• The Web UI communicates with the backend’s API endpoints to fetch real-time DAG statuses, execution logs, and to submit new workflow requests.
• When a user initiates a DAG via the UI or CLI, the engine reads the YAML configuration, builds the DAG (using components in internal/digraph), and triggers the appropriate executors.
• The Scheduler component periodically checks for scheduled workflows and triggers execution.
• Outputs from one step are passed into subsequent tasks via a well-defined “data passing” mechanism (note the use of output variables).
• Notification and logging components (mailer, logger) record execution details and may interact externally (e.g., sending email notifications or HTTP requests).
Architectural Patterns & Design Principles
• Monolithic Single Binary: Despite having distinct logical modules, everything is compiled into one executable without an external dependency, contributing to a “local-first” design.
• Declarative Configuration: The system is built around YAML-defined workflows, making the execution flow clear and maintainable.
• Modular Design: Though in one binary, the code is well separated into packages (scheduler, executor, persistence, logger), each responsible for a specific function.
Relevant Technologies and Frameworks
• Backend is implemented in Go. Note the use of Go’s package structure (cmd, internal packages) and tools like Goreleaser for builds.
• The UI is built with React (TypeScript/TSX files) and bundled with Webpack.
• Configuration parsers and workflow engines (YAML processing, JSON databases) are integral parts of how workflows are executed.
• Tools such as Docker and docker-compose are also featured for containerized deployment.