How runabol/tork works
Based on our analysis of the file tree and the README, “Tork” is a fully featured, highly scalable workflow engine. It’s designed to manage and execute workflows (jobs and tasks) in both stand-alone and distributed setups. The project is written in Go and is composed of several well-structured packages that together implement a rich set of features including a REST API, CLI, containerized task isolation (via Docker, Podman, and Shell runtimes), integrated brokers (supporting in-memory and RabbitMQ), middleware extensions, and datastore support (using PostgreSQL).
• Frontend/Client Layer: Although the Web UI is hosted elsewhere, note that Tork exposes a REST API that the Web UI (or any client) can use.
• CLI Interface: Represent the “cli” package as a user-facing component that initiates various commands (run, health, migrate, etc.).
• API Gateway/REST Layer: Mark the REST API endpoint which sits along with the CLI commands to expose functionality.
• Core Workflow Engine:
• “engine” package – the central orchestration layer handling job coordination, scheduling, and execution.
• “internal/coordinator” – where the workflow coordination and API handlers live.
• “internal/worker” – which manages task execution.
• “internal/eval”, “internal/host”, and similar packages that support evaluation of expressions, host management etc.
• Broker System:
• “broker” package – supports messaging through both in-memory and RabbitMQ implementations.
• Datastore:
• “datastore” and “db/postgres” directories – for persistence, particularly storing job/task state within a PostgreSQL database.
• Middleware:
• “middleware” with distinct subdirectories (job, task, node, web) – illustrates the extensible aspects used for adding functionalities like logging, redaction, and webhook handling.
• Runtime Components:
• “runtime” package – handles execution environments. This includes Docker, Podman, and Shell-based runtimes. These modules are responsible for providing isolation and executing tasks in containers.
• Utilities and Internal Libraries:
• “internal” directories for caching, synchronization, UUID generation, and error management that support the core engine.
• Configuration and Infrastructure:
• “conf” and “configs” – provide configuration management and sample setup files.
• Docker Compose and Makefiles – indicate CI/CD integrations and deployment as well as orchestration details.
Relationships and Data Flow
• Internal data flows within the workflow engine such as:
• API → Coordinator → Scheduler and Worker.
• Workers, via middleware and runtime layers, execute tasks and then report back status to the engine.
• The Broker communicates between various modules to handle messaging and buffering of events.
• Persistence flows: tasks and job statuses are stored into and read from the Datastore (Postgres).
• External dependencies (such as RabbitMQ) are specifically noted connecting to the Broker.
Architectural Patterns
• Modular and layered structure of the application which encourages clean separation of concerns.
• Extensibility provided by the middleware architecture.
• Support for different runtime environments which provides flexibility and isolation for task execution.
• Coordination patterns between schedulers and workers for achieving horizontal scalability and fault tolerance.
Technologies and Frameworks
• Go as the programming language.
• PostgreSQL for data persistence.
• RabbitMQ (along with an in-memory alternative) as the messaging broker.
• Docker, Podman, and shell execution environments.
• GitHub Actions workflows (as seen in the .github directory) to illustrate CI/CD integration.