How labulakalia/crocodile works
Main Components
• Scheduler Center: This is the heart of the application (see core/cmd/server.go and related modules). It handles scheduling tasks, managing system state, and interacting with other components.
• Worker Nodes: Represent nodes (see core/cmd/client.go and Dockerfile.worker) that receive tasks via gRPC calls from the scheduling center and execute them. They also push real-time logs back.
• Frontend Web UI: Located under the “web” directory, it provides a rich client interface for task management (creating, editing, deleting, cloning tasks) and viewing logs.
• Database and Cache: The system uses MySQL (SQL schema in sql/) for persistent storage and Redis for handling real-time data such as task states, logs, and locks.
• Notification & Alarm: Based on the “common/notify” directory, this module integrates with various external alarm platforms (email, WeChat, DingDing, Slack, Telegram, etc.) for custom notifications.
• Security Components: The project supports certificate-based authentication and access tokens (see core/cert and common/jwt) to encrypt communications and control access.
• Supporting Utilities: Common modules for database operations (common/db), logging (common/log), error groups (common/errgroup), and utilities (common/utils) provide cross-cutting functionality.
Relationships and Interactions
• The Web UI communicates (typically using RESTful API calls, routed via core/router/api/v1) with the Scheduler Center to submit and control tasks.
• The Scheduler Center uses gRPC (core/middleware/grpcmidd.go and core/schedule/grpc.go) to interact with multiple Worker nodes.
• Worker nodes report back real-time task logs to the Scheduler Center.
• Both Scheduler and Worker nodes access shared resources like MySQL (persistent task and user data) and Redis (caching and real-time status).
• Security and Certificate modules ensure that communication between Scheduler and Worker nodes is encrypted and authenticated.
• Notification modules trigger alerts based on the task execution outcomes, utilizing integrations with multiple external services.
Architectural Patterns and Design Principles
• Separation of Concerns: The project clearly separates the concerns by modularizing scheduling (core), task execution (worker/client), frontend (web), and common/utility libraries.
• Distributed Design: The system supports multiple dispatch centers to avoid single points of failure. Worker nodes are designed to scale and be discovered automatically.
• Extensibility: The notification module includes different subdirectories for multiple platforms, indicating a plug-in or extensible notification strategy.
• Secure Communication: Use of JWT authentication, TLS certificates, and secure communication channels is emphasized.
Relevant Technologies/Frameworks and Their Roles
• Golang is used for the core logic and backend services.
• gRPC is the communication protocol between scheduling centers and worker nodes.
• MySQL and Redis are the underlying persistence and caching systems.
• A modern JavaScript framework (Vue.js is indicated by the .vue files) is used for building a rich web UI.
• Docker and Docker Compose facilitate container deployment, as indicated by the Dockerfiles and docker-compose.yml.
Additional Elements to Capture
• Modular structure from “core”, “common”, and “web” directories to indicate that you are capturing low-level details such as business logic, middleware, and request routing.
• Certificate and JWT authentication modules as part of the secure connection between distributed components.
• Mention that logs, task state, and scheduling strategies (random, rotation, worker weight, minimum worker number) are key parts of how the Scheduler Center interacts with Worker Nodes.
• The Web Frontend (user interaction layer)
• The Scheduler Center (core scheduling and control logic)
• Multiple Worker Nodes (distributed task execution)
• Data Stores (MySQL for persistent data and Redis for caching/locks)
• Security layers (JWT and certificate-based authentication)
• Notification services (integrated external services for alerts)
• Communication channels (gRPC and REST)
Happy diagramming!