How agronholm/apscheduler works
Main Components
a. Scheduler Core:
• This is the central engine that manages job scheduling, triggering, and execution.
b. Triggers:
• Various scheduling mechanisms – cron, interval, calendar interval, date and combinatorial triggers.
• As dedicated components or submodules that feed configuration into the scheduler.
c. Data Stores (Persistent Storage):
• In-memory (memory.py)
• SQL databases (sqlalchemy.py)
• MongoDB (mongodb.py)
• These modules are responsible for persisting the job schedules, allowing multiple scheduler instances to share the same job information and survive restarts.
d. Event Brokers (for Distributed Coordination):
• Event broker components that facilitate communication in multi-node setups:
• PostgreSQL, Redis, MQTT integration modules.
• API integration that allows the scheduler to send/receive events (job added, job executed, etc.) to these brokers.
e. Executors:
• These components manage how jobs are executed. Include:
• Thread-based (thread.py)
• Process/subprocess-based (subprocess.py)
• Asynchronous executors (async_.py, qt.py)
• The executors provide the bridge between scheduling and the actual execution of tasks.
Integration (API) Details for the 3 Key Areas
a. Datastores:
• How the scheduler core communicates with the datastore APIs to load, update, and persist job information.
• Abstraction layers (base API) that allow different datastore implementations to plug in seamlessly.
b. Event Brokers:
• That the scheduler connects with these brokers via a standardized event broker interface, which abstracts the differences between implementations (asyncpg, redis, mqtt).
c. Executors:
• That the executor layer adapts the scheduler’s job data into task execution, taking into account both synchronous and asynchronous paradigms.
Architectural Patterns and Design Principles
• Separation of Concerns: Clearly separate the core scheduling logic from the persistence (datastores), messaging (event brokers), and execution (executors) aspects.
• Pluggable Architecture: Use abstract base classes (e.g., in datastores and event brokers directories) to allow for multiple implementations.
• Modularity: Each component (or submodule) – schedulers, triggers, datastores, event brokers, executors – should be drawn as separate blocks with clear APIs that indicate how they integrate with the scheduler core.