How apache/oozie works
Main Components
• Client Module: This includes the “client” directory where command‐line utilities (CLIParser, OozieCLI, etc.) reside. It is responsible for job submission, monitoring and interacting with the server via web service APIs.
• Core Module: This is at the heart (“core” directory) of the system. It contains the workflow engine’s logic (execution of workflow definitions, job scheduling, service management such as event queues, lock management, and integration with Zookeeper in some parts). It includes the implementation of control nodes (start, decision, fork, join, end) and action nodes.
• Server Module: Under “server”, you have the Oozie server implementation (web application code including servlets, REST endpoints, Guice modules, etc.). This acts as the façade that receives client requests and dispatches them to the underlying services.
• Sharelib: The “sharelib” modules (with subdirectories for distcp, git, hive, hive2, oozie, pig, spark, sqoop, streaming, etc.) provide libraries and action executors that are loaded dynamically by the server to run various Hadoop-related job actions.
• Distro and Tools: The “distro” and “tools” directories include scripts for packaging, deployment, environment setup, and diagnostic tools that support the overall operation and maintenance of Oozie.
• Documentation and Examples: “docs” and “examples” provide user-level documentation and sample workflows that help users understand and extend Oozie.
Relationships and Interaction
• Server (and its Core components) takes XML workflow definitions, processes them through the workflow engine (managing control flow nodes & action nodes), and then triggers specific actions. Represent this by subdividing the Server/Core into components such as “Workflow Engine”, “Coordinator Engine”, “Bundle Engine”, “Scheduler”, “Event Queue”, and “Service Layer”.
Architectural Patterns and Design Principles
• The overall design follows a modular and layered architecture. The separation between front-end (client and web application interface in the Server), business logic (Core Workflow Engine, scheduling, event processing), and helper/libraries (Sharelib) is key.