How temporalio/sdk-python works
Main components
• Client Application – This represents the user’s code that uses the temporalio.Client API to start workflows and signal them.
• Workflow Sandbox – Workflows run inside a sandbox to ensure determinism. The sandbox mechanism (found in files like temporalio/worker/workflow_sandbox/*) is a distinct component in the architecture.
• Activity Execution – There are multiple types of activities (synchronous threaded, multiprocess, asynchronous) implemented in the SDK. These components include heartbeat and cancellation logic.
• Bridge (Rust Integration) – The “bridge” directory contains Rust code (Cargo.toml, src/*) that implements low-level communication with Temporal’s SDK Core. This bridge is essential to interoperate with the Temporal engine.
• API / gRPC Interfaces – The extensive “temporalio/api” directory shows auto-generated code from protobuf definitions. Additionally, proto files under scripts/_proto highlight that gRPC interfaces and message serialization are fundamental.
• Test & Development Environment – The tests directory and related scripts (e.g., run_bench.py, gen_docs.py) show integration tests, a time-skipping test environment, and development tooling.
• Build and Deployment Scripts – Several GitHub workflows (in .github/workflows) and build scripts (in scripts) indicate continuous integration and release processes.
Relationships and data flows
• The Worker pulls tasks from a task queue and executes workflows and activities using the Python SDK. Represent “Worker” as hosting two main sub-components: “Workflow Execution” (with sandbox isolation) and “Activity Execution”.
Architectural patterns and design principles
• Separation of Concerns – The design clearly separates client code, workflow sandboxing, and activity execution.
• Deterministic Workflow Execution via Sandbox – The sandboxed workflows enforce determinism.
• Multi-Execution Strategies – The SDK supports both asynchronous and synchronous (threaded or multiprocess) activity executions.
• Extensibility via Interceptors and Data Converters – The architecture documents support for custom data converters and optional OpenTelemetry instrumentation.
• gRPC and Protobuf Integration – The API layer heavily relies on gRPC and Protobuf (visible in scripts and temporalio/api) to support cross-language interoperability.
• From Client to Temporal Server via Client API.
• From Worker’s Workflow Sandbox to Activity Executor.
• From both Worker and Client to the Bridge that communicates with the external Temporal service.
• That testing components (like WorkflowEnvironment or tests in the tests directory) simulate these interactions.
• Details like “data conversion”, “heartbeating”, and “cancellation handling” (documented within the SDK) are annotated in component notes, especially around the activity execution block.
– Start by dividing the system into multiple layers: user-level code (clients, workflows, activities), runtime execution (workers, sandbox, activity executors), integration layers (bridge with underlying Rust SDK Core and gRPC interfaces), and external systems (Temporal server).
– Annotate each part with relevant technology names (Python, gRPC, Protobuf, Rust) and design principles (deterministic execution, asynchronous processing, sandbox isolation).
– Be detailed: include labels for components such as “temporalio/api” for message serialization, “workflow_sandbox” for deterministic workflow execution, and “bridge” for the integration layer.
This approach will provide a comprehensive visual representation tailored to the Temporal Python SDK’s complex architecture and development process.