How joseph-wardle/sandwich-pipeline works
1. Overall shape
- This is a Python-based production pipeline repo for multiple DCCs, not a single app.
- The architecture is split into three major layers: a DCC-agnostic core, per-DCC integrations, and bundled runtime/site assets.
- The repo is designed to run on multiple OSes and is configured externally through src/env.py rather than hardcoding studio-specific paths and credentials.
2. Process entrypoint and dispatch
- src/__main__.py is the top-level entrypoint.
- It dispatches into a DCC-specific launcher under src/dcc/<dcc>/launch.py via the shared framework layer.
- This means the same repo can start Blender, Maya, Houdini, Nuke, Substance Painter, or Substance Designer workflows through a common startup path.
3. Framework boundary
- src/framework defines the launcher/runtime abstraction layer.
- It provides the common interface for “launch a DCC” and “run inside a DCC runtime,” plus dispatch logic.
- This is the seam that keeps platform startup code separate from pipeline logic and DCC-specific implementation details.
4. Cross-DCC core services
- src/core holds the reusable pipeline services shared across all DCCs.
- Key domains include asset paths/version adapters, shot/version adapters, versioning storage and service logic, ShotGrid integration, playblast generation, texture conversion, telemetry, and UI dialogs.
- These modules are the main business logic boundary: they should be reusable without knowing which DCC is currently running.
5. Versioning and file identity
- src/core/versioning appears to be the central model/service/store for versioned pipeline artifacts.
- Asset, shot, and environment-specific version_adapter.py files suggest that different content types are normalized into a common versioning scheme.
- This likely underpins save/version browser dialogs, publish flows, and asset/shot file naming.
6. Asset, shot, and environment modeling
- src/core/asset, src/core/shot, and src/core/environment provide content-domain abstractions.
- src/core/util/paths.py, src/core/asset/paths.py, and src/core/shotgrid/paths.py indicate a strong path-convention layer that maps studio storage layout into objects and services.
- src/core/struct contains serializable data models like timeline and material structures, likely used across publishing, UI, and exports.
7. Publishing and playblasts
- src/core/playblast centralizes playblast naming, encoding, presets, temp directory handling, and review upload flow.
- The review subpackage suggests a review-oriented workflow: create playblasts, associate versions/playlists, and upload or sync them to an external review system.
- DCC-specific playblast modules in Maya and Houdini wrap this core logic with host-specific HUDs, dialogs, and launchers.
8. ShotGrid integration
- src/core/shotgrid is a dedicated client boundary for the studio’s ShotGrid backend.
- It includes entities, errors, caching/memoization, and path helpers, implying the pipeline reads and writes production metadata from ShotGrid rather than embedding that logic in each DCC.
- This is one of the main external-service integrations in the repo.
9. Telemetry subsystem
- src/core/telemetry is a self-contained event collection and upload pipeline.
- It includes event definitions, scope/config, local spooling, a local stack/ingester, and a standalone __main__.py, suggesting telemetry can run independently of the DCC tools.
- The telemetry-backend directory provides the server-side destination: PostgreSQL schema plus Grafana dashboards/provisioning for monitoring tool health and usage.
10. Per-DCC integration model
- Each DCC directory under src/dcc follows the same pattern: launch.py, runtime.py, site/, and DCC-specific tool modules.
- These packages bridge the generic core into host APIs and startup conventions for Blender, Houdini, Maya, Nuke, Substance Painter, and Substance Designer.
- The architecture keeps host-specific code localized, while shared pipeline rules remain in src/core.
11. Embedded runtime/site customization
- The site/ trees contain startup scripts, shelves, menus, packages, HDAs/gizmos/toolsets, and configuration files that are installed into each DCC’s runtime environment.
- Examples include Maya shelves and startup scripts, Houdini packages/OTLs/python startup, Nuke menu and gizmos, and Substance Painter startup hooks.
- This is how the repo injects pipeline behavior into third-party applications without patching those applications directly.
12. DCC-specific workflows
- Maya includes substantial subdomains for asset files, shot files, layout, rig building, publishing, playblasts, and symmetry tools.
- Houdini includes Solaris/Lop-centric publishing, playblast, shading, shot processing, and bundled third-party tools.
- Blender has a smaller but focused integration for asset-file handling, launch, publish, and plugin registration.
- Nuke, Substance Painter, and Substance Designer are narrower integrations centered on launch, startup, and specialized export/publish flows.
13. Shared resources and packaged data
- resources/ holds first-party non-Python assets such as HDRIs, icons, splash screens, textures, USD kinds, and Substance Designer files.
- These resources are part of the runtime data model, not just documentation; they support launchers, tool UIs, and render/lookdev workflows.
- desktop_launchers/ and install_desktop_launchers.py provide OS-level entry points for launching the DCC tooling.
14. Operational boundaries and configuration
- The repo depends on external environment configuration and studio infrastructure, especially src/env.py, which is intentionally documented separately.
- That configuration likely supplies executable paths, ShotGrid credentials, and site-specific filesystem roots.
- The project also includes Git hooks and CI, but they support code quality rather than altering runtime architecture.
Main entry points: dispatch.py Dispatch · versioning Versioning · maya Maya · houdini Houdini
Components 23
Entry & framework
Process entrypoint, dispatch, and runtime abstraction.
- Main entry · Top-level process entrypoint for the pipeline tools.
src/__main__.py - Dispatch · Selects the appropriate DCC launcher/runtime path.
src/framework/dispatch.py - Interface · Defines the shared launch/runtime abstraction boundary.
src/framework/interface.py - Launcher · Common launcher helpers used by DCC integrations.
src/framework/launcher.py - Env config · Studio/site-specific paths, credentials, and executable settings.
src/env.py.md
Shared pipeline core
Cross-DCC business logic and shared services.
- Versioning · Central model, service, and storage for versioned pipeline artifacts.
src/core/versioning - Asset model · Asset path conventions and asset version adaptation.
src/core/asset - Shot model · Shot version adaptation and shot-oriented content rules.
src/core/shot - Playblast · Shared playblast naming, encoding, temp handling, and review upload flow.
src/core/playblast - ShotGrid · Client boundary for production metadata, entities, caching, and paths.
src/core/shotgrid - Telemetry · Event emission, local spool, ingestion, and standalone telemetry runtime.
src/core/telemetry - Texture tools · Shared texture conversion utilities used by DCC workflows.
src/core/texture/converter.py - Shared UI · Common dialogs, progress UI, save/version browser components.
src/core/ui - Data structs · Reusable serializable structures for timeline, material, and JSON data.
src/core/struct
DCC integrations
Host-specific launch, runtime, and workflows for each DCC.
- Maya · Maya launch/runtime, site customization, and deep workflow modules.
src/dcc/maya - Houdini · Houdini launch/runtime plus Solaris, publish, playblast, and site assets.
src/dcc/houdini - Blender · Blender launch, asset handling, publish, and addon registration.
src/dcc/blender - Nuke · Nuke launch plus studio menu, gizmo, and toolset bundle.
src/dcc/nuke - Painter · Substance Painter launch, runtime, startup hooks, and export publish flow.
src/dcc/substance_painter - Designer · Substance Designer launch and packaged project/site configuration.
src/dcc/substance_designer
Bundled assets
Packaged runtime data, desktop launchers, and telemetry backend.
- Resources · First-party runtime assets such as HDRIs, icons, splash screens, and USD kinds.
resources - Desktop launchers · OS-level desktop files and installer for launching the tool suite.
desktop_launchers - Telemetry backend · PostgreSQL schema and Grafana provisioning for telemetry storage and dashboards.
telemetry-backend
Connections 27
- Main entry → Dispatch: starts · Process entrypoint hands off to shared dispatch.
- Dispatch → Interface: uses · Dispatch routes through the common runtime contract.
- Dispatch → Launcher: invokes · Shared launcher helpers coordinate startup.
- Launcher → Env config: reads config · Executable paths and studio settings come from external env config.
- Dispatch → Maya: selects · Dispatch can enter the Maya integration.
- Dispatch → Houdini: selects · Dispatch can enter the Houdini integration.
- Dispatch → Blender: selects · Dispatch can enter the Blender integration.
- Dispatch → Nuke: selects · Dispatch can enter the Nuke integration.
- Dispatch → Painter: selects · Dispatch can enter Substance Painter.
- Dispatch → Designer: selects · Dispatch can enter Substance Designer.
- Maya → Versioning: uses · Maya workflows rely on shared versioning services.
- Maya → Playblast: uses · Maya playblasts wrap the shared playblast pipeline.
- Maya → ShotGrid: uses · Maya tools consume production metadata from ShotGrid.
- Houdini → Versioning: uses · Houdini workflows rely on shared versioning services.
- Houdini → Playblast: uses · Houdini playblasts wrap the shared playblast pipeline.
- Houdini → ShotGrid: uses · Houdini tools consume production metadata from ShotGrid.
- Blender → Versioning: uses · Blender workflows rely on shared versioning services.
- Painter → Texture tools: uses · Painter export flows depend on texture and material utilities.
- Painter → Versioning: uses · Painter publish flows participate in versioned artifacts.
- Telemetry → Telemetry backend: uploads to · Telemetry events are stored and visualized by the backend stack.
- Resources → Playblast: supports · Packaged fonts and runtime assets support playblast/review output.
- Desktop launchers → Main entry: starts · Desktop entries launch the pipeline entrypoint.
- Shared UI → Versioning: supports · Dialogs and browser UI sit on top of versioned artifacts.
- Asset model → Versioning: feeds · Asset naming and adapters normalize into versioning.
- Shot model → Versioning: feeds · Shot naming and adapters normalize into versioning.
- Data structs → Versioning: supports · Serializable domain structs are shared across services and UI.
- ShotGrid → Env config: configured by · Backend credentials and paths are supplied externally.