# crim-ca/weaver architecture

> Architecture diagram of the GitHub repository crim-ca/weaver, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/crim-ca/weaver
- Repository: https://github.com/crim-ca/weaver

## Overview

Weaver is an OGC API Processes execution-management system that can also act as an application-deployment and execution service. Its principal workflow accepts process/job requests, represents process definitions using CWL, dispatches workflow work to registered remote providers or ADES instances, and exposes job results and provenance. The repository also includes a WPS interface, process discovery and registration, quotation, and file transformations. The README establishes the overall EMS/ADES workflow; source excerpts are partial, so internal call paths and several subsystem connections are intentionally not inferred.

Main entry points: [`weaver/processes/execution.py`](https://github.com/crim-ca/weaver/blob/master/weaver/processes/execution.py) (Process Execution), [`weaver/wps_restapi/jobs/jobs.py`](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/jobs/jobs.py) (Jobs API), [`weaver/processes/wps_workflow.py`](https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_workflow.py) (Workflow Dispatch), [`weaver/wps_restapi/processes/processes.py`](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/processes/processes.py) (Processes API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Request Interfaces"]
  node_rest["Processes API<br/>[processes.py]"]
  node_jobapi["Jobs API<br/>[jobs.py]"]
  node_wps["WPS Service<br/>[service.py]"]
  node_openapi["Request Schemas"]
  node_providers["Remote Providers<br/>[providers.py]"]
end

subgraph group_process["Process Execution"]
  node_workflow["Workflow Dispatch<br/>[wps_workflow.py]"]
  node_execution["Process Execution<br/>[execution.py]"]
  node_package["CWL Packages<br/>[wps_package.py]"]
  node_conversion["Format Conversion<br/>[convert.py]"]
  node_builtin["Built-in Processes"]
end

subgraph group_state["Jobs and State"]
  node_jobs["Job Operations<br/>[jobs.py]"]
  node_jobmodel["Job Model<br/>[datatype.py]"]
  node_store[("Persistent Store<br/>[mongodb.py]")]
  node_database["Database Access<br/>[__init__.py]"]
end

subgraph group_support["Supporting Capabilities"]
  node_provenance["Provenance<br/>[provenance.py]"]
  node_quotation["Quotation<br/>[estimation.py]"]
  node_vault["File Vault<br/>[views.py]"]
  node_transform["File Transforms<br/>[handlers.py]"]
  node_opensearch["Process Discovery<br/>[opensearch.py]"]
end

node_client(("API Client"))
node_ades["Remote ADES"]

node_client -->|"requests processes"| node_rest
node_client -->|"submits and queries jobs"| node_jobapi
node_client -->|"submits WPS requests"| node_wps
node_workflow -->|"dispatches remote work"| node_ades
node_workflow -->|"uses registered providers"| node_providers
node_execution -->|"executes process package"| node_package
node_package -->|"interoperates formats"| node_conversion
node_jobapi -->|"handles job operations"| node_jobs
node_database -->|"provides database backend"| node_store
node_jobapi -->|"exposes job provenance"| node_provenance
node_client -.->|"requests quotations"| node_quotation
node_client -.->|"accesses stored files"| node_vault
node_execution -.->|"supports built-in processes"| node_builtin
node_rest -.->|"supports process search"| node_opensearch
node_execution -.->|"uses file transformations"| node_transform

click node_rest "https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/processes/processes.py"
click node_jobapi "https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/jobs/jobs.py"
click node_wps "https://github.com/crim-ca/weaver/blob/master/weaver/wps/service.py"
click node_openapi "https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/swagger_definitions.py"
click node_providers "https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/providers/providers.py"
click node_workflow "https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_workflow.py"
click node_execution "https://github.com/crim-ca/weaver/blob/master/weaver/processes/execution.py"
click node_package "https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_package.py"
click node_conversion "https://github.com/crim-ca/weaver/blob/master/weaver/processes/convert.py"
click node_builtin "https://github.com/crim-ca/weaver/tree/master/weaver/processes/builtin"
click node_jobs "https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/jobs/jobs.py"
click node_jobmodel "https://github.com/crim-ca/weaver/blob/master/weaver/datatype.py"
click node_store "https://github.com/crim-ca/weaver/blob/master/weaver/store/mongodb.py"
click node_database "https://github.com/crim-ca/weaver/blob/master/weaver/database/__init__.py"
click node_provenance "https://github.com/crim-ca/weaver/blob/master/weaver/provenance.py"
click node_quotation "https://github.com/crim-ca/weaver/blob/master/weaver/quotation/estimation.py"
click node_vault "https://github.com/crim-ca/weaver/blob/master/weaver/vault/views.py"
click node_transform "https://github.com/crim-ca/weaver/blob/master/weaver/transform/handlers.py"
click node_opensearch "https://github.com/crim-ca/weaver/blob/master/weaver/processes/opensearch.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_rest,node_jobapi,node_wps,node_openapi,node_providers,node_client toneBlue
class node_workflow,node_execution,node_package,node_conversion,node_builtin toneAmber
class node_jobs,node_jobmodel,node_store,node_database toneMint
class node_provenance,node_quotation,node_vault,node_transform,node_opensearch toneRose
class node_ades toneIndigo
```

## Components

### Request Interfaces

- [Processes API](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/processes/processes.py): `weaver/wps_restapi/processes/processes.py`
- [Jobs API](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/jobs/jobs.py): `weaver/wps_restapi/jobs/jobs.py`
- [WPS Service](https://github.com/crim-ca/weaver/blob/master/weaver/wps/service.py): `weaver/wps/service.py`
- [Request Schemas](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/swagger_definitions.py): `weaver/wps_restapi/swagger_definitions.py`
- [Remote Providers](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/providers/providers.py): `weaver/wps_restapi/providers/providers.py`

### Process Execution

- [Workflow Dispatch](https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_workflow.py): `weaver/processes/wps_workflow.py`
- [Process Execution](https://github.com/crim-ca/weaver/blob/master/weaver/processes/execution.py): `weaver/processes/execution.py`
- [CWL Packages](https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_package.py): `weaver/processes/wps_package.py`
- [Format Conversion](https://github.com/crim-ca/weaver/blob/master/weaver/processes/convert.py): `weaver/processes/convert.py`
- [Built-in Processes](https://github.com/crim-ca/weaver/tree/master/weaver/processes/builtin): `weaver/processes/builtin`

### Jobs and State

- [Job Operations](https://github.com/crim-ca/weaver/blob/master/weaver/wps_restapi/jobs/jobs.py): `weaver/wps_restapi/jobs/jobs.py`
- [Job Model](https://github.com/crim-ca/weaver/blob/master/weaver/datatype.py): `weaver/datatype.py`
- [Persistent Store](https://github.com/crim-ca/weaver/blob/master/weaver/store/mongodb.py): `weaver/store/mongodb.py`
- [Database Access](https://github.com/crim-ca/weaver/blob/master/weaver/database/__init__.py): `weaver/database/__init__.py`

### Supporting Capabilities

- [Provenance](https://github.com/crim-ca/weaver/blob/master/weaver/provenance.py): `weaver/provenance.py`
- [Quotation](https://github.com/crim-ca/weaver/blob/master/weaver/quotation/estimation.py): `weaver/quotation/estimation.py`
- [File Vault](https://github.com/crim-ca/weaver/blob/master/weaver/vault/views.py): `weaver/vault/views.py`
- [File Transforms](https://github.com/crim-ca/weaver/blob/master/weaver/transform/handlers.py): `weaver/transform/handlers.py`
- [Process Discovery](https://github.com/crim-ca/weaver/blob/master/weaver/processes/opensearch.py): `weaver/processes/opensearch.py`

### Other components

- **API Client**
- **Remote ADES**

## Connections

- API Client → Processes API: requests processes
- API Client → Jobs API: submits and queries jobs
- API Client → WPS Service: submits WPS requests
- Workflow Dispatch → Remote ADES: dispatches remote work
- Workflow Dispatch → Remote Providers: uses registered providers
- Process Execution → CWL Packages: executes process package
- CWL Packages → Format Conversion: interoperates formats (evidence: `weaver/processes/wps_package.py`)
- Jobs API → Job Operations: handles job operations (evidence: `weaver/wps_restapi/jobs/jobs.py`)
- Database Access → Persistent Store: provides database backend (evidence: `weaver/database/__init__.py`)
- Jobs API → Provenance: exposes job provenance
- API Client → Quotation: requests quotations
- API Client → File Vault: accesses stored files
- Process Execution → Built-in Processes: supports built-in processes
- Processes API → Process Discovery: supports process search (evidence: `weaver/wps_restapi/processes/processes.py`)
- Process Execution → File Transforms: uses file transformations
