# controlm/automation-api-quickstart architecture

> Architecture diagram of the GitHub repository controlm/automation-api-quickstart, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/controlm/automation-api-quickstart
- Repository: https://github.com/controlm/automation-api-quickstart
- Video tour (about a minute): https://gitdiagram.com/controlm/automation-api-quickstart/video

## Overview

##### Project type and purpose
- This is not a monolithic application but a “samples repository” demonstrating how to use the Control-M Automation API.
- It provides end-to-end examples (JSON job-flow definitions, scripts in Python/Node/Powershell/SQL, Java JUnit tests, Dockerfiles, Kubernetes manifests).
- Two major categories at the top level:
– control-m – samples for the standard Control-M Automation API
– helix-control-m – samples for the Helix Control-M API flavor

##### File structure
- Top-level: LICENSE, README.md, two folders (control-m, helix-control-m).
- Under each category: numbered folders 101, 102, 103,… grouping samples by capability (first job, file transfer + DB, Hadoop/Spark, script jobs, code deployment, Docker, onboarding, JUnit, DEV→PROD, corrective flows, REST examples, CSV→JSON, Kubernetes agent, monitoring).
- Inside each sample folder:
– A README.md explaining the sample
– One or more definitions (.json for flows, descriptors, Kubernetes YAML, Dockerfiles)
– Scripts or code in Python, Shell, PowerShell, Node, Java, SQL
- No large server-side code: everything is client-side artifacts that invoke the Automation API to drive a Control-M environment.

##### Key architecture insights from README
- This is a quickstart/demo for the Control-M Automation API.
- Users need: a local Docker Workbench image (Control-M environment), the ctm-cli Automation API CLI, and connectivity to a Control-M server.
- Samples show how to deploy job definitions, run jobs, query status, build agents for Docker or Kubernetes.

##### Main components
- Client workstation (developer’s laptop)
– Sample code folders (control-m / helix-control-m)
– Automation API CLI (ctm-cli) running locally or in Docker
- Control-M server environment
– Control-M server REST API endpoint
– Scheduler and database
- Control-M agents
– Traditional agents (installed on hosts)
– Docker agents (built from Dockerfiles in samples)
– Kubernetes agents (statefulset, YAML manifests)
- External execution targets
– File servers (for file-transfer jobs)
– Databases (for SQL jobs)
– Hadoop/Spark cluster (for big-data jobs)
– Generic compute (shell/PowerShell/Node/Python)
- Monitoring / corrective flows
– Optional external monitoring listener (alertsListener.ts sample)
– Sample corrective job flows

##### Relationships and data flows
- Developer edits JSON or script → invokes ctm-cli (or uses sample code via REST libraries / JUnit) → sends HTTP calls (POST/PUT) to Control-M REST API → Control-M server persists job-flow definitions in its database → Scheduler triggers jobs at runtime → Agents pick up job steps → Jobs execute on external resources → Agents report status back → Control-M server updates state → CLI / sample code polls GET /status.

##### Important architectural patterns
- Separation of concerns
– Client side (samples, JSON, scripts) vs server side (Control-M components) vs execution side (agents + targets)
- API-driven deployment
– All artifacts are declarative (JSON, YAML) and deployed by REST calls
- Extensibility
– Demonstrated via multiple language bindings (CLI/shell, Python, Node, PowerShell, Java)
- Containerization
– Dockerfiles to package agents
– Kubernetes manifests for stateful agents

##### Checklist
- All three layers with clear separation
- Docker build contexts (samples that contain Dockerfile), Kubernetes manifests (stateful_ha.yaml)
- Technology stacks alongside components (Java JUnit, Node/Axios, PowerShell, Python, SQL)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Developer Workstation Layer
    subgraph "Developer Workstation"
        direction TB
        REPO["Samples Repository\nREADME.md, LICENSE"]:::client
        CONTROL_M["control-m/"]:::client
        HELIX["helix-control-m/"]:::client
        CLI["ctm-cli (Docker/Local)"]:::client
        JOB1["First-Job JSON\n(AutomationAPISampleFlow.json)"]:::client
        FILEDB_JSON["FileTransfer JSON\n(AutomationAPIFileTransferDatabaseSampleFlow.json)"]:::client
        FILEDB_SQL["FileTransfer SQL\n(processRecords.sql)"]:::client
        HDP_JSON["Hadoop/Spark JSON\n(AutomationAPISampleHadoopFlow.json)"]:::client
        PYCSV["CSV2JSON Python\n(aapi_csv2json.py)"]:::client
        DOCKER_CLI["ctm-cli Dockerfile\n(Dockerfile)"]:::client
        K8S_MAN["K8s StatefulSet YAML\n(stateful_ha.yaml)"]:::client
        ALERTS["External Monitoring Listener\n(alertsListener.ts)"]:::client
    end

    %% Control-M Environment Layer
    subgraph "Control-M Environment"
        direction TB
        API["Control-M REST API\nPOST/PUT /flow\nGET /status"]:::server
        SCHED["Scheduler"]:::server
        DB["Control-M DB"]:::server
    end

    %% Execution Layer
    subgraph "Execution Layer"
        direction TB
        subgraph "Agents"
            direction TB
            TRAD["Traditional Agents"]:::agent
            DOCKER_AGENT["Docker Agents"]:::agent
            K8S_AGENT["Kubernetes Agents"]:::agent
        end
        subgraph "External Resources"
            direction TB
            FS["File Server"]:::external
            DB_TGT["Databases"]:::external
            HDP["Hadoop/Spark Cluster"]:::external
            EXEC["Generic Compute"]:::external
        end
    end

    %% Connections
    CONTROL_M --> CLI
    HELIX --> CLI
    JOB1 --> CLI
    FILEDB_JSON --> CLI
    FILEDB_SQL --> CLI
    HDP_JSON --> CLI
    PYCSV --> CLI
    DOCKER_CLI --> CLI
    K8S_MAN --> CLI
    ALERTS --> CLI

    CLI -->|"POST /flow"| API
    API --> SCHED
    SCHED --> TRAD
    SCHED --> DOCKER_AGENT
    SCHED --> K8S_AGENT

    TRAD --> FS
    TRAD --> DB_TGT
    TRAD --> HDP
    TRAD --> EXEC

    DOCKER_AGENT --> FS
    DOCKER_AGENT --> DB_TGT
    DOCKER_AGENT --> HDP
    DOCKER_AGENT --> EXEC

    K8S_AGENT --> FS
    K8S_AGENT --> DB_TGT
    K8S_AGENT --> HDP
    K8S_AGENT --> EXEC

    TRAD -->|"status update"| API
    DOCKER_AGENT -->|"status update"| API
    K8S_AGENT -->|"status update"| API

    %% Click Events
    click REPO "https://github.com/controlm/automation-api-quickstart/blob/master/README.md"
    click CONTROL_M "https://github.com/controlm/automation-api-quickstart/tree/master/control-m/"
    click HELIX "https://github.com/controlm/automation-api-quickstart/tree/master/helix-control-m/"
    click JOB1 "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/101-create-first-job-flow/AutomationAPISampleFlow.json"
    click FILEDB_JSON "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/101-running-file-transfer-and-database-query-job-flow/AutomationAPIFileTransferDatabaseSampleFlow.json"
    click FILEDB_SQL "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/101-running-file-transfer-and-database-query-job-flow/processRecords.sql"
    click HDP_JSON "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/101-running-hadoop-spark-job-flow/AutomationAPISampleHadoopFlow.json"
    click PYCSV "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/202-convert-csv2json-using-python/aapi_csv2json.py"
    click DOCKER_CLI "https://github.com/controlm/automation-api-quickstart/tree/master/control-m/102-build-docker-containers-for-automation-api/alpinelinux-ctm-cli/Dockerfile"
    click K8S_MAN "https://github.com/controlm/automation-api-quickstart/blob/master/control-m/301-statefulset-agent-to-run-k8s-jobs-using-ai-job/9.0.20.200/stateful_ha.yaml"
    click ALERTS "https://github.com/controlm/automation-api-quickstart/blob/master/helix-control-m/302-external-monitoring-tools-example/alertsListener.ts"

    %% Styles
    classDef client fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
    classDef server fill:#DFFFE0,stroke:#228B22,color:#228B22;
    classDef agent fill:#FFE5B4,stroke:#FF8C00,color:#FF8C00;
    classDef external fill:#F0F0F0,stroke:#A9A9A9,color:#555;
```
