# atlanhq/application-sdk architecture

> Architecture diagram of the GitHub repository atlanhq/application-sdk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/atlanhq/application-sdk
- Repository: https://github.com/atlanhq/application-sdk

## Overview

##### Project type & purpose
- This is an “open-source toolkit / PaaS SDK” in Python.
- It provides:
– A FastAPI server for local development & extension integrations
– A Temporal-driven worker for running workflows & activities
– A suite of pluggable clients, inputs, outputs, and transformers
– Doc-generation and test utilities

##### High-level components
A. Core SDK library (application_sdk/)
1. activities/ – business-logic units (metadata_extraction, query_extraction)
2. workflows/ – orchestration definitions (metadata_extraction, query_extraction)
3. clients/ – connectors to Temporal, SQL, workflow orchestration APIs
4. server/fastapi/ – HTTP server, routers, middleware for local dev & partner apps
5. worker.py – entry point for Temporal worker, registering activities & workflows
6. common/ – shared utilities (AWS creds, dataframe helpers, error codes)
7. inputs/ & outputs/ – pluggable I/O adapters (SQL, JSON, Parquet, object store, statestore, secretstore, eventstore, iceberg)
8. transformers/ – data-transformation templates (SQL schemas → metadata resources)
9. observability/ – instrumentation adapters (metrics, traces, logging)
10. docgen/ – automated documentation exporters (mkdocs), parsers, model definitions
11. test_utils/ – e2e, Hypothesis, scale-data generators

B. Infrastructure definitions (components/)
– Kubernetes or CloudFormation YAMLs for secretstore, objectstore, statestore, eventstore

C. Examples & docs
– examples/ showing how a partner app uses the SDK
– docs/ – user guides, architecture.md, getting-started

D. External systems
1. Atlan platform (REST API & metadata store)
2. Temporal server (workflow engine)
3. External data sources & sinks: SQL databases, object stores, secret stores, iceberg catalog, event buses, state stores
4. Observability backends (Prometheus, Jaeger, CloudWatch, etc.)

##### Relationships & data flows
- A partner’s application → imports SDK → invokes CLI/server or directly instantiates clients/workflows
- If using FastAPI: HTTP request → SDK server layer → triggers a workflow (via workflow client)
- workflow client → Temporal server → orchestrates activities → worker pulls tasks
- Worker → executes activities (e.g., metadata_extraction, query_extraction) → uses input adapters to read data → transformer modules → uses output adapters to write results → may call Atlan REST APIs via client or SQL via client/sql.py
- Observability decorators wrap activities & server routes, emitting metrics & traces to external backends
- Docgen module introspects the SDK’s manifest and directory structure to produce mkdocs pages

##### Key architectural patterns & principles
– Separation of concerns: server vs. worker vs. clients vs. IO vs. transformers
– Adapter/strategy pattern for inputs & outputs (common interface, pluggable implementations)
– Decorator pattern for observability
– Declarative workflow definitions (workflows/) vs. imperative activity code (activities/)
– Infrastructure as code under components/ for reproducible environments

##### Detail level
- Major sub-packages in the Core SDK layer
- Both synchronous (HTTP) and asynchronous (Temporal) flows
- Infrastructure YAMLs (components/) as deployment artifacts

## Diagram (Mermaid)

```mermaid
graph TB
    %% Partner Application
    subgraph "Partner Application"
        App["Partner Application"]
    end

    %% Core SDK
    subgraph "Core SDK Library" 
        direction TB
        subgraph "Server"
            direction TB
            FastAPI["FastAPI Server"]:::sdkComp
        end
        subgraph "Worker"
            direction TB
            WorkerEP["Temporal Worker Entry Point"]:::sdkComp
        end
        subgraph "Core Modules"
            direction TB
            Common["Common Utilities"]:::sdkComp
            Clients["Clients"]:::sdkComp
            subgraph "Activities"
                direction TB
                Activities["Activities Module"]:::sdkComp
            end
            subgraph "Workflows"
                direction TB
                Workflows["Workflows Module"]:::sdkComp
            end
            subgraph "Inputs & Outputs"
                direction TB
                Inputs["Input Adapters"]:::sdkComp
                Outputs["Output Adapters"]:::sdkComp
            end
            Transformers["Data Transformers"]:::sdkComp
            Observability["Observability"]:::crossComp
            Docgen["Documentation Generation"]:::crossComp
            TestUtils["Test Utilities"]:::crossComp
        end
    end

    %% Infrastructure Definitions
    Infra["Infrastructure Definitions"]:::infraComp

    %% Examples & Docs
    Examples["Examples & Sample Apps"]:::sdkComp
    Docs["User Guides & Architecture Docs"]:::sdkComp

    %% External Systems
    subgraph "External Systems"
        direction TB
        Atlan["Atlan Platform"]:::external
        Temporal["Temporal Server"]:::external
        DataSources["Data Sources & Sinks"]:::infraComp
        ObservBackends["Observability Backends"]:::infraComp
    end

    %% Relationships
    App -->|imports SDK & invokes| FastAPI
    App -->|imports SDK & invokes| WorkerEP

    FastAPI -->|HTTP request| Clients
    Clients -->|calls workflows| Workflows
    Workflows -->|orchestrate activities| Temporal
    Temporal -->|task pull| WorkerEP
    WorkerEP -->|execute| Activities
    Activities -->|read via| Inputs
    Activities -->|write via| Outputs
    Activities -->|transform via| Transformers
    Activities -->|Atlan API calls| Atlan
    Activities -->|SQL via clients| Clients

    FastAPI -->|observability| Observability
    WorkerEP -->|observability| Observability
    Observability -->|metrics/traces| ObservBackends

    Docgen -->|reads manifest| CoreModules(( ))
    Docgen -->|outputs mkdocs| Docs

    TestUtils -->|provides| Examples

    Infra -->|deploys| DataSources
    Infra -->|deploys| ObservBackends

    %% Click Events
    click Core_SDK_Library "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/"
    click Activities "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/activities/"
    click Workflows "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/workflows/"
    click Clients "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/clients/"
    click FastAPI "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/server/fastapi/"
    click WorkerEP "https://github.com/atlanhq/application-sdk/blob/main/application_sdk/worker.py"
    click Common "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/common/"
    click Inputs "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/inputs/"
    click Outputs "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/outputs/"
    click Transformers "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/transformers/"
    click Observability "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/observability/"
    click Docgen "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/docgen/"
    click TestUtils "https://github.com/atlanhq/application-sdk/tree/main/application_sdk/test_utils/"
    click Infra "https://github.com/atlanhq/application-sdk/tree/main/components/"
    click Examples "https://github.com/atlanhq/application-sdk/tree/main/examples/"
    click Docs "https://github.com/atlanhq/application-sdk/tree/main/docs/"

    %% Styles
    classDef sdkComp fill:#D0E8FF,stroke:#1E70BF,color:#000
    classDef infraComp fill:#DFFFE0,stroke:#4CAF50,color:#000
    classDef external fill:#FFE0B2,stroke:#FB8C00,color:#000
    classDef crossComp fill:#E8D0FF,stroke:#7B1FA2,color:#000
```
