# tracecathq/tracecat architecture

> Architecture diagram of the GitHub repository tracecathq/tracecat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tracecathq/tracecat
- Repository: https://github.com/tracecathq/tracecat

## Overview

##### Project Type & Purpose
- Tracecat is a full-stack, open-source workflow automation platform for security and IT engineers.
- It consists of:
– A user-facing frontend (Next.js/React with a no-code UI)
– A Python-based backend REST API and orchestration engine
– A standalone “registry” CLI/tool for managing and publishing YAML-based integration templates

##### High-Level Architecture
- Clients (web browser or CLI) → Next.js Frontend & Registry CLI
- Frontend → HTTP(S) → Backend API (FastAPI/Starlette)
- Backend API → SQL (PostgreSQL), Temporal Server (workflow orchestration), Redis (caching/locks), optional S3 (secrets, artifacts)
- Temporal workers (Python processes) ← Backend schedules workflows/events → execute long-running tasks
- Registry CLI ← Python package tracecat_registry → interacts with local/remote template store (YAML files + schema validation)

##### Key Components to Model
A. Frontend (folder “frontend”)
– Next.js app
– Auth pages (OAuth, SAML), organization & workspace UIs, workflow builder, tables, registry browser
– API client layer (OpenAPI-generated TypeScript client)
B. API Server (folder “tracecat/api” + related modules)
– Routers (auth, cases, workflows, registry, secrets, webhooks, etc.)
– Services & business logic
– SQLAlchemy models + Alembic migrations
– Middleware (rate-limit, security, request tracing)
C. Orchestration & Workers
– tracecat/dsl scheduler & executor
– Temporal server & Python SDK
– Worker processes drive long-running tasks (email, HTTP calls, integrations)
D. Persistence & Infrastructure
– PostgreSQL (primary datastore)
– Redis (locking, caching, rate limits)
– Object store (S3-compatible—for secrets, artifacts)
– Temporal server (workflow orchestration)
E. Registry Package & CLI (folder “registry/tracecat_registry”)
– CLI entrypoint, template loader, schema validator, integrations library
– Publishes/validates YAML templates for integrations

##### Relationships & Data Flows
- User → (1) HTTP → Frontend → (2) REST calls → API Server → (3a) SQL (Postgres) / (3b) Temporal (schedule a workflow) / (3c) Redis → responds → Frontend → renders
- API Server → (4) Temporal Workers process workflow steps → call external services (AWS, Slack, LDAP, etc.) → update state back in Postgres or signal workflow progress
- Registry CLI → (A) Local filesystem (templates) or remote API → (B) Validates & publishes templates

##### Architectural Patterns & Principles
- Layered architecture (routers → services → data access)
- Event-driven orchestration (Temporal) for resilience & scaling
- Separation of concerns: UI vs. API vs. orchestration vs. CLI
- Infrastructure as code (Docker Compose, Terraform for AWS Fargate)
- CI/CD workflows (GitHub Actions for build, lint, test, release)

##### Detailing Enterprise vs. OSS
- “ee” directory as an extension module loaded by the backend for enterprise features (e.g., Kubernetes sandbox, advanced connectors)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Zone
    subgraph "Frontend" 
        direction TB
        Browser["Web Browser"]:::ui
        FrontendApp["Next.js App"]:::frontend
        Auth["Auth Pages"]:::frontend
        OrgUI["Org & Workspace UIs"]:::frontend
        Builder["Workflow Builder"]:::frontend
        APIClient["API Client (OpenAPI)"]:::frontend
    end

    %% API Server Zone
    subgraph "API Server" 
        direction TB
        App["app.py"]:::backend
        Common["common.py"]:::backend
        Routers["Routers"]:::backend
        Services["Services"]:::backend
        Models["SQLAlchemy Models"]:::backend
        DB["DB Adapters & Schemas"]:::backend
        Middleware["Middleware"]:::backend
        Migrations["Alembic Migrations"]:::backend
        EE["Enterprise Module"]:::backend
    end

    %% Orchestration Zone
    subgraph "Orchestration" 
        direction TB
        TemporalServer["Temporal Server"]:::workflow
        Scheduler["DSL Scheduler & Executor"]:::workflow
        Workers["Worker Processes"]:::workflow
    end

    %% Persistence & Infrastructure Zone
    subgraph "Persistence & Infra" 
        direction TB
        Postgres["PostgreSQL"]:::db
        Redis["Redis"]:::db
        S3["Object Store (S3)"]:::db
    end

    %% External Services Zone
    subgraph "External Services" 
        direction TB
        AWS["AWS"]:::external
        Slack["Slack"]:::external
        LDAP["LDAP"]:::external
    end

    %% Registry CLI Zone
    subgraph "Registry CLI" 
        direction TB
        RegistryPkg["tracecat_registry"]:::cli
        CLIEntrypoint["CLI Entrypoint"]:::cli
        Loader["Template Loader & Validator"]:::cli
        Schemas["Schemas"]:::cli
        Integrations["Integrations Library"]:::cli
        Templates["Templates"]:::cli
    end

    %% Data Flows
    Browser --> FrontendApp
    FrontendApp --> Auth
    FrontendApp --> OrgUI
    FrontendApp --> Builder
    FrontendApp --> APIClient
    APIClient -->|"REST API"| App
    App -->|"SQL Queries"| Postgres
    App -->|"Cache/Locks"| Redis
    App -->|"Schedule Workflow"| TemporalServer
    App --> Scheduler
    Scheduler --> Workers
    Workers -->|"call external"| AWS
    Workers -->|"call external"| Slack
    Workers -->|"call external"| LDAP
    Workers -->|"update state"| Postgres
    Workers -->|"signal progress"| TemporalServer
    RegistryPkg --> CLIEntrypoint
    CLIEntrypoint --> Loader
    Loader --> Schemas
    Loader --> Templates
    CLIEntrypoint -->|"Publish/Validate"| App

    %% Click Events
    click FrontendApp "https://github.com/tracecathq/tracecat/tree/main/frontend/"
    click Auth "https://github.com/tracecathq/tracecat/tree/main/frontend/src/app/auth/"
    click OrgUI "https://github.com/tracecathq/tracecat/tree/main/frontend/src/app/organization/"
    click Builder "https://github.com/tracecathq/tracecat/tree/main/frontend/src/components/builder/"
    click APIClient "https://github.com/tracecathq/tracecat/tree/main/frontend/src/client/"
    click App "https://github.com/tracecathq/tracecat/blob/main/tracecat/api/app.py"
    click Common "https://github.com/tracecathq/tracecat/blob/main/tracecat/api/common.py"
    click DB "https://github.com/tracecathq/tracecat/blob/main/tracecat/db/adapter.py"
    click Middleware "https://github.com/tracecathq/tracecat/blob/main/tracecat/middleware/rate_limit.py"
    click Migrations "https://github.com/tracecathq/tracecat/tree/main/alembic/"
    click Scheduler "https://github.com/tracecathq/tracecat/blob/main/tracecat/dsl/scheduler.py"
    click Workers "https://github.com/tracecathq/tracecat/blob/main/tracecat/executor/engine.py"
    click RegistryPkg "https://github.com/tracecathq/tracecat/tree/main/registry/tracecat_registry/"
    click CLIEntrypoint "https://github.com/tracecathq/tracecat/blob/main/registry/tracecat_registry/cli/entrypoint.py"
    click Loader "https://github.com/tracecathq/tracecat/blob/main/registry/tracecat_registry/_internal/registry.py"
    click Schemas "https://github.com/tracecathq/tracecat/tree/main/registry/tracecat_registry/schemas/"
    click Integrations "https://github.com/tracecathq/tracecat/tree/main/registry/tracecat_registry/integrations/"
    click Templates "https://github.com/tracecathq/tracecat/tree/main/registry/tracecat_registry/templates/"

    %% Styles
    classDef ui fill:#DDEBF7,stroke:#1F497D,color:#1F497D
    classDef frontend fill:#E2EFDA,stroke:#4472C4,color:#4472C4
    classDef backend fill:#FCE4D6,stroke:#C65911,color:#C65911
    classDef workflow fill:#FFF2CC,stroke:#7F6000,color:#7F6000
    classDef db fill:#F8CBAD,stroke:#B45F06,color:#B45F06
    classDef external fill:#E1DFEC,stroke:#4B0082,color:#4B0082
    classDef cli fill:#D9EAD3,stroke:#274E13,color:#274E13
```
