# netskopeoss/ta_cloud_exchange architecture

> Architecture diagram of the GitHub repository netskopeoss/ta_cloud_exchange, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/netskopeoss/ta_cloud_exchange
- Repository: https://github.com/netskopeoss/ta_cloud_exchange

## Overview

##### Project type and purpose
- This is a self-hosted, containerized “Cloud Exchange” integration platform (Netskope CE).
- It uses Docker-Compose (and Podman-Compose) to spin up a Python-based application, a MongoDB datastore, a RabbitMQ broker, and support services (data initialization, cert management, diagnostics).
- It provides a web UI and REST API over HTTPS for ingesting and routing cloud-security events via plugins.

##### File structure
Top-level files & scripts
- docker-compose.yml / podman-compose.yml – defines all containers & their relationships.
- setup, start, stop – wrapper scripts that bootstrap the environment and launch/teardown the Compose stack.
- database-migrate.py, migrate.sh, migrate_ce – for schema/data migrations.
- diagnose – collects logs and environment info.
- replica_set, restore_ha_backup – for MongoDB HA and backup/restore.
Data directory
- data/ca_certs – trusted CA certificates.
- data/ssl_certs – TLS cert/key for the HTTPS endpoint.
- data/custom_plugins – Python hooks or extensions mounted into the app.
- data/database-init – scripts to initialize MongoDB (create users, indexes).
- data/mongo-data – persistent MongoDB data volume.
- data/rabbitmq – RabbitMQ volume (custom.conf + data).

##### README for additional insights
- The user must have Python 3.8, Docker/Compose, and minimum hardware.
- Installation steps simply clone, run setup, then start.
- SSL certs are hot-pluggable via data/ssl_certs.
- RabbitMQ and Mongo ownership/permissions matter for container mounts.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Clients
    subgraph "External"
        Clients["External Clients\n(Browsers/API)"]:::external
    end

    %% Application Layer
    subgraph "Application Layer"
        CEApp["CE App Container\nHTTPS(80/443)\nEnv Vars"]:::app
        DBMigrate["Migration Scripts\n(database-migrate.py,\n migrate.sh, migrate_ce)"]:::script
        Replicas["HA Scripts\n(replica_set,\n restore_ha_backup)"]:::script
        Mongo["MongoDB Container"]:::db
        Rabbit["RabbitMQ Container"]:::broker
    end

    %% Mounted Volumes
    subgraph "Volumes"
        CA["CA Certificates\n(data/ca_certs)"]:::volume
        SSL["SSL Certificates\n(data/ssl_certs)"]:::volume
        Plugins["Custom Plugins\n(data/custom_plugins)"]:::volume
        DBInit["DB Init Scripts\n(data/database-init)"]:::volume
        MongoData["Mongo Data\n(data/mongo-data)"]:::volume
        RMQConfig["RabbitMQ Config\n(data/rabbitmq/custom.conf)"]:::volume
        RMQData["RabbitMQ Data\n(data/rabbitmq/data)"]:::volume
    end

    %% Orchestration
    Orchestrator["Docker-Compose /\nPodman-Compose"]:::orch

    %% Host Scripts
    subgraph "Host-level Management Scripts"
        Setup["setup"]:::script
        Start["start"]:::script
        Stop["stop"]:::script
        Diagnose["diagnose"]:::script
    end

    %% Relationships
    Clients -->|"HTTPS Request"| CEApp
    CEApp -->|"Reads/Writes"| Mongo
    CEApp -->|"Publishes/Subscribes"| Rabbit
    CEApp -->|"Runs"| DBMigrate
    Mongo -->|"Runs"| Replicas

    %% Volume Mounts to CE App
    CA --> CEApp
    SSL --> CEApp
    Plugins --> CEApp

    %% Volume Mounts to MongoDB
    DBInit --> Mongo
    MongoData --> Mongo

    %% Volume Mounts to RabbitMQ
    RMQConfig --> Rabbit
    RMQData --> Rabbit

    %% Orchestrator manages containers
    Orchestrator --> CEApp
    Orchestrator --> Mongo
    Orchestrator --> Rabbit

    %% Host Scripts invoke orchestration
    Setup --> Orchestrator
    Start --> Orchestrator
    Stop --> Orchestrator
    Diagnose --> Orchestrator

    %% Click Events
    click Orchestrator "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/docker-compose.yml"
    click Orchestrator "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/podman-compose.yml"
    click DBMigrate "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/database-migrate.py"
    click DBMigrate "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/migrate.sh"
    click DBMigrate "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/migrate_ce"
    click DBInit "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/database-init"
    click DBInit "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/data/database-init/load_env.sh"
    click DBInit "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/data/database-init/mongo-init.sh"
    click MongoData "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/mongo-data"
    click Replicas "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/replica_set"
    click Replicas "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/restore_ha_backup"
    click RMQConfig "https://github.com/netskopeoss/ta_cloud_exchange/blob/main/data/rabbitmq/custom.conf"
    click RMQData "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/rabbitmq/data"
    click CA "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/ca_certs"
    click SSL "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/ssl_certs"
    click Plugins "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/data/custom_plugins"
    click Setup "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/setup"
    click Start "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/start"
    click Stop "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/stop"
    click Diagnose "https://github.com/netskopeoss/ta_cloud_exchange/tree/main/diagnose"

    %% Styles
    classDef external fill:#f0f,stroke:#333,stroke-width:1px;
    classDef app fill:#bbdefb,stroke:#333,stroke-width:1px;
    classDef db fill:#fff59d,stroke:#333,stroke-width:1px;
    classDef broker fill:#ffcc80,stroke:#333,stroke-width:1px;
    classDef volume fill:#c8e6c9,stroke:#333,stroke-width:1px;
    classDef orch fill:#e0e0e0,stroke:#333,stroke-width:1px;
    classDef script fill:#ffccbc,stroke:#333,stroke-width:1px;
```
