# permitio/opal architecture

> Architecture diagram of the GitHub repository permitio/opal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/permitio/opal
- Repository: https://github.com/permitio/opal

## Overview

##### Main Components

a) Core Services:
- OPAL Server
- OPAL Client(s)
- Policy Engine instances (OPA/Cedar)
- Data Sources
- Policy Sources

b) Communication Layer:
- WebSocket PubSub Channel
- REST APIs
- Data Fetch Interfaces

##### Architectural Layers

**┌─ Data Sources Layer ─────────────────┐**
- APIs                               │
- Databases                          │
- Git Repositories                   │
- S3/Cloud Storage                   │
- Third-party Services               │
- ┘
- ↕ (Fetch Providers)
**┌─ OPAL Server Layer ──────────────────┐**
- Policy Watcher                     │
- Data Update Publisher              │
- Authentication/Security            │
- Scope Management                   │
- ┘
- ↕ (WebSocket/PubSub)
**┌─ OPAL Client Layer ──────────────────┐**
- Policy Updater                     │
- Data Updater                       │
- Policy Store Client                │
- Callbacks Handler                  │
- ┘
- ↕ (Direct Integration)
**┌─ Policy Engine Layer ────────────────┐**
- OPA Instances                      │
- Cedar Agent Instances              │
- ┘

##### Important Details

a) Server Components:
- Policy watcher mechanisms
- Data update publisher
- WebSocket broadcast interface
- Security layer (JWT, authentication)

b) Client Components:
- Policy store client implementations
- Data fetcher system
- Update handlers
- Health check system

c) Integration Points:
- External data source connections
- Policy engine interfaces
- Monitoring/metrics endpoints

##### Additional Elements

- Numbered sequence indicators for key flows
- Scalability aspects (multiple clients/engines)
- Security boundaries
- Monitoring/observability points

##### Technical Details to Highlight

- WebSocket-based real-time updates
- Stateless architecture
- Direct data fetching paths
- Policy/data synchronization mechanisms
- Authentication/authorization flows

- Real-time policy and data synchronization
- Distributed architecture
- Support for multiple policy engines
- Extensible data source integration
- Security and authentication mechanisms

Remember to maintain a balance between detail and clarity, focusing on the core architectural components while providing enough technical detail to understand the system's operation.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and classes
    classDef datasource fill:#90EE90,stroke:#333,stroke-width:2px
    classDef server fill:#ADD8E6,stroke:#333,stroke-width:2px
    classDef client fill:#87CEEB,stroke:#333,stroke-width:2px
    classDef engine fill:#FFA500,stroke:#333,stroke-width:2px
    classDef communication fill:#DDA0DD,stroke:#333,stroke-width:2px
    classDef security fill:#D3D3D3,stroke:#333,stroke-width:2px

    %% Data Sources Layer
    subgraph DataSourcesLayer["Data Sources Layer"]
        API[APIs]:::datasource
        DB[(Databases)]:::datasource
        Git[Git Repositories]:::datasource
        S3[S3/Cloud Storage]:::datasource
        ThirdParty[Third-party Services]:::datasource
        GitPolicy[Git Policy Source]:::datasource
        APIPolicy[API Policy Source]:::datasource
    end

    %% OPAL Server Layer
    subgraph ServerLayer["OPAL Server Layer"]
        OPALServer[OPAL Server]:::server
        PolicyWatcher[Policy Watcher]:::server
        DataPublisher[Data Update Publisher]:::server
        Auth[Authentication/Security]:::security
        ScopeMgmt[Scope Management]:::server
        WSPubSub[WebSocket PubSub]:::communication
        TopicsMgmt[Topics Management]:::server
    end

    %% OPAL Client Layer
    subgraph ClientLayer["OPAL Client Layer"]
        OPALClient[OPAL Client]:::client
        PolicyUpdater[Policy Updater]:::client
        DataUpdater[Data Updater]:::client
        PolicyStore[Policy Store Client]:::client
        DataFetcher[Data Fetch Providers]:::client
    end

    %% Policy Engine Layer
    subgraph EngineLayer["Policy Engine Layer"]
        OPA[OPA Instances]:::engine
        Cedar[Cedar Agent]:::engine
        EngineRunner[Engine Runner]:::engine
    end

    %% Connections
    DataSourcesLayer --> |Fetch Data| ServerLayer
    ServerLayer --> |Real-time Updates| ClientLayer
    ClientLayer --> |Integration| EngineLayer

    %% Detailed connections
    PolicyWatcher --> WSPubSub
    DataPublisher --> WSPubSub
    WSPubSub --> PolicyUpdater
    WSPubSub --> DataUpdater
    DataFetcher --> DataSourcesLayer
    PolicyStore --> OPA
    PolicyStore --> Cedar
    Auth --> WSPubSub
    ScopeMgmt --> WSPubSub

    %% Click events
    click OPALServer "https://github.com/permitio/opal/tree/master/packages/opal-server/opal_server/"
    click OPALClient "https://github.com/permitio/opal/tree/master/packages/opal-client/opal_client/"
    click PolicyStore "https://github.com/permitio/opal/tree/master/packages/opal-client/opal_client/policy_store/"
    click DataPublisher "https://github.com/permitio/opal/blob/master/packages/opal-server/opal_server/data/data_update_publisher.py"
    click PolicyWatcher "https://github.com/permitio/opal/tree/master/packages/opal-server/opal_server/policy/watcher/"
    click WSPubSub "https://github.com/permitio/opal/blob/master/packages/opal-server/opal_server/pubsub.py"
    click DataFetcher "https://github.com/permitio/opal/tree/master/packages/opal-common/opal_common/fetcher/providers/"
    click DataUpdater "https://github.com/permitio/opal/blob/master/packages/opal-client/opal_client/data/updater.py"
    click EngineRunner "https://github.com/permitio/opal/blob/master/packages/opal-client/opal_client/engine/runner.py"
    click Auth "https://github.com/permitio/opal/tree/master/packages/opal-common/opal_common/authentication/"
    click TopicsMgmt "https://github.com/permitio/opal/tree/master/packages/opal-common/opal_common/topics/"
    click GitPolicy "https://github.com/permitio/opal/blob/master/packages/opal-common/opal_common/sources/git_policy_source.py"
    click APIPolicy "https://github.com/permitio/opal/blob/master/packages/opal-common/opal_common/sources/api_policy_source.py"
    click ScopeMgmt "https://github.com/permitio/opal/tree/master/packages/opal-server/opal_server/scopes/"
```
