# gristlabs/grist-core architecture

> Architecture diagram of the GitHub repository gristlabs/grist-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gristlabs/grist-core
- Repository: https://github.com/gristlabs/grist-core

## Overview

##### Main Components

a. Client Layer:
- UI Components (app/client/ui)
- Models (app/client/models)
- Components (app/client/components)
- Widgets (app/client/widgets)
- Client-side APIs and hooks

b. Server Layer:
- Split into three main services:
- Home Server (handles authentication, user management, and document listing)
- Doc Worker (handles document operations and Python formula execution)
- Static Server (serves static assets)

c. Storage Layer:
- Multiple storage systems:
- Document Storage (SQLite-based .grist files)
- Metadata Database (TypeORM with PostgreSQL/SQLite)
- Redis (for sessions and caching)
- File Storage (for attachments)

##### Key Interactions

a. Authentication Flow:
- Support for multiple auth methods:
- Forward Authentication
- SAML
- OAuth
- Default Authentication

b. Document Processing:
- Flow from client to server:
- Action Bundles
- Formula Execution
- Data Updates
- Real-time Sync

c. Plugin Architecture:
- Plugin system components:
- Custom Widgets
- Import Sources
- File Parsers
- Sandboxed Execution

##### Security Boundaries

a. Sandbox Implementations:
- Different sandbox options:
- gVisor
- macSandboxExec
- Pyodide (WASM)
- Docker

b. Access Control:
- ACL implementation:
- User Permissions
- Row-level Access
- Column-level Access

##### Integration Points

a. External Services:
- Connections to:
- Webhooks
- Google Drive
- Custom Widgets
- AI Assistant (OpenAI/Llama)

b. API Endpoints:
- Main API groups:
- REST API
- WebSocket Connections
- SCIM API

##### Specific Implementation Details

- Blue: Core application components
- Green: Security boundaries
- Yellow: Integration points
- Red: Critical paths
- Gray: Optional features

- Databases
- API endpoints
- User interfaces
- Security boundaries
- External services

Annotations for:
- Environment variable configuration points
- Scaling considerations
- Security checkpoints
- Data flow directions

##### Data storage and caching strategy

This will provide a clear understanding of Grist's architecture while maintaining the right level of detail for a principal software engineer to understand the system's design and implementation.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Main Client Layer
    subgraph Client ["Client Layer"]
        direction TB
        Browser["Web Browser"]
        subgraph ClientComponents ["Components"]
            UI["UI Layer"]:::client
            Models["Models"]:::client
            Components["Components"]:::client
            Widgets["Widgets"]:::client
            ClientAPI["Client APIs"]:::client
        end
        Browser --> UI
        UI --> Models
        UI --> Components
        UI --> Widgets
        UI --> ClientAPI
    end

    %% Server Layer with organized subgroups
    subgraph ServerLayer ["Server Layer"]
        direction TB
        subgraph CoreServers ["Core Servers"]
            HomeServer["Home Server"]:::server
            DocWorker["Doc Worker"]:::server
            StaticServer["Static Server"]:::server
        end
        
        subgraph ServerComponents ["Server Components"]
            AuthSystem["Authentication System"]:::security
            PluginSystem["Plugin System"]:::plugin
            RTSync["Real-time Sync"]:::server
            APILayer["API Layer"]:::api
            DocProcess["Document Processing"]:::server
            WSHandler["WebSocket Handler"]:::server
            ImportExport["Import/Export"]:::server
            FormulaEngine["Formula Engine"]:::server
        end
    end

    %% Storage Layer
    subgraph StorageLayer ["Storage Layer"]
        direction LR
        DocStorage[("Document Storage")]:::storage
        MetaDB[("Metadata DB")]:::storage
        Redis[("Redis Cache")]:::storage
        FileStore[("File Storage")]:::storage
    end

    %% Security Components
    subgraph SecurityLayer ["Security"]
        Sandbox["Sandbox"]:::security
        ACL["Access Control"]:::security
    end

    %% External Integrations
    subgraph ExternalLayer ["External Integration"]
        direction LR
        Webhooks["Webhooks"]:::external
        GDrive["Google Drive"]:::external
        CustomWidgets["Custom Widgets"]:::external
        AI["AI Assistant"]:::external
    end

    %% Core Connections
    ClientAPI --> APILayer
    ClientAPI --> WSHandler
    
    %% Server Interactions
    HomeServer --> AuthSystem
    HomeServer --> MetaDB
    DocWorker --> DocProcess
    DocWorker --> FormulaEngine
    DocProcess --> DocStorage
    RTSync --> WSHandler
    AuthSystem --> Redis
    
    %% Security and Plugin Connections
    DocWorker --> Sandbox
    PluginSystem --> CustomWidgets
    AuthSystem --> ACL
    
    %% Storage Connections
    DocProcess --> FileStore
    ImportExport --> DocStorage
    
    %% External Service Connections
    APILayer --> Webhooks
    APILayer --> GDrive
    APILayer --> AI

    %% Click Events
    click UI "https://github.com/gristlabs/grist-core/tree/main/app/client/ui/"
    click Models "https://github.com/gristlabs/grist-core/tree/main/app/client/models/"
    click Components "https://github.com/gristlabs/grist-core/tree/main/app/client/components/"
    click Widgets "https://github.com/gristlabs/grist-core/tree/main/app/client/widgets/"
    click HomeServer "https://github.com/gristlabs/grist-core/blob/main/app/gen-server/ApiServer.ts"
    click DocWorker "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/DocWorker.ts"
    click AuthSystem "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/Authorizer.ts"
    click PluginSystem "https://github.com/gristlabs/grist-core/tree/main/app/plugin/"
    click DocStorage "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/DocStorage.ts"
    click ACL "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/Authorizer.ts"
    click Sandbox "https://github.com/gristlabs/grist-core/tree/main/sandbox/"
    click RTSync "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/DocClients.ts"
    click APILayer "https://github.com/gristlabs/grist-core/blob/main/app/common/BaseAPI.ts"
    click DocProcess "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/ActiveDoc.ts"
    click MetaDB "https://github.com/gristlabs/grist-core/tree/main/app/gen-server/entity/"
    click FileStore "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/ExternalStorage.ts"
    click ClientAPI "https://github.com/gristlabs/grist-core/tree/main/app/client/lib/"
    click FormulaEngine "https://github.com/gristlabs/grist-core/blob/main/sandbox/grist/engine.py"
    click ImportExport "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/Import.ts"
    click WSHandler "https://github.com/gristlabs/grist-core/blob/main/app/server/lib/GristServerSocket.ts"

    %% Styling
    classDef client fill:#2374ab
    classDef server fill:#2a9d8f
    classDef security fill:#e63946
    classDef storage fill:#457b9d
    classDef external fill:#f4a261
    classDef plugin fill:#ffb703
    classDef api fill:#8338ec

    %% Legend
    subgraph Legend
        direction LR
        Client1["Client Component"]:::client
        Server1["Server Component"]:::server
        Security1["Security Component"]:::security
        Storage1["Storage Component"]:::storage
        External1["External Integration"]:::external
        Plugin1["Plugin Component"]:::plugin
        API1["API Component"]:::api
    end
```
