# jitsi/skynet architecture

> Architecture diagram of the GitHub repository jitsi/skynet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jitsi/skynet
- Repository: https://github.com/jitsi/skynet

## Overview

##### Main Components
- Core API Server: Represent the main application entry point (skynet/main.py) that orchestrates incoming API requests.
- STT (Speech-to-text) modules like streaming_whisper and vox.
- TTT modules (for assistant, openai_api, persistence, rag, summaries, etc.).
- Authentication services (skynet/auth – bearer, jwt, etc.).
- File watchers, monitoring tools, logging, and metrics.
- External Dependencies:
- Redis – used for caching or pub/sub as noted in the README.
- External Model Services – vLLM, Ollama, and possibly local ffmpeg for audio processing.
- Docker Infrastructure – the Dockerfile, compose files (compose-dev.yaml) and docker scripts indicate containerized deployment.
- Demos/Clients: The JavaScript client demo and the streaming-whisper demo (located in the demos/ folder).
- Supporting Scripts & Tools: Utility scripts like run.sh, Makefile, and CI/CD workflows that impact deployment and maintenance.

##### Relationships and Interactions
- Similarly, indicate interactions between the assistant modules and external AI model services (vLLM/Ollama) for summarization and decision making.
- How authentication components (JWT, bearer, customer ID) integrate by verifying and securing API requests.
- External services like Redis with the main server, indicating caching or state sharing.
- Docker and CI/CD pipelines (placed under .github/workflows and docker directories) as deployment/infrastructure layers which package the entire application.

##### Architectural Patterns and Technologies
- Modular/Plugin Architecture: Highlight that the system is built in a modular fashion so that specific functionalities (e.g., STT streaming, summarization) can be enabled/disabled as needed.
- Monolithic API Entry Point: Even though it’s modularized internally, the main server is a single API endpoint.
- Containerization: Indicate that Docker is used for deployment and development flows.
- Asynchronous Communication: Some modules (like streaming whisper) are built to handle websockets and asynchronous requests.
- Technologies & Frameworks: Note the use of Python (source files), Poetry for dependency management, Redis for caching, potentially GPU-enabled inference stacks (vLLM, Ollama), and ffmpeg for audio processing.

##### Specific Elements
- Use clear labels for each component. For example, “Core API Server”, “STT: Streaming Whisper”, “TTT: Assistant & Summaries”, “Authentication Service”, “Redis”, “Docker Infrastructure”, “Client Demo (JS)” etc.
- Rectangles for internal modules,

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Clients"
        client1["Client Demo (JS)"]:::client
        client2["Streaming Whisper Demo"]:::client
    end

    subgraph "Core Service"
        api["Core API Server"]:::main
    end

    subgraph "Modular Architecture"
        subgraph "STT Modules"
            streaming["STT: Streaming Whisper"]:::module
            vox["STT: Vox"]:::module
        end
        subgraph "TTT Modules"
            assistant["TTT: Assistant"]:::module
            summaries["TTT: Summaries & Action Items"]:::module
            rag["TTT: RAG/Assistant Features"]:::module
        end
    end

    subgraph "Authentication Services"
        auth_jwt["Auth: JWT"]:::auth
        auth_bearer["Auth: Bearer"]:::auth
        auth_cust["Auth: Customer ID"]:::auth
        auth_user["Auth: User Info"]:::auth
    end

    subgraph "Monitoring & Tools"
        fileWatcher["File Watcher"]:::monitoring
        monitoringTools["Monitoring Tools"]:::monitoring
        metrics["Metrics Exposure"]:::monitoring
    end

    subgraph "External Services"
        redis["Redis"]:::external
        model["Model Services (vLLM/Ollama)"]:::external
        ffmpeg["Local ffmpeg"]:::external
    end

    subgraph "Infrastructure & Deployment"
        docker["Docker Infrastructure"]:::infra
        subgraph "Deployment Tools"
            runScript["Run Script"]:::infra
            makefile["Makefile"]:::infra
            ci_cd["CI/CD Workflows"]:::infra
        end
    end

    %% Connections from Clients to API Server
    client1 -->|"HTTP_Request"| api
    client2 -->|"HTTP_Request"| api

    %% API Server calls Modular Components
    api -->|"API_Call"| streaming
    api -->|"API_Call"| vox
    api -->|"API_Call"| assistant
    api -->|"API_Call"| summaries
    api -->|"API_Call"| rag

    %% API Server uses Authentication Services
    api -->|"VerifyToken"| auth_jwt
    api -->|"VerifyToken"| auth_bearer
    api -->|"VerifyToken"| auth_cust
    api -->|"VerifyToken"| auth_user

    %% API Server connects to Monitoring & Tools
    api -->|"Monitor"| fileWatcher
    api -->|"Monitor"| monitoringTools
    api -->|"ExposeMetrics"| metrics

    %% API Server uses External Services
    api -->|"Cache/Share"| redis

    %% TTT Modules interact with External Model Services
    assistant -->|"Inference"| model
    summaries -->|"Inference"| model
    rag -->|"Inference"| model
    summaries -->|"ProcessAudio"| ffmpeg

    %% Deployment and Infrastructure interactions
    docker -->|"Containerizes"| api
    runScript -->|"Deploy"| api
    makefile -->|"Build"| api
    ci_cd -->|"CI/CD"| api

    %% Click Events
    click api "https://github.com/jitsi/skynet/blob/master/skynet/main.py"
    click streaming "https://github.com/jitsi/skynet/tree/master/skynet/modules/stt/streaming_whisper"
    click vox "https://github.com/jitsi/skynet/tree/master/skynet/modules/stt/vox"
    click assistant "https://github.com/jitsi/skynet/tree/master/skynet/modules/ttt/assistant"
    click summaries "https://github.com/jitsi/skynet/tree/master/skynet/modules/ttt/summaries"
    click rag "https://github.com/jitsi/skynet/tree/master/skynet/modules/ttt/rag"
    click auth_jwt "https://github.com/jitsi/skynet/blob/master/skynet/auth/jwt.py"
    click auth_bearer "https://github.com/jitsi/skynet/blob/master/skynet/auth/bearer.py"
    click auth_cust "https://github.com/jitsi/skynet/blob/master/skynet/auth/customer_id.py"
    click auth_user "https://github.com/jitsi/skynet/blob/master/skynet/auth/user_info.py"
    click fileWatcher "https://github.com/jitsi/skynet/blob/master/skynet/modules/file_watcher.py"
    click monitoringTools "https://github.com/jitsi/skynet/blob/master/skynet/modules/monitoring.py"
    click metrics "https://github.com/jitsi/skynet/blob/master/skynet/metrics.py"
    click docker "https://github.com/jitsi/skynet/tree/master/docker"
    click runScript "https://github.com/jitsi/skynet/blob/master/run.sh"
    click makefile "https://github.com/jitsi/skynet/tree/master/Makefile"
    click ci_cd "https://github.com/jitsi/skynet/tree/master/.github/workflows"
    click client1 "https://github.com/jitsi/skynet/tree/master/demos/js-client"
    click client2 "https://github.com/jitsi/skynet/tree/master/demos/streaming-whisper"

    %% Styles
    classDef client fill:#FFCC00,stroke:#000,stroke-width:1px
    classDef main fill:#b3e6ff,stroke:#000,stroke-width:1px
    classDef module fill:#AAD751,stroke:#000,stroke-width:1px
    classDef auth fill:#ffb3b3,stroke:#000,stroke-width:1px
    classDef monitoring fill:#d0e1f9,stroke:#000,stroke-width:1px
    classDef external fill:#eab0f9,stroke:#000,stroke-width:1px
    classDef infra fill:#fcbad3,stroke:#000,stroke-width:1px
```
