# yeti-platform/yeti architecture

> Architecture diagram of the GitHub repository yeti-platform/yeti, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yeti-platform/yeti
- Repository: https://github.com/yeti-platform/yeti

## Overview

Based on our analysis of the file tree and README, Yeti Platform is a CTI/DFIR intelligence platform whose primary purpose is to store and correlate threat intelligence data (observables, TTPs, etc.), provide powerful search and enrichment capabilities over that data, and deliver results both via a web API and a web UI. The project is clearly highly modular, supporting plugins, contributors, and external data source integrations.

##### Main Components
- API & Web Interface – Represented mainly by the “core/web” folder. This includes:
- RESTful endpoints under “apiv2” (modules such as auth, audit, entities, indicators, observables, etc.)
- The web frontend (core/web/frontend) that provides the UI.
- Core Business Logic – Found in the “core” directory:
- Business and orchestration modules (core/helpers.py, core/interfaces.py, core/logger.py)
- Database interactions (core/database_arango.py, migrations)
- Schema definitions and entities (core/schemas) for handling CTI objects (attack patterns, malware, intrusion sets, etc.)
- Event processing and messaging (core/events, with producer and consumer components)
- Task management and scheduling (core/taskmanager.py, core/taskscheduler.py)
- Storage & External Resources – Under “core/clients/file_storage” you see support for both local and S3 storage.
- Plugin Architecture – With directories like “plugins/analytics”, “plugins/events”, “plugins/feeds”, and “plugins/inline”. These extend core functionality, providing:
- Additional analytics integrations
- Data feeds from external threat intelligence sources
- Custom event processing logic
- Contrib Integrations – The “contrib” folder includes third-party integrations (such as IVRE API in analytics and multiple feeds integrations) that can be seen as external plug-ins or connectors.
- Configuration and Dependencies – The “core/config” and various configuration files (e.g., yeti.conf.sample, .pylintrc) support environmental configuration and operational parameters.
- DevOps & Deployment – Notably, the “extras/docker” directory and GitHub workflows indicate a containerized deployment and CI/CD setup.

##### Relationships and Data Flow
- The Web API layer (REST endpoints) interacts with the core business logic to process incoming queries. Commands and data flow from the API layer to core modules that perform validations, business rules checks, and orchestrate tasks.
- The Event Bus (via core/events producer/consumer) facilitates asynchronous notifications and triggers internal actions (such as audit log writing, task scheduling, etc.).
- The plugin system sits on top of or alongside core components, allowing public and private plugins to intercept, extend, or add new processing capabilities (for instance, analytics or feed ingestion). These might be connected via clearly defined interfaces.
- File storage clients (local storage or S3) are used by the system when handling file persistence for artifacts or exported data.
- External sources (contrib integrations and plugins for feeds) import CTI data which is then normalized, stored, and made available via the API.

##### Important Architectural Patterns & Design Principles
- Modular and Extensible Architecture – The clear separation between “core”, “plugins”, and “contrib” enables adding or modifying features without disrupting the core application.
- API-Driven Design – The “core/web/apiv2” indicates a RESTful design to allow integration with incident management systems, DFIR applications, and external enrichment tools.
- Event-Driven & Task Scheduling – With components for event production/consumption and task management, the design supports asynchronous processing and scheduling.
- Separation of Concerns – Different directories and layered components (data storage, business logic, API, plugins) ensure maintainability and robustness.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% API & Web Interface Layer
    subgraph "API & Web Interface"
        api1["REST Endpoints"]:::api
        api2["Web Frontend UI"]:::api
    end

    %% Core Business Logic Layer
    subgraph "Core Business Logic"
        core1["Business & Orchestration"]:::core
        core2["Schema Definitions (CTI Entities)"]:::core
        core3["Database Interaction (ArangoDB)"]:::db
        core4["Event Processing & Messaging"]:::core
    end

    %% Storage & External Resources Layer
    subgraph "Storage & External Resources"
        storage1["File Storage Clients"]:::storage
        storage2["Local Storage"]:::storage
        storage3["S3 Storage"]:::storage
    end

    %% Plugin Architecture Layer
    subgraph "Plugin Architecture"
        plugin1["Analytics Plugins"]:::plugin
        plugin2["Events Plugins"]:::plugin
        plugin3["Feeds Plugins"]:::plugin
        plugin4["Inline Plugins"]:::plugin
    end

    %% Contrib Integrations Layer
    subgraph "Contrib Integrations"
        contrib1["Third-Party Integrations"]:::contrib
    end

    %% Configuration & Dependencies Layer
    subgraph "Configuration & Dependencies"
        config1["Configuration & Dependencies"]:::config
    end

    %% DevOps & Deployment Layer
    subgraph "DevOps & Deployment"
        devops1["Docker Deployment"]:::devops
        devops2["CI/CD Workflows"]:::devops
    end

    %% Connections - API to Core
    api1 -->|"calls"| core1
    api2 -->|"renders"| core1

    %% Connections - Core internal flows
    core1 -->|"validates"| core2
    core1 -->|"persists"| core3
    core1 -->|"triggers"| core4
    core1 -->|"saves files"| storage1
    core4 -->|"notifies"| plugin2

    %% Connections - Database Interaction
    core3 ---|"data stored"| core2

    %% Connections - Storage details
    storage1 --> storage2
    storage1 --> storage3

    %% Connections - Plugin integrations
    core1 -->|"extends"| plugin1
    core1 -->|"extends"| plugin3
    core1 -->|"extends"| plugin4

    %% Connections - Contrib Ingestion
    contrib1 -->|"imports CTI data"| core1

    %% Connections - Configuration influence
    config1 -->|"configures"| core1

    %% Connections - DevOps impact
    devops1 -->|"deploys"| api1
    devops1 -->|"deploys"| core1
    devops2 -->|"integrates"| devops1

    %% Click Events
    click api1 "https://github.com/yeti-platform/yeti/tree/main/core/web/apiv2"
    click api2 "https://github.com/yeti-platform/yeti/tree/main/core/web/frontend"
    click core1 "https://github.com/yeti-platform/yeti/blob/main/core/helpers.py"
    click core3 "https://github.com/yeti-platform/yeti/blob/main/core/database_arango.py"
    click core2 "https://github.com/yeti-platform/yeti/tree/main/core/schemas"
    click core4 "https://github.com/yeti-platform/yeti/tree/main/core/events"
    click storage1 "https://github.com/yeti-platform/yeti/tree/main/core/clients/file_storage"
    click storage2 "https://github.com/yeti-platform/yeti/blob/main/core/clients/file_storage/classes/local_storage.py"
    click storage3 "https://github.com/yeti-platform/yeti/blob/main/core/clients/file_storage/classes/s3.py"
    click plugin1 "https://github.com/yeti-platform/yeti/tree/main/plugins/analytics"
    click plugin2 "https://github.com/yeti-platform/yeti/tree/main/plugins/events"
    click plugin3 "https://github.com/yeti-platform/yeti/tree/main/plugins/feeds"
    click plugin4 "https://github.com/yeti-platform/yeti/tree/main/plugins/inline"
    click contrib1 "https://github.com/yeti-platform/yeti/tree/main/contrib"
    click config1 "https://github.com/yeti-platform/yeti/tree/main/core/config"
    click devops1 "https://github.com/yeti-platform/yeti/tree/main/extras/docker"
    click devops2 "https://github.com/yeti-platform/yeti/tree/main/.github/workflows"

    %% Styles
    classDef api fill:#FFCCCC,stroke:#FF0000,stroke-width:2px;
    classDef core fill:#CCFFCC,stroke:#008000,stroke-width:2px;
    classDef db fill:#FFFFCC,stroke:#CCCC00,stroke-width:2px;
    classDef storage fill:#CCCCFF,stroke:#0000FF,stroke-width:2px;
    classDef plugin fill:#FFCCFF,stroke:#FF00FF,stroke-width:2px;
    classDef contrib fill:#CCFFFF,stroke:#00FFFF,stroke-width:2px;
    classDef config fill:#F0E68C,stroke:#BDB76B,stroke-width:2px;
    classDef devops fill:#D3D3D3,stroke:#A9A9A9,stroke-width:2px;
```
