# opentidehq/coretide architecture

> Architecture diagram of the GitHub repository opentidehq/coretide, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opentidehq/coretide
- Repository: https://github.com/opentidehq/coretide

## Overview

##### Main Components
- Configurations – These files (in the “Configurations” directory) define the setup for deployment, documentation, global settings, and system lookups. They serve as the foundational “source of truth” for system behavior.
- Engines – This is the heart of CoreTide. It is subdivided into several subdirectories:
– Deployment engines (e.g., carbon_black_cloud.py, sentinel.py, etc.) to integrate with various security systems.
– Documentation engines generating self-documenting artifacts.
– Extraction, indexing, mutation, and validation engines that process threat data, schema validation, and object generation.
– Modules that provide a shared library of helper functions and common routines.
- External – Contains external prototypes, threat intelligence feeds, and additional schema specs (e.g., ATC/react playbooks, incident actions). These files show how CoreTide connects with external attack frameworks and detection data sources.
- Framework – Provides meta schemas, vocabularies, and templates that standardize object definitions and ensure cross-system compatibility. This component defines how data structures and detection logic are described.
- Orchestration and Pipelines – Automated scripts and CI/CD pipelines (e.g., in “Orchestration” and “Pipelines” directories for both Azure and GitLab) that trigger deployments, documentation generation, validations, and other automated tasks.

##### Relationships and Data Flows
- The Configurations layer informs various Engines – the engines consume these YAML/TOML definitions to drive deployments, schema generation, validations, and documentation.
- The Framework (meta schemas and vocabularies) is consumed by both the Engines and the Orchestration pipelines to ensure consistency throughout the system. Indicate data flow from the Framework into Engines (e.g., validation, indexing modules) and into CI/CD pipelines that rely on templated configurations.
- Orchestration and Pipelines are responsible for automating processes such as deployment, documentation generation, mutation, and validation. They act like controllers, bridging development configurations with execution in production environments.

##### Architectural Patterns and Technologies
- Modular Structure: The project is clearly split into modular components (Engines, Framework, Orchestration, External), promoting separation of concerns.
- CI/CD and Automation: With extensive pipeline definitions, emphasize how the system supports continuous integration/delivery to decouple content from the code.
- Schema-Driven Design: With YAML-based meta schemas and self-documenting JSON schemas, the platform is built around a schema-first approach that drives validation, index generation, and documentation.
- Integration with External Systems: Components for deployment against various security systems illustrate a plugin-like design—a core engine that leverages multiple integration modules.

##### Final Steps
- Annotations (perhaps using callouts or legends) to explain key design principles such as the schema-first approach, plugin integrations for external systems, and the decoupling between code and configuration (content injection via the client OpenTide instance).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Configuration Layer
    Configurations["Configurations (System Setup and Global Settings)"]:::config
    Framework["Framework (Meta Schemas & Vocabularies)"]:::config

    %% Processing & Business Logic
    subgraph "Processing Engines"
        Deployment["Deployment Engines\n(Integrates with security systems)"]:::engine
        Documentation["Documentation Engines\n(Generates self-documenting artifacts)"]:::engine
        Extraction["Extraction Engines\n(Processes threat data)"]:::engine
        Indexing["Indexing Engines\n(Generates indexes and mappings)"]:::engine
        Mutation["Mutation Engines\n(Handles schema/object mutation)"]:::engine
        Validation["Validation Engines\n(Performs schema and data validations)"]:::engine
        Modules["Modules (Shared Library)\n(Common routines and helpers)"]:::engine
    end

    %% Automation & CI/CD Layer
    subgraph "Automation & CI/CD"
        Orchestration["Orchestration Components\n(Triggers deployments and tasks)"]:::orchestration
        CICDPipelines["CI/CD Pipelines\n(Automates testing and deployments)"]:::orchestration
    end

    %% External Integration
    External["External Interfaces\n(Attack frameworks, threat feeds, prototypes)"]:::external

    %% Relationships
    Configurations -->|"informs"| Deployment
    Configurations -->|"informs"| Documentation
    Configurations -->|"informs"| Extraction
    Configurations -->|"informs"| Indexing
    Configurations -->|"informs"| Mutation
    Configurations -->|"informs"| Validation
    Configurations -->|"informs"| Modules

    Framework -->|"provides schemas to"| Documentation
    Framework -->|"provides schemas to"| Extraction
    Framework -->|"provides schemas to"| Indexing
    Framework -->|"provides schemas to"| Mutation
    Framework -->|"provides schemas to"| Validation
    Framework -->|"provides templates to"| Modules
    Framework -->|"guides"| Orchestration

    Orchestration -->|"triggers"| CICDPipelines
    CICDPipelines -->|"automates"| Deployment
    CICDPipelines -->|"automates"| Documentation
    CICDPipelines -->|"automates"| Validation

    Deployment -->|"integrates with"| External

    %% Click Events
    click Configurations "https://github.com/opentidehq/coretide/tree/development/Configurations/"
    click Deployment "https://github.com/opentidehq/coretide/tree/development/Engines/deployment/"
    click Documentation "https://github.com/opentidehq/coretide/tree/development/Engines/documentation/"
    click Extraction "https://github.com/opentidehq/coretide/tree/development/Engines/extraction/"
    click Indexing "https://github.com/opentidehq/coretide/tree/development/Engines/indexing/"
    click Mutation "https://github.com/opentidehq/coretide/tree/development/Engines/mutation/"
    click Validation "https://github.com/opentidehq/coretide/tree/development/Engines/validation/"
    click Modules "https://github.com/opentidehq/coretide/tree/development/Engines/modules/"
    click Framework "https://github.com/opentidehq/coretide/tree/development/Framework/"
    click External "https://github.com/opentidehq/coretide/tree/development/External/"
    click Orchestration "https://github.com/opentidehq/coretide/tree/development/Orchestration/"
    click CICDPipelines "https://github.com/opentidehq/coretide/tree/development/Pipelines/"

    %% Styles
    classDef config fill:#ffe6e6,stroke:#cc0000,stroke-width:2px;
    classDef engine fill:#e6f7ff,stroke:#0066cc,stroke-width:2px;
    classDef orchestration fill:#e6ffe6,stroke:#009933,stroke-width:2px;
    classDef external fill:#fff0b3,stroke:#cc9900,stroke-width:2px;
```
