How opentidehq/coretide works
Main Components
• Configurations – These files (in the “Configurations” directory) define the setup for deployment, documentation, global settings, and system lookups. They serve as the foundational “source of truth” for system behavior.
• Engines – This is the heart of CoreTide. It is subdivided into several subdirectories:
– Deployment engines (e.g., carbon_black_cloud.py, sentinel.py, etc.) to integrate with various security systems.
– Documentation engines generating self-documenting artifacts.
– Extraction, indexing, mutation, and validation engines that process threat data, schema validation, and object generation.
– Modules that provide a shared library of helper functions and common routines.
• External – Contains external prototypes, threat intelligence feeds, and additional schema specs (e.g., ATC/react playbooks, incident actions). These files show how CoreTide connects with external attack frameworks and detection data sources.
• Framework – Provides meta schemas, vocabularies, and templates that standardize object definitions and ensure cross-system compatibility. This component defines how data structures and detection logic are described.
• Orchestration and Pipelines – Automated scripts and CI/CD pipelines (e.g., in “Orchestration” and “Pipelines” directories for both Azure and GitLab) that trigger deployments, documentation generation, validations, and other automated tasks.
Relationships and Data Flows
• The Configurations layer informs various Engines – the engines consume these YAML/TOML definitions to drive deployments, schema generation, validations, and documentation.
• The Framework (meta schemas and vocabularies) is consumed by both the Engines and the Orchestration pipelines to ensure consistency throughout the system. Indicate data flow from the Framework into Engines (e.g., validation, indexing modules) and into CI/CD pipelines that rely on templated configurations.
• Orchestration and Pipelines are responsible for automating processes such as deployment, documentation generation, mutation, and validation. They act like controllers, bridging development configurations with execution in production environments.
Architectural Patterns and Technologies
• Modular Structure: The project is clearly split into modular components (Engines, Framework, Orchestration, External), promoting separation of concerns.
• CI/CD and Automation: With extensive pipeline definitions, emphasize how the system supports continuous integration/delivery to decouple content from the code.
• Schema-Driven Design: With YAML-based meta schemas and self-documenting JSON schemas, the platform is built around a schema-first approach that drives validation, index generation, and documentation.
• Integration with External Systems: Components for deployment against various security systems illustrate a plugin-like design—a core engine that leverages multiple integration modules.
Final Steps
• Annotations (perhaps using callouts or legends) to explain key design principles such as the schema-first approach, plugin integrations for external systems, and the decoupling between code and configuration (content injection via the client OpenTide instance).