# v1d1an/s1em architecture

> Architecture diagram of the GitHub repository v1d1an/s1em, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/v1d1an/s1em
- Repository: https://github.com/v1d1an/s1em

## Overview

##### Project type and purpose
- This is a full-stack, containerized cybersecurity platform (S1EM) that bundles open-source SIEM, SIRP, threat-intel, full-packet capture, and SOAR tooling.
- It uses Docker Compose (single vs. multi-node) to deploy Elasticsearch, Beats, Logstash, Kibana, security agents (Suricata, Zeek, Auditbeat, Filebeat, Metricbeat, Heartbeat), triage/response tools (TheHive, Cortex, MISP, OpenCTI), workflow automation (n8n), packet capture (Arkime), EDR (Velociraptor, Zircolite), threat-intel (Spiderfoot), alerting (Elastalert), plus supporting services (Traefik, Redis, RabbitMQ, MySQL/Postgres, Syslog-ng).

##### File structure
- Top-level scripts (00_*.sh, 01_deploy.sh, etc.) orchestrate instance creation, deployment, and cleanup.
- Docker Compose manifests: docker-compose-single.yml and docker-compose-multi.yml define service sets for single-node vs. cluster.
- Directories grouped by function:
- beats/ (filebeat, metricbeat, auditbeat, heartbeat)
- logstash/ (pipelines, templates, configs)
- arkime/, suricata/, zeek/ for network capture/IDS
- misp/, cortex/, thehive/, n8n/ for threat-intel & SOAR
- redis/, postgres/, mysql/ for state & caching
- traefik/ for ingress routing
- kibana/ for dashboards & index patterns
- elastalert/, replay/, rules/, sigma/ for alerts and detection rules

##### README for additional insights
- Confirms the stack components (Elasticsearch cluster, Beats, Logstash, Kibana, N8n, Zircolite, Velociraptor, Spiderfoot, Syslog-ng, etc.)
- References an Architecture Guide in the wiki
- Mentions single vs. multi-node modes

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Collection & Agents
    subgraph "Data Collection & Agents"
        Suricata["Suricata"]:::collection
        Zeek["Zeek"]:::collection
        Auditbeat["Auditbeat"]:::collection
        Filebeat["Filebeat"]:::collection
        Metricbeat["Metricbeat"]:::collection
        Heartbeat["Heartbeat"]:::collection
        Velociraptor["Velociraptor"]:::collection
        Zircolite["Zircolite"]:::collection
        SyslogNG["Syslog-ng"]:::collection
    end

    %% Ingestion & Processing
    subgraph "Ingestion & Processing"
        Logstash["Logstash"]:::ingestion
        Fleet["Fleet Server"]:::ingestion
    end

    %% Storage & Indexing
    subgraph "Storage & Indexing"
        Elasticsearch["Elasticsearch Cluster"]:::storage
    end

    %% UI & Visualization
    subgraph "User Interfaces & Visualization"
        Kibana["Kibana"]:::ui
        ArkimeViewer["Arkime Viewer"]:::ui
        Homer["Homer"]:::ui
        CyberChef["CyberChef"]:::ui
        Codimd["Codimd"]:::ui
    end

    %% Alerting & Detection
    subgraph "Alerting & Detection"
        Elastalert["Elastalert"]:::alert
        Sigma["Sigma Rules Engine"]:::alert
    end

    %% Threat Intel & Case Management
    subgraph "Threat Intel & Case Management"
        Cortex["Cortex"]:::threat
        TheHive["TheHive"]:::threat
        MISP["MISP"]:::threat
        OpenCTI["OpenCTI"]:::threat
        Spiderfoot["Spiderfoot"]:::threat
    end

    %% Automation & Orchestration
    subgraph "Automation & Orchestration"
        N8n["n8n"]:::automation
        Replay["replay.sh"]:::automation
    end

    %% Infrastructure
    subgraph "Infrastructure"
        Traefik["Traefik"]:::infra
        Redis["Redis"]:::infra
        RabbitMQ["RabbitMQ"]:::infra
        MySQL["MySQL"]:::infra
        PostgreSQL["PostgreSQL"]:::infra
    end

    %% Connections
    Suricata -->|logs| Logstash
    Zeek -->|pcap logs| Logstash
    SyslogNG -->|syslog| Logstash
    Auditbeat -->|metrics| Elasticsearch
    Filebeat -->|logs| Elasticsearch
    Metricbeat -->|metrics| Elasticsearch
    Heartbeat -->|monitoring| Elasticsearch
    Velociraptor -->|EDR data| Elasticsearch
    Zircolite -->|EDR data| Elasticsearch

    Replay -->|replay data| Logstash

    Logstash -->|ingest pipeline| Elasticsearch
    Fleet -->|policy & ingest| Elasticsearch

    Elasticsearch -->|read| Kibana
    Elasticsearch -->|read| ArkimeViewer
    Elasticsearch -->|queries| Elastalert
    Elasticsearch -->|queries| Cortex
    Elasticsearch -->|queries| TheHive

    Elastalert -->|alerts| N8n
    N8n -->|create case| TheHive
    N8n -->|trigger alerts| Elastalert

    MISP -->|intel| Cortex
    MISP -->|intel| TheHive
    Spiderfoot -->|scan| MISP
    Spiderfoot -->|push intel| OpenCTI
    OpenCTI -->|sync intel| MISP

    Traefik -->|HTTP/HTTPS| Kibana
    Traefik -->|HTTP/HTTPS| ArkimeViewer
    Traefik -->|HTTP/HTTPS| Homer
    Traefik -->|HTTP/HTTPS| CyberChef
    Traefik -->|HTTP/HTTPS| Codimd
    Traefik -->|HTTP/HTTPS| TheHive
    Traefik -->|HTTP/HTTPS| MISP
    Traefik -->|HTTP/HTTPS| N8n

    Redis -->|cache| TheHive
    Redis -->|cache| Cortex
    RabbitMQ -->|jobs| TheHive
    RabbitMQ -->|jobs| Cortex

    MySQL -->|state| MISP
    MySQL -->|state| N8n
    PostgreSQL -->|state| N8n
    PostgreSQL -->|state| MISP

    %% Click Events
    click Suricata "https://github.com/v1d1an/s1em/blob/master/suricata/suricata.yaml"
    click Zeek "https://github.com/v1d1an/s1em/blob/master/zeek/config.zeek"
    click Auditbeat "https://github.com/v1d1an/s1em/tree/master/auditbeat/"
    click Filebeat "https://github.com/v1d1an/s1em/tree/master/filebeat/"
    click Metricbeat "https://github.com/v1d1an/s1em/tree/master/metricbeat/"
    click Heartbeat "https://github.com/v1d1an/s1em/tree/master/heartbeat/"
    click Logstash "https://github.com/v1d1an/s1em/tree/master/logstash/config/"
    click Logstash "https://github.com/v1d1an/s1em/tree/master/logstash/pipeline/"
    click Kibana "https://github.com/v1d1an/s1em/tree/master/kibana/"
    click ArkimeViewer "https://github.com/v1d1an/s1em/tree/master/arkime/"
    click Homer "https://github.com/v1d1an/s1em/tree/master/homer/"
    click Elastalert "https://github.com/v1d1an/s1em/blob/master/elastalert/elastalert.yaml"
    click Sigma "https://github.com/v1d1an/s1em/tree/master/sigma/"
    click Sigma "https://github.com/v1d1an/s1em/blob/master/sigma.yml"
    click Cortex "https://github.com/v1d1an/s1em/tree/master/cortex/"
    click TheHive "https://github.com/v1d1an/s1em/tree/master/thehive/"
    click MISP "https://github.com/v1d1an/s1em/blob/master/misp/config.php"
    click N8n "https://github.com/v1d1an/s1em/tree/master/n8n/"
    click Replay "https://github.com/v1d1an/s1em/blob/master/replay/replay.sh"
    click Traefik "https://github.com/v1d1an/s1em/tree/master/traefik/"
    click Redis "https://github.com/v1d1an/s1em/blob/master/redis/redis.conf"
    click MySQL "https://github.com/v1d1an/s1em/blob/master/mysql/databases.sql"
    click PostgreSQL "https://github.com/v1d1an/s1em/blob/master/postgres/databases.sh"

    %% Styles
    classDef collection fill:#E0F7FA,stroke:#0288D1,color:#000;
    classDef ingestion fill:#FFE0B2,stroke:#FB8C00,color:#000;
    classDef storage fill:#C8E6C9,stroke:#43A047,color:#000;
    classDef ui fill:#E1BEE7,stroke:#8E24AA,color:#000;
    classDef alert fill:#FFCDD2,stroke:#E53935,color:#000;
    classDef threat fill:#F8BBD0,stroke:#D81B60,color:#000;
    classDef automation fill:#FFF9C4,stroke:#FBC02D,color:#000;
    classDef infra fill:#CFD8DC,stroke:#546E7A,color:#000;
```
