# arnavmahajan630/veyra architecture

> Architecture diagram of the GitHub repository arnavmahajan630/veyra, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arnavmahajan630/veyra
- Repository: https://github.com/arnavmahajan630/veyra

## Overview

Veyra is an air-gapped log preprocessor: it receives logs, seals raw events, normalizes them into OCSF with lineage and confidence tiers, routes them to a SIEM, and supports contract learning, drift review, and evidence verification. The principal path is source → ingest → Kafka → normalization → delivery; raw archiving and lineage indexing branch from the raw stream. Control-plane and evidence relationships are documented in the README, but most service-to-service wiring was not sampled, so those connections cite the README rather than implying inspected implementation details.

Main entry points: [`services/control_api/src/control_api/app.py`](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/app.py) (Control API), [`services/ingest_gateway/src/ingest_gateway/app.py`](https://github.com/arnavmahajan630/veyra/blob/main/services/ingest_gateway/src/ingest_gateway/app.py) (HTTP ingest), [`packages/veyra_engine/src/veyra_engine/engine.py`](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_engine/src/veyra_engine/engine.py) (Normalization engine), [`console/src/App.tsx`](https://github.com/arnavmahajan630/veyra/blob/main/console/src/App.tsx) (Web console)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion and custody"]
  node_edge["Edge collectors<br/>[render.py]"]
  node_gateway["HTTP ingest<br/>[app.py]"]
  node_raw["Raw Kafka topics"]
  node_archiver["Raw archiver<br/>[archiver.py]"]
  node_common_envelope["Envelope models<br/>[envelope.py]"]
  node_common_core["Shared settings and IDs<br/>[settings.py]"]
  node_common_logging["Shared logging<br/>[logging.py]"]
  node_common_ids["ID utilities<br/>[ids.py]"]
end

subgraph group_processing["Normalize and deliver"]
  node_normalizer["Normalizer service<br/>[control.py]"]
  node_engine["Normalization engine<br/>[engine.py]"]
  node_norm["Normalized Kafka topics"]
  node_router["Delivery router<br/>[routes.py]"]
end

subgraph group_control["Control and learning"]
  node_control_api["Control API<br/>[app.py]"]
  node_control_db[("Control database<br/>[tables.py]")]
  node_contracts["Contracts and drafting<br/>[drafter.py]"]
  node_drift["Drift worker<br/>[worker.py]"]
  node_auth["Session and role auth<br/>[auth.py]"]
  node_context["Control context<br/>[context.py]"]
  node_contract_routes["Contract workflow routes"]
  node_source_routes["Source and key routes<br/>[routes_sources.py]"]
end

subgraph group_evidence["Evidence and lineage"]
  node_vault[("Sealed vault")]
  node_lineage["Lineage indexer<br/>[indexer.py]"]
  node_clickhouse[("Lineage store")]
  node_evidence_api["Evidence API<br/>[app.py]"]
  node_integrity["Integrity verifier<br/>[integrity.py]"]
  node_common_hash["Hashing utilities<br/>[hashing.py]"]
  node_common_models["Shared record models<br/>[__init__.py]"]
end

subgraph group_console["Operator console"]
  node_console["Web console<br/>[App.tsx]"]
  node_api_client["Console API client<br/>[client.ts]"]
  node_hotkeys["Hotkey registry<br/>[registry.ts]"]
  node_query_client["Query client<br/>[queryClient.ts]"]
end

node_devices(("Log sources"))
node_siem["Wazuh / SIEM"]
node_user(("Operators"))

node_devices -->|"send syslog"| node_edge
node_devices -->|"send HTTP"| node_gateway
node_edge -->|"publish raw"| node_raw
node_gateway -->|"publish raw"| node_raw
node_raw -->|"consume raw"| node_archiver
node_archiver -->|"seal raw"| node_vault
node_raw -->|"consume raw"| node_normalizer
node_normalizer -.->|"normalize"| node_engine
node_engine -.->|"emit OCSF"| node_norm
node_norm -->|"consume normalized"| node_router
node_router -->|"deliver events"| node_siem
node_norm -->|"index lineage"| node_lineage
node_lineage -->|"write records"| node_clickhouse
node_clickhouse -.->|"query lineage"| node_evidence_api
node_vault -->|"verify seals"| node_integrity
node_user -->|"operate"| node_console
node_console -.->|"request API"| node_api_client
node_api_client -->|"call REST"| node_control_api
node_control_api -.->|"read/write"| node_control_db
node_control_api -->|"manage contracts"| node_contracts
node_control_api -->|"review drift"| node_drift
node_control_api -->|"publish control"| node_gateway
node_control_api -->|"publish control"| node_normalizer
node_control_api -.->|"authenticate routes"| node_auth
node_control_api -.->|"provide shared context"| node_context
node_contract_routes -->|"use context"| node_context
node_source_routes -->|"use context"| node_context
node_console -->|"create query client"| node_query_client
node_console -->|"install listener"| node_hotkeys
node_engine -->|"uses hashing"| node_common_hash
node_engine -->|"uses IDs"| node_common_ids
node_engine -->|"uses record models"| node_common_models
node_gateway -->|"builds envelopes"| node_common_envelope
node_gateway -->|"uses hashing"| node_common_hash
node_gateway -.->|"uses settings"| node_common_core
node_control_api -->|"uses settings"| node_common_core
node_query_client -->|"handles API errors"| node_api_client
node_auth -->|"uses context"| node_context

click node_edge "https://github.com/arnavmahajan630/veyra/blob/main/edge/render.py"
click node_gateway "https://github.com/arnavmahajan630/veyra/blob/main/services/ingest_gateway/src/ingest_gateway/app.py"
click node_archiver "https://github.com/arnavmahajan630/veyra/blob/main/services/archiver/src/archiver/archiver.py"
click node_normalizer "https://github.com/arnavmahajan630/veyra/blob/main/services/normalizer/src/normalizer/control.py"
click node_engine "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_engine/src/veyra_engine/engine.py"
click node_router "https://github.com/arnavmahajan630/veyra/blob/main/services/router/src/router/routes.py"
click node_control_api "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/app.py"
click node_control_db "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/tables.py"
click node_contracts "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_contracts/src/veyra_contracts/drafting/drafter.py"
click node_drift "https://github.com/arnavmahajan630/veyra/blob/main/services/drift_worker/src/drift_worker/worker.py"
click node_lineage "https://github.com/arnavmahajan630/veyra/blob/main/services/lineage_indexer/src/lineage_indexer/indexer.py"
click node_evidence_api "https://github.com/arnavmahajan630/veyra/blob/main/services/evidence_api/src/evidence_api/app.py"
click node_integrity "https://github.com/arnavmahajan630/veyra/blob/main/services/integrity/src/integrity/integrity.py"
click node_console "https://github.com/arnavmahajan630/veyra/blob/main/console/src/App.tsx"
click node_api_client "https://github.com/arnavmahajan630/veyra/blob/main/console/src/api/client.ts"
click node_hotkeys "https://github.com/arnavmahajan630/veyra/blob/main/console/src/hotkeys/registry.ts"
click node_common_envelope "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/models/envelope.py"
click node_common_core "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/settings.py"
click node_common_hash "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/hashing.py"
click node_common_logging "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/logging.py"
click node_common_ids "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/ids.py"
click node_common_models "https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/models/__init__.py"
click node_auth "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/auth.py"
click node_context "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/context.py"
click node_contract_routes "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/routes_contracts.py"
click node_source_routes "https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/routes_sources.py"
click node_query_client "https://github.com/arnavmahajan630/veyra/blob/main/console/src/shell/queryClient.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_edge,node_gateway,node_raw,node_archiver,node_common_envelope,node_common_core,node_common_logging,node_common_ids toneBlue
class node_normalizer,node_engine,node_norm,node_router toneAmber
class node_control_api,node_control_db,node_contracts,node_drift,node_auth,node_context,node_contract_routes,node_source_routes toneMint
class node_vault,node_lineage,node_clickhouse,node_evidence_api,node_integrity,node_common_hash,node_common_models toneRose
class node_console,node_api_client,node_hotkeys,node_query_client,node_devices,node_siem,node_user toneIndigo
```

## Components

### Ingestion and custody

- [Edge collectors](https://github.com/arnavmahajan630/veyra/blob/main/edge/render.py): `edge/render.py`
- [HTTP ingest](https://github.com/arnavmahajan630/veyra/blob/main/services/ingest_gateway/src/ingest_gateway/app.py): `services/ingest_gateway/src/ingest_gateway/app.py`
- **Raw Kafka topics**
- [Raw archiver](https://github.com/arnavmahajan630/veyra/blob/main/services/archiver/src/archiver/archiver.py): `services/archiver/src/archiver/archiver.py`
- [Envelope models](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/models/envelope.py): `packages/veyra_common/src/veyra_common/models/envelope.py`
- [Shared settings and IDs](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/settings.py): `packages/veyra_common/src/veyra_common/settings.py`
- [Shared logging](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/logging.py): `packages/veyra_common/src/veyra_common/logging.py`
- [ID utilities](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/ids.py): `packages/veyra_common/src/veyra_common/ids.py`

### Normalize and deliver

- [Normalizer service](https://github.com/arnavmahajan630/veyra/blob/main/services/normalizer/src/normalizer/control.py): `services/normalizer/src/normalizer/control.py`
- [Normalization engine](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_engine/src/veyra_engine/engine.py): `packages/veyra_engine/src/veyra_engine/engine.py`
- **Normalized Kafka topics**
- [Delivery router](https://github.com/arnavmahajan630/veyra/blob/main/services/router/src/router/routes.py): `services/router/src/router/routes.py`

### Control and learning

- [Control API](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/app.py): `services/control_api/src/control_api/app.py`
- [Control database](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/tables.py): `services/control_api/src/control_api/tables.py`
- [Contracts and drafting](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_contracts/src/veyra_contracts/drafting/drafter.py): `packages/veyra_contracts/src/veyra_contracts/drafting/drafter.py`
- [Drift worker](https://github.com/arnavmahajan630/veyra/blob/main/services/drift_worker/src/drift_worker/worker.py): `services/drift_worker/src/drift_worker/worker.py`
- [Session and role auth](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/auth.py): `services/control_api/src/control_api/auth.py`
- [Control context](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/context.py): `services/control_api/src/control_api/context.py`
- [Contract workflow routes](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/routes_contracts.py): `services/control_api/src/control_api/routes_contracts.py`
- [Source and key routes](https://github.com/arnavmahajan630/veyra/blob/main/services/control_api/src/control_api/routes_sources.py): `services/control_api/src/control_api/routes_sources.py`

### Evidence and lineage

- **Sealed vault**
- [Lineage indexer](https://github.com/arnavmahajan630/veyra/blob/main/services/lineage_indexer/src/lineage_indexer/indexer.py): `services/lineage_indexer/src/lineage_indexer/indexer.py`
- **Lineage store**
- [Evidence API](https://github.com/arnavmahajan630/veyra/blob/main/services/evidence_api/src/evidence_api/app.py): `services/evidence_api/src/evidence_api/app.py`
- [Integrity verifier](https://github.com/arnavmahajan630/veyra/blob/main/services/integrity/src/integrity/integrity.py): `services/integrity/src/integrity/integrity.py`
- [Hashing utilities](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/hashing.py): `packages/veyra_common/src/veyra_common/hashing.py`
- [Shared record models](https://github.com/arnavmahajan630/veyra/blob/main/packages/veyra_common/src/veyra_common/models/__init__.py): `packages/veyra_common/src/veyra_common/models/__init__.py`

### Operator console

- [Web console](https://github.com/arnavmahajan630/veyra/blob/main/console/src/App.tsx): `console/src/App.tsx`
- [Console API client](https://github.com/arnavmahajan630/veyra/blob/main/console/src/api/client.ts): `console/src/api/client.ts`
- [Hotkey registry](https://github.com/arnavmahajan630/veyra/blob/main/console/src/hotkeys/registry.ts): `console/src/hotkeys/registry.ts`
- [Query client](https://github.com/arnavmahajan630/veyra/blob/main/console/src/shell/queryClient.ts): `console/src/shell/queryClient.ts`

### Other components

- **Log sources**
- **Wazuh / SIEM**
- **Operators**

## Connections

- Log sources → Edge collectors: send syslog (evidence: `README.md`)
- Log sources → HTTP ingest: send HTTP (evidence: `README.md`)
- Edge collectors → Raw Kafka topics: publish raw (evidence: `README.md`)
- HTTP ingest → Raw Kafka topics: publish raw (evidence: `README.md`)
- Raw Kafka topics → Raw archiver: consume raw (evidence: `README.md`)
- Raw archiver → Sealed vault: seal raw (evidence: `README.md`)
- Raw Kafka topics → Normalizer service: consume raw (evidence: `README.md`)
- Normalizer service → Normalization engine: normalize
- Normalization engine → Normalized Kafka topics: emit OCSF
- Normalized Kafka topics → Delivery router: consume normalized (evidence: `README.md`)
- Delivery router → Wazuh / SIEM: deliver events (evidence: `README.md`)
- Normalized Kafka topics → Lineage indexer: index lineage (evidence: `README.md`)
- Lineage indexer → Lineage store: write records (evidence: `README.md`)
- Lineage store → Evidence API: query lineage
- Sealed vault → Integrity verifier: verify seals (evidence: `README.md`)
- Operators → Web console: operate (evidence: `README.md`)
- Web console → Console API client: request API
- Console API client → Control API: call REST (evidence: `README.md`)
- Control API → Control database: read/write
- Control API → Contracts and drafting: manage contracts (evidence: `README.md`)
- Control API → Drift worker: review drift (evidence: `README.md`)
- Control API → HTTP ingest: publish control (evidence: `README.md`)
- Control API → Normalizer service: publish control (evidence: `README.md`)
- Control API → Session and role auth: authenticate routes
- Control API → Control context: provide shared context
- Contract workflow routes → Control context: use context (evidence: `services/control_api/src/control_api/routes_contracts.py`)
- Source and key routes → Control context: use context (evidence: `services/control_api/src/control_api/routes_sources.py`)
- Web console → Query client: create query client (evidence: `console/src/App.tsx`)
- Web console → Hotkey registry: install listener (evidence: `console/src/App.tsx`)
- Normalization engine → Hashing utilities: uses hashing (evidence: `packages/veyra_engine/src/veyra_engine/engine.py`)
- Normalization engine → ID utilities: uses IDs (evidence: `packages/veyra_engine/src/veyra_engine/engine.py`)
- Normalization engine → Shared record models: uses record models (evidence: `packages/veyra_engine/src/veyra_engine/engine.py`)
- HTTP ingest → Envelope models: builds envelopes (evidence: `services/ingest_gateway/src/ingest_gateway/app.py`)
- HTTP ingest → Hashing utilities: uses hashing (evidence: `services/ingest_gateway/src/ingest_gateway/app.py`)
- HTTP ingest → Shared settings and IDs: uses settings
- Control API → Shared settings and IDs: uses settings
- Query client → Console API client: handles API errors (evidence: `console/src/shell/queryClient.ts`)
- Session and role auth → Control context: uses context (evidence: `services/control_api/src/control_api/auth.py`)