# vaibhav300503/agent architecture

> Architecture diagram of the GitHub repository vaibhav300503/agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vaibhav300503/agent
- Repository: https://github.com/vaibhav300503/agent

## Overview

1. This repository is a two-sided SOC pipeline: an endpoint agent (`soc-agent/src`) and a central server (`soc-agent/server`). The agent runs close to the source systems, while the server handles ingest, normalization, detection, and alert forwarding.

2. The agent’s core runtime is the `agent.py` entrypoint, which wires together platform-specific collectors, enrichment helpers, buffering/transport, and the SOAR/detection engine. It is packaged for Windows via PyInstaller and also has Linux install scripts and a Windows service wrapper.

3. Collection is split by data source and platform. `collectors/windows.py`, `collectors/linux.py`, `collectors/network.py`, and `collectors/fim.py` indicate separate subsystems for Windows event/security data, Linux logs, network telemetry, and file integrity monitoring. `collectors/base.py` likely defines common collector behavior and event normalization hooks.

4. Before data leaves the endpoint, it is enriched and sanitized. `geo_enricher.py`, `ip_resolver.py`, and `sanitizer.py` add context such as IP metadata and remove unsafe content. `config.py`, `utils.py`, and `agent_config.yaml` provide runtime settings like server URL, enabled collectors, and host identity.

5. Transport is resilient rather than fire-and-forget. The README and file layout show a buffered delivery model with local SQLite storage (`agent_buffer.db`) and `transport.py` for retrying sends over HTTPS to the SOC server. This is the main reliability boundary between endpoint collection and central processing.

6. The server side is organized as an ingest-and-process pipeline. `server/api/ingest.py` receives agent submissions, `server/main.py` starts the application, and `server/migrate_db.py` plus `seed_owasp_rules.py` manage persistence/bootstrap. The README indicates MongoDB as the main datastore and Redis as supporting infrastructure for asynchronous work.

7. Post-ingest processing is decomposed into workers rather than a single monolith. `server/workers/parser_worker.py`, `enricher_worker.py`, `detector_worker.py`, and `thehive_worker.py` suggest a staged flow: parse raw events, enrich them further, run detection rules, then open cases or alerts in TheHive for high-priority findings.

8. Detection is rule-driven and prioritized. The README describes OWASP, Windows, Linux, network, HTTP/web, authentication, and baseline rules with severity-based ordering and deduplication. That means the server’s detection layer is a rule engine over normalized log events, not a model-based analytics service.

9. The repo also contains deployment and operational boundaries. `install/` has OS-specific install scripts and a `soc-agent.service` unit, while `docs/` captures schema, rule sets, installation, and validation guidance. This makes the project easy to diagram as an agent package, a server service, and a documentation/ops layer.

10. Validation is treated as a first-class subsystem. `validate_agent.py`, `validate_enrichment.py`, `cicd_validate.py`, `tests/`, and `server`-side scripts provide automated checks for enrichment, detection behavior, and integration. The presence of `mock_server.py` and event-generation scripts shows the repo supports end-to-end testing without a live SOC backend.

11. Important external integrations shape the architecture: HTTPS API communication from agent to server, MongoDB for server-side storage, Redis for background processing, and TheHive for incident creation. The README also references a dashboard consumer, so the server is not only an alerting backend but also the data source for visualization.

12. For an architecture diagram, the clearest sequence is: endpoint sources → platform collectors → enrichment/sanitization → local buffer/transport → server ingest API → parser/enricher/detector workers → database/alerts → TheHive and dashboard consumers.

Main entry points: [`soc-agent/src/agent.py`](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/agent.py) (Agent runtime), [`soc-agent/src/geo_enricher.py`](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/geo_enricher.py) (Agent enrichers), [`soc-agent/server/workers/detector_worker.py`](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/detector_worker.py) (Detector worker), [`soc-agent/server/api/ingest.py`](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/api/ingest.py) (Ingest API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Endpoint agent"]
  node_agent_entry["Agent runtime<br/>entrypoint<br/>[agent.py]"]
  node_collectors_base["Collector base<br/>collector-core<br/>[base.py]"]
  node_collectors_windows["Windows collector<br/>[windows.py]"]
  node_collectors_linux["Linux collector<br/>[linux.py]"]
  node_collectors_network["Network collector<br/>[network.py]"]
  node_collectors_fim["FIM collector<br/>[fim.py]"]
  node_agent_enrichment["Agent enrichers<br/>enrichment<br/>[geo_enricher.py]"]
  node_agent_config["Agent config<br/>settings<br/>[config.py]"]
  node_agent_transport["Buffered transport<br/>[transport.py]"]
  node_agent_soar["SOAR engine<br/>policy-engine<br/>[soar_engine.py]"]
end

subgraph group_server["SOC server"]
  node_server_main["Server app<br/>service-entrypoint<br/>[main.py]"]
  node_server_ingest["Ingest API<br/>[ingest.py]"]
  node_server_parser["Parser worker<br/>[parser_worker.py]"]
  node_server_enricher["Server enricher<br/>worker<br/>[enricher_worker.py]"]
  node_server_detector["Detector worker<br/>[detector_worker.py]"]
  node_server_thehive["TheHive worker<br/>[thehive_worker.py]"]
  node_server_db[("MongoDB<br/>datastore")]
  node_server_redis["Redis queue<br/>broker"]
  node_server_rules["Rule seed<br/>bootstrap"]
  node_dashboard["Dashboard<br/>consumer"]
  node_thehive["TheHive<br/>external-case-system"]
end

subgraph group_ops["Ops & validation"]
  node_deploy_docs["Deploy & docs<br/>ops<br/>[README.md]"]
  node_validation["Validation<br/>quality<br/>[validate_agent.py]"]
end

node_agent_entry -->|"uses"| node_collectors_base
node_agent_entry -->|"starts"| node_collectors_windows
node_agent_entry -->|"starts"| node_collectors_linux
node_agent_entry -->|"starts"| node_collectors_network
node_agent_entry -->|"starts"| node_collectors_fim
node_collectors_windows -->|"feeds"| node_agent_enrichment
node_collectors_linux -->|"feeds"| node_agent_enrichment
node_collectors_network -->|"feeds"| node_agent_enrichment
node_collectors_fim -->|"feeds"| node_agent_enrichment
node_agent_enrichment -->|"buffers"| node_agent_transport
node_agent_config -->|"configures"| node_agent_entry
node_agent_soar -->|"integrates"| node_agent_entry
node_agent_transport -->|"sends"| node_server_ingest
node_server_main -->|"hosts"| node_server_ingest
node_server_ingest -->|"queues"| node_server_parser
node_server_parser -->|"normalizes"| node_server_enricher
node_server_enricher -->|"prepares"| node_server_detector
node_server_detector -->|"stores"| node_server_db
node_server_detector -->|"escalates"| node_server_thehive
node_server_thehive -->|"opens cases"| node_thehive
node_server_db -->|"serves"| node_dashboard
node_server_rules -->|"seeds rules"| node_server_detector
node_validation -->|"verifies"| node_agent_entry
node_validation -->|"tests"| node_server_ingest
node_deploy_docs -->|"packages"| node_agent_entry
node_deploy_docs -->|"deploys"| node_server_main

click node_agent_entry "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/agent.py"
click node_collectors_base "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/base.py"
click node_collectors_windows "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/windows.py"
click node_collectors_linux "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/linux.py"
click node_collectors_network "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/network.py"
click node_collectors_fim "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/fim.py"
click node_agent_enrichment "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/geo_enricher.py"
click node_agent_config "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/config.py"
click node_agent_transport "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/transport.py"
click node_agent_soar "https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/soar_engine.py"
click node_server_main "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/main.py"
click node_server_ingest "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/api/ingest.py"
click node_server_parser "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/parser_worker.py"
click node_server_enricher "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/enricher_worker.py"
click node_server_detector "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/detector_worker.py"
click node_server_thehive "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/thehive_worker.py"
click node_server_rules "https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/seed_owasp_rules.py"
click node_deploy_docs "https://github.com/vaibhav300503/agent/blob/main/soc-agent/docs/README.md"
click node_validation "https://github.com/vaibhav300503/agent/blob/main/soc-agent/validate_agent.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent_entry,node_collectors_base,node_collectors_windows,node_collectors_linux,node_collectors_network,node_collectors_fim,node_agent_enrichment,node_agent_config,node_agent_transport,node_agent_soar toneBlue
class node_server_main,node_server_ingest,node_server_parser,node_server_enricher,node_server_detector,node_server_thehive,node_server_db,node_server_redis,node_server_rules,node_dashboard,node_thehive toneAmber
class node_deploy_docs,node_validation toneMint
```

## Components

### Endpoint agent

Endpoint-side collection, enrichment, buffering, and transport.

- [Agent runtime](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/agent.py): entrypoint · Main endpoint process wiring collectors, enrichment, SOAR logic, and transport. · `soc-agent/src/agent.py`
- [Collector base](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/base.py): collector-core · Shared collector behavior and event normalization hooks. · `soc-agent/src/collectors/base.py`
- [Windows collector](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/windows.py): collector · Windows event and security data collection. · `soc-agent/src/collectors/windows.py`
- [Linux collector](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/linux.py): collector · Linux log and host telemetry collection. · `soc-agent/src/collectors/linux.py`
- [Network collector](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/network.py): collector · Network telemetry collection. · `soc-agent/src/collectors/network.py`
- [FIM collector](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/collectors/fim.py): collector · File integrity monitoring collection. · `soc-agent/src/collectors/fim.py`
- [Agent enrichers](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/geo_enricher.py): enrichment · Geolocation, IP resolution, and payload sanitization before send. · `soc-agent/src/geo_enricher.py`
- [Agent config](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/config.py): settings · Runtime configuration for collectors, host identity, and server endpoint. · `soc-agent/src/config.py`
- [Buffered transport](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/transport.py): transport · Local buffering and retrying HTTPS delivery to the server. · `soc-agent/src/transport.py`
- [SOAR engine](https://github.com/vaibhav300503/agent/blob/main/soc-agent/src/soar_engine.py): policy-engine · Endpoint-side detection or response logic integrated into the agent runtime. · `soc-agent/src/soar_engine.py`

### SOC server

Central ingest, processing, detection, and alert forwarding.

- [Server app](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/main.py): service-entrypoint · Starts the central SOC server application. · `soc-agent/server/main.py`
- [Ingest API](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/api/ingest.py): api · Receives submissions from agents. · `soc-agent/server/api/ingest.py`
- [Parser worker](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/parser_worker.py): worker · Converts raw submissions into normalized records. · `soc-agent/server/workers/parser_worker.py`
- [Server enricher](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/enricher_worker.py): worker · Adds server-side enrichment to parsed events. · `soc-agent/server/workers/enricher_worker.py`
- [Detector worker](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/detector_worker.py): worker · Runs rule-based detection and prioritization. · `soc-agent/server/workers/detector_worker.py`
- [TheHive worker](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/workers/thehive_worker.py): worker · Forwards high-priority findings into TheHive. · `soc-agent/server/workers/thehive_worker.py`
- **MongoDB**: datastore · Primary server-side persistence for events and detection state.
- **Redis queue**: broker · Background-work infrastructure for asynchronous processing.
- [Rule seed](https://github.com/vaibhav300503/agent/blob/main/soc-agent/server/seed_owasp_rules.py): bootstrap · Seeds OWASP and other detection rules into the server. · `soc-agent/server/seed_owasp_rules.py`
- **Dashboard**: consumer · Downstream visualization or reporting consumer of server data.
- **TheHive**: external-case-system · External incident/case management target for alerts.

### Ops & validation

Deployment, docs, packaging, and validation support.

- [Deploy & docs](https://github.com/vaibhav300503/agent/blob/main/soc-agent/docs/README.md): ops · Install scripts, packaging, and architecture documentation. · `soc-agent/docs/README.md`
- [Validation](https://github.com/vaibhav300503/agent/blob/main/soc-agent/validate_agent.py): quality · Automated checks, mock server, and end-to-end test helpers. · `soc-agent/validate_agent.py`

## Connections

- Agent runtime → Collector base: uses · Agent runtime builds on shared collector behavior.
- Agent runtime → Windows collector: starts · Windows-specific collection path.
- Agent runtime → Linux collector: starts · Linux-specific collection path.
- Agent runtime → Network collector: starts · Network telemetry path.
- Agent runtime → FIM collector: starts · File integrity monitoring path.
- Windows collector → Agent enrichers: feeds · Collected events are enriched and sanitized before buffering.
- Linux collector → Agent enrichers: feeds · Collected events are enriched and sanitized before buffering.
- Network collector → Agent enrichers: feeds · Collected events are enriched and sanitized before buffering.
- FIM collector → Agent enrichers: feeds · Collected events are enriched and sanitized before buffering.
- Agent enrichers → Buffered transport: buffers · Enriched events are staged for resilient delivery.
- Agent config → Agent runtime: configures · Controls enabled collectors and server connection details.
- SOAR engine → Agent runtime: integrates · Endpoint policy and response engine runs within the agent.
- Buffered transport → Ingest API: sends · HTTPS delivery from endpoint to central ingest API.
- Server app → Ingest API: hosts · Application entrypoint exposes ingest service.
- Ingest API → Parser worker: queues · Raw submissions move into parsing stage.
- Parser worker → Server enricher: normalizes · Parsed records are enriched further.
- Server enricher → Detector worker: prepares · Enriched events feed detection rules.
- Detector worker → MongoDB: stores · Detection output and state are persisted.
- Detector worker → TheHive worker: escalates · High-priority findings become cases or alerts.
- TheHive worker → TheHive: opens cases · External case-management integration.
- MongoDB → Dashboard: serves · Dashboard consumes persisted SOC data.
- Rule seed → Detector worker: seeds rules · Bootstraps OWASP and detection rule sets.
- Validation → Agent runtime: verifies · Covers agent behavior and enrichment checks.
- Validation → Ingest API: tests · Supports mock-server and end-to-end validation.
- Deploy & docs → Agent runtime: packages · Build and installation paths for the endpoint agent.
- Deploy & docs → Server app: deploys · Server installation and operational docs.