# fortheswarm/moloch architecture

> Architecture diagram of the GitHub repository fortheswarm/moloch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fortheswarm/moloch
- Repository: https://github.com/fortheswarm/moloch

## Overview

##### Main Components
- Component:
– Located in the “capture” directory, this C-based, single-threaded executable is responsible for capturing network traffic from one or more interfaces. It invokes several parsers (for protocols such as DNS, HTTP, IRC, MySQL, etc.) and uses libraries like libpcap, libnids, yara, and others.
- Viewer Component:
– Found in the “viewer” directory, this Node.js application handles the web interface, session browsing, API access, and file transfers. It relies on packages (see package.json) and numerous JavaScript, CSS, and template (jade) files.
- Database / Elasticsearch:
– Not part of the repository per se but described in the “db” directory and README. This is an external component where session data and indexes are stored, backed by Elasticsearch. The “db” directory contains configuration scripts and sample elasticsearch.yml.
- Supporting Scripts & Tools:
– There are additional support scripts (e.g., “easybutton-build.sh”, “single-host”, and “tests”) that simplify building, configuration, and testing of the system.

##### Relations and Interactions
- Data Flow:
– Network interfaces send packets to the capture processes.
– The capture component processes the packets, extracts session data, writes PCAP files, and pushes structured JSON data (with session metadata) to Elasticsearch.
– The viewer component communicates with Elasticsearch (and may also work with the capture system) to retrieve and display the indexed session information through its web interface.
- Communication:
– The viewer also exposes RESTful APIs for remote access to the session and PCAP data.
– There is inter-process communication between capture and viewer (for example, a shared configuration file or secure tokens for communication, as the README explains regarding encrypted passwords for inter-Moloch communications).
– Elasticsearch uses mesh connections (ports 9300-930x) for clustering and plain TCP/HTTP communication (ports 9200-920x) for receiving data and queries.

##### Key Architectural Patterns and Principles
- Distributed and Scalable Architecture:
– Multiple capture processes can run independently on various machines (or multiple processes per interface) aiming toward high throughput.
– Elasticsearch is used as a horizontally scalable search/indexing engine across multiple nodes.
- Separation of Concerns:
– Capture: Dedicated to packet capture and session extraction.
– Viewer: Dedicated to presenting data via a web interface and handling interactions with users.
– Search/Index: Handled by Elasticsearch, decoupling data storage and search functionality from capture processes.
- Modularity and Extensibility:
– The parsers and plugins in the “capture” section suggest a modular design where new protocol support can be added by introducing additional files.

##### Technologies/Frameworks
- C language with supporting libraries (libpcap, libnids, yara, glib) for the capture module.
- Node.js for the viewer with JavaScript libraries and jade templates to render views.
- Elasticsearch for high-performance indexing and search.
- Additional support for testing (pcap files and test scripts) and build configuration (Makefiles, easybutton-build.sh, etc.).

## Diagram (Mermaid)

```mermaid
flowchart TD
    NI["Network Interfaces"]:::external
    CP["Capture Component\n(C-based, single-threaded)"]:::capture
    ES["Elasticsearch Cluster\n(External, db config)"]:::database
    VC["Viewer Component\n(Node.js Web Interface)"]:::viewer
    UB["User Browser"]:::user
    SS["Support Scripts & Tools"]:::script

    NI -->|"IPv4_packets"| CP
    CP -->|"Session_data(JSON)_PCAP_metadata"| ES
    CP -.->|"Secure_tokens"| VC
    UB -->|"HTTP/REST_API"| VC
    VC <-->|"query/response"| ES
    SS -->|"Deployment_scripts"| CP
    SS -->|"Deployment_commands"| VC
    SS -->|"Deployment_commands"| ES

    click CP "https://github.com/fortheswarm/moloch/tree/master/capture/"
    click VC "https://github.com/fortheswarm/moloch/tree/master/viewer/"
    click ES "https://github.com/fortheswarm/moloch/tree/master/db/"
    click SS "https://github.com/fortheswarm/moloch/tree/master/single-host/"

    classDef external fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef capture fill:#ffcccc,stroke:#aa0000,stroke-width:2px;
    classDef database fill:#ccffcc,stroke:#008800,stroke-width:2px;
    classDef viewer fill:#ffffcc,stroke:#888800,stroke-width:2px;
    classDef script fill:#e6e6fa,stroke:#6666ff,stroke-width:2px;
    classDef user fill:#ffe6e6,stroke:#cc0000,stroke-width:2px;
```
