# nationalsecurityagency/emissary architecture

> Architecture diagram of the GitHub repository nationalsecurityagency/emissary, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nationalsecurityagency/emissary
- Repository: https://github.com/nationalsecurityagency/emissary

## Overview

##### Project Context and Purpose
- Emissary is a peer-to-peer (P2P) data-driven workflow engine. Its core idea is that data “discovers” its workflow through mobile agents rather than following a rigid, pre-planned itinerary.
- The system is designed for highly configurable, distributed processing where compute nodes – whether in standalone mode or in a clustered P2P network – cooperate to work on data payloads.
- Users are expected to extend certain “places” (e.g., ServiceProviderPlace) to define work, and the system revolves around mobile agents (in the MobileAgent class) and the Directory service (DirectoryPlace) for coordinating nodes and workflows.

##### Analyzing the File Structure for Architectural Patterns
- The Java source tree (src/main/java) is organized into many packages: core, client, command, config, directory, id, jni, kff, pickup, place, pool, roll, server, spi, transform, and util. This modular breakdown indicates a layered architecture where each package tends to operate in a particular area of responsibility (workflow management, networking/communication, service registration, etc.).
- There are also separate configuration directories (src/main/config) and resource files (under src/main/resources) that drive runtime behavior – for example, multiple configuration files for SSL, places, and client connections.
- Docker, Helm, and CI/CD directories in the top-level file tree suggest a containerized deployment strategy and robust automation pipelines.
- The integration tests (under src/test) further reveal the emphasis on verifying subsystem interactions, such as command execution (server, agents, pool, feed) and network coordination (cluster vs. standalone).

##### Core Architectural Components
a. Mobile Agents – Represented by the MobileAgent class and its derivatives, they “carry” data payloads (IBaseDataObject) through various workflow stages.
b. Directory Service – Represented by DirectoryPlace, it is responsible for maintaining a registry of available services (places) and tracking their quality, cost, and connectivity. It is essential for the dynamic task routing and P2P connectivity.
d. Workflow Stages – The engine manages stages such as STUDY, ID, COORDINATE, TRANSFORM, ANALYZE, IO, and REVIEW. These should be indicated as logical phases or sub-modules within the mobile agent lifecycle.
e. Command and Control Interfaces – This includes the CLI commands (server, agents, pool, config, feed, topology) that act as entry-points for deployment, monitoring, and configuration.
f. Networking/Cluster Components – Emphasize the dual-mode operation (standalone vs. cluster). In cluster mode, nodes (servers) and feeders communicate with each other.
g. External Interfaces – SSL configuration, API endpoints (both RESTful API calls and command-line interactions via Picocli), and logging/monitoring infrastructure (logback, metrics).

##### Relationships and Data Flows
- How data payloads originate (for example, from a feeder in cluster mode or a pick-up place in standalone mode) and how they are picked up by mobile agents.
- That once a mobile agent carries a payload through various places, its progress is monitored through API endpoints (e.g., via the server’s MVC APIs and config commands).

##### Architectural Patterns & Principles
- Component-Based Design: Each major package (agent, directory, place, pickup, server, etc.) is a distinct component/module.
- Modular and Extensible Framework: Emphasize that users introduce new “places” by extending provided base classes.
- P2P Network Patterns: Demonstrate how nodes discover and communicate with each other in a distributed network.
- Separation of Concerns: Clearly separate configuration, business logic (workflow handling), and infrastructure (networking, logging, etc.).
- Deployment Patterns: Indicate support for containerized (Docker/Helm) and CI/CD pipelines alongside runtime configurations (e.g., SSL settings).

- Clear Labels: Each component must be clearly labeled with its corresponding package or class function.
- Logical Groupings: Group components by functionality (e.g., “Data Ingestion”, “Workflow Engine”, “Service Registry/Directory”, “External Controls & Monitoring”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Application/Core Tier
    subgraph "Application/Core Tier"
        MA("MobileAgent"):::core
        DP("DirectoryPlace"):::core
        SPP("ServiceProviderPlace"):::core
        TL("ToLowerPlace"):::core
        TU("ToUpperPlace"):::core
        ST("Stage"):::core
    end

    %% Communication/Networking Tier
    subgraph "Communication/Networking Tier"
        CS("EmissaryServer"):::network
        CC("Command Interfaces"):::network
        API("API Endpoints"):::network
    end

    %% Configuration & Infrastructure Tier
    subgraph "Configuration & Infrastructure Tier"
        PC("Peer Configuration"):::config
        EC("External Configuration"):::config
    end

    %% Relationships and Data Flows
    MA -->|"register"| DP
    MA -->|"executesWorkflow"| SPP
    DP -->|"routesTasks"| SPP
    SPP -->|"sampleTransform"| TL
    SPP -->|"sampleTransform"| TU
    MA -->|"progressesThrough"| ST
    CC -->|"controls"| CS
    API -->|"exposes"| CS
    EC -->|"configures"| MA
    EC -->|"configures"| DP
    EC -->|"configures"| CS
    PC -->|"enablesCluster"| CS
    CS -->|"communicatesPeers"| PC

    %% Click Events for Component Mapping
    click MA "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/core/MobileAgent.java"
    click DP "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/directory/DirectoryPlace.java"
    click DP "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/resources/emissary/directory/DirectoryPlace.cfg"
    click SPP "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/place/ServiceProviderPlace.java"
    click TL "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/place/sample/ToLowerPlace.java"
    click TU "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/place/sample/ToUpperPlace.java"
    click ST "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/core/Stage.java"
    click CC "https://github.com/nationalsecurityagency/emissary/tree/main/src/main/java/emissary/command"
    click CS "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/java/emissary/server/EmissaryServer.java"
    click PC "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/config/peer.cfg"
    click PC "https://github.com/nationalsecurityagency/emissary/blob/main/src/main/config/peer-SSL.cfg"
    click EC "https://github.com/nationalsecurityagency/emissary/tree/main/src/main/config"
    click API "https://github.com/nationalsecurityagency/emissary/tree/main/src/main/java/emissary/server/api"

    %% Styles
    classDef core fill:#A7D7C5,stroke:#333,stroke-width:1px;
    classDef network fill:#F9E79F,stroke:#333,stroke-width:1px;
    classDef config fill:#AED6F1,stroke:#333,stroke-width:1px;
```
