# metlo-labs/metlo architecture

> Architecture diagram of the GitHub repository metlo-labs/metlo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/metlo-labs/metlo
- Repository: https://github.com/metlo-labs/metlo

## Overview

##### Main System Components
- Backend Services: The “backend” folder comprises the core API server, with routes, middleware, migrations, models, services, jobs, and error handling for tasks like endpoint analysis, trace processing, alerting, and API configuration.
- Frontend Application: The “frontend” folder represents the user interface (likely built using Next.js) where dashboards, alert listings, endpoint statuses, and configuration screens are provided.
- Shared Common Library: The “common” directory houses shared models, utilities, and API definitions that are used by both frontend and backend.
- CLI Tool: The “cli” component is a separate command-line interface that likely provides management tasks, integration with cloud services, and developer utilities.
- Ingestors: The “ingestors” directory shows multi-language integration modules (Node, Golang, Java, Python, etc.) that are deployed on the client side to intercept or mirror API traffic for Metlo’s analysis engine.
- Testing Framework: The “testing” folder shows scripts, resource definitions, and templates created to simulate API calls and validate security tests.

##### Relationships and Interactions
- The ingestors (for various languages/frameworks) monitor and collect API traffic from customers’ environments. They forward the traces and logs to the Backend Services.
- The Backend then parses this data via its API endpoints and services (organized into modules such as log-request, analyzer, alert, testing-config, etc.). It communicates with the database using models and migration scripts to persist the inventory of endpoints, traces, and alert information.
- Scheduled and on-demand Jobs (indicated by the jobs.ts and job runner scripts) process the accumulated data—aggregating traces, detecting anomalies, and generating alerts.
- The CLI and Deployment components facilitate setup, management, and automation of Metlo in various cloud environments.
- The Testing component acts as an automated layer that uses defined YAML or JavaScript templates to simulate API scenarios (like broken authentication or SQL injection) against Metlo’s endpoints.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Each component (Backend, Frontend, Ingestors, CLI) has a clear responsibility.
- Modularization: The project’s structure with directories for “services”, “models”, “api”, “ingestors”, etc., shows a modular design that allows extension or customization.
- Consistent Interfacing: The “common” directory centralizes shared data models and utility functions to ensure consistency between clients (CLI, ingestors, etc.) and server.
- Multi-language Integration: Ingestors are written in multiple languages to integrate with varied technology stacks at customer sites.
- Job Processing/Asynchronous Handling: The “jobs” and “services/jobs” components indicate processing pipelines for background tasks related to alerting and trace analysis.

##### Detailed Element Suggestions
- Use annotations to note that the ingestors are language/platform-specific adapters that feed data into the central backend.
- A “Security Testing” module where your testing framework sends simulated attack requests to validate system behavior.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "API Security Backend"
        BE1["API Endpoints"]:::backend
        BE2["Middleware"]:::backend
        BE3["Service Layer (Alert,Logging,Trace Analysis)"]:::backend
        BE4["Job Scheduler"]:::backend
        BE1 -->|"processes"| BE2
        BE2 -->|"processes"| BE3
        BE3 -->|"triggers"| BE4
    end

    DB["Database (Persistent Data Layer)"]:::database
    FE["Frontend UI"]:::frontend
    CLI["CLI Tool"]:::cli
    COM["Shared Common Library"]:::common
    ING["Ingestors"]:::ingestors
    DEP["Deployment & Infrastructure"]:::deploy
    TEST["Testing Framework"]:::testing

    ING -->|"sendsTraceData"| BE1
    BE3 -->|"CRUDops"| DB
    FE -->|"APIcalls"| BE1
    CLI -->|"adminOps"| BE1
    CLI -->|"setupMgmt"| DEP
    TEST -->|"simulateAttack"| BE1

    COM ---|"sharedModels"| BE1
    COM ---|"utils"| ING
    COM ---|"APIDefinitions"| CLI

    classDef backend fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef frontend fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef common fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef cli fill:#FFA500,stroke:#000,stroke-width:2px;
    classDef ingestors fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef deploy fill:#D8BFD8,stroke:#000,stroke-width:2px;
    classDef testing fill:#FF7F7F,stroke:#000,stroke-width:2px;
    classDef database fill:#C0C0C0,stroke:#000,stroke-width:2px;

    click BE1 "https://github.com/metlo-labs/metlo/tree/develop/backend"
    click FE "https://github.com/metlo-labs/metlo/tree/develop/frontend"
    click COM "https://github.com/metlo-labs/metlo/tree/develop/common"
    click CLI "https://github.com/metlo-labs/metlo/tree/develop/cli"
    click ING "https://github.com/metlo-labs/metlo/tree/develop/ingestors"
    click DEP "https://github.com/metlo-labs/metlo/tree/develop/deploy"
    click TEST "https://github.com/metlo-labs/metlo/tree/develop/testing"
```
