# hydro-project/hydro architecture

> Architecture diagram of the GitHub repository hydro-project/hydro, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hydro-project/hydro
- Repository: https://github.com/hydro-project/hydro

## Overview

##### High-Level Architecture Overview

**┌─────────────────────────────────────────────┐**
- Hydro Language Layer            │
- ┤
- DFIR (Dataflow IR) Layer            │
- ┤
- Runtime & Deployment Layer          │
- ┘

##### Core Components

a) Hydro Language Layer:
- hydro_lang (compiler and IR)
- hydro_std (standard library)
- lattices (distributed data structures)
- stageleft (metaprogramming support)

b) DFIR Layer:
- dfir_lang (core IR implementation)
- dfir_datalog (datalog integration)
- pusherator (stream operators)
- relalg (relational algebra)

c) Runtime & Deployment Layer:
- hydro_deploy (deployment management)
- hydro_cli (command-line interface)
- topolotree (network topology)

##### Data Flow and Communication

**User Code → Hydro Language → DFIR Compilation → Runtime Distribution**

##### Deployment Architecture
A separate section showing deployment options:

**┌─────────────┐    ┌─────────────┐    ┌─────────────┐**
- Local     │    │    Cloud    │    │   Custom    │
- Deployment  │←→│  Deployment  │←→│  Deployment  │
- ┘    └─────────────┘    └─────────────┘

##### Key Design Elements to Highlight

a) Compiler Pipeline:
- Source code parsing
- IR generation
- Optimization passes
- Code generation

b) Runtime Features:
- Distributed coordination
- State management
- Network communication
- Error handling

c) Development Tools:
- Testing frameworks
- Benchmarking tools
- Deployment utilities

##### Component Relationships
Relationships between components using different line styles:

##### Implementation Details
Technical details in annotations:
- Rust version requirements
- External dependencies
- Configuration options
- Performance characteristics

##### Optimization Layer
A section showing the optimization pipeline:
**Source → IR → Optimization Passes → Distributed Execution Plan**

##### Testing & Verification
A section for testing components:
- Unit tests
- Integration tests
- Distributed tests
- Property-based testing

##### The integration with Rust's ecosystem

Remember to:
- Use consistent notation
- Provide zoom levels (overview → detailed views)
- Document cross-cutting concerns

This architecture captures Hydro's role as a "LLVM for the cloud" while showing its integration with Rust and its focus on distributed systems programming.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles
    classDef language fill:#b3d9ff,stroke:#333,color:black,stroke-width:4px
    classDef dfir fill:#99ff99,stroke:#333,color:black,stroke-width:4px
    classDef runtime fill:#ffcc99,stroke:#333,color:black,stroke-width:4px
    classDef testing fill:#e6b3ff,stroke:#333
    classDef deployment fill:#ffd9b3,stroke:#333

    %% Main Layers
    subgraph Language["Hydro Language Layer"]
        HydroLang["hydro_lang"]:::language
        HydroStd["hydro_std"]:::language
        Lattices["lattices"]:::language
        StageLeft["stageleft"]:::language
    end

    subgraph DFIR["DFIR (Dataflow IR) Layer"]
        DfirLang["dfir_lang"]:::dfir
        DfirDatalog["dfir_datalog"]:::dfir
        Pusherator["pusherator"]:::dfir
        RelAlg["relalg"]:::dfir
    end

    subgraph Runtime["Runtime & Deployment Layer"]
        HydroDeploy["hydro_deploy"]:::runtime
        HydroCLI["hydro_cli"]:::runtime
        TopoloTree["topolotree"]:::runtime
    end

    %% Compiler Pipeline
    subgraph Pipeline["Compiler Pipeline"]
        direction TB
        Source["Source Code"] --> IRGen["IR Generation"]
        IRGen --> Optimize["Optimization"]
        Optimize --> CodeGen["Code Generation"]
    end

    %% Deployment Options
    subgraph Deploy["Deployment Options"]
        direction TB
        Local["Local Deployment"]:::deployment
        Cloud["Cloud Deployment"]:::deployment
        Custom["Custom Deployment"]:::deployment
    end

    %% Testing Components
    subgraph Testing["Testing & Verification"]
        direction TB
        UnitTest["Unit Tests"]:::testing
        IntegTest["Integration Tests"]:::testing
        DistTest["Distributed Tests"]:::testing
        PropTest["Property Tests"]:::testing
    end

    %% Relationships
    Language --> DFIR
    DFIR --> Runtime
    Runtime --> Deploy
    Pipeline --> Testing

    %% Click Events
    click HydroLang "https://github.com/hydro-project/hydro/tree/main/hydro_lang"
    click HydroStd "https://github.com/hydro-project/hydro/tree/main/hydro_std"
    click Lattices "https://github.com/hydro-project/hydro/tree/main/lattices"
    click StageLeft "https://github.com/hydro-project/hydro/tree/main/stageleft"
    click DfirLang "https://github.com/hydro-project/hydro/tree/main/dfir_lang"
    click DfirDatalog "https://github.com/hydro-project/hydro/tree/main/dfir_datalog"
    click Pusherator "https://github.com/hydro-project/hydro/tree/main/pusherator"
    click RelAlg "https://github.com/hydro-project/hydro/tree/main/relalg"
    click HydroDeploy "https://github.com/hydro-project/hydro/tree/main/hydro_deploy/core"
    click HydroCLI "https://github.com/hydro-project/hydro/tree/main/hydro_deploy/hydro_cli"
    click TopoloTree "https://github.com/hydro-project/hydro/tree/main/topolotree"

    %% Legend
    subgraph Legend
        L1["Language Components"]:::language
        L2["DFIR Components"]:::dfir
        L3["Runtime Components"]:::runtime
        L4["Testing Components"]:::testing
        L5["Deployment Components"]:::deployment
    end
```
