# noooop/wde architecture

> Architecture diagram of the GitHub repository noooop/wde, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/noooop/wde
- Repository: https://github.com/noooop/wde

## Overview

##### Main Components
- Client Interfaces:
- Command-line interfaces (chat_cli and related agents)
- Webservers (examples/webserver) that are compatible with both OpenAI and Ollama.
- Core Engine/Workflow Components:
- The main engine orchestration found under the “wde” package (including wde/engine, wde/workflows, wde/agents, wde/tasks). These handle scheduling, session management, workflow definition, and the execution of tasks.
- Workflow modules subdivided into decoding, prefill-only, encode-only, and task-specific pipelines.
- Microservices Framework:
- The microservices module under “wde/microservices”, including entrypoints (HTTP, ZeroMQ-based endpoints for OpenAI and Ollama compatibility) and underlying frameworks (core communication, nameserver, and zero_manager components).
- Benchmark and Testing Infrastructure:
- Benchmarks (e.g., deepseek_v3, offloading_KV_cache, persistence_kv_cache, remote_kv_cache, prefix_caching) for performance testing and profiling.
- Extensive tests that cover core workflow, engine strategy and asynchronous scheduling.
- External Integrations:
- Integration with external APIs and tools like vLLM, and compatibility modules linking to third-party systems (e.g., Ollama, OpenAI).

##### Relationships and Data Flow
- How a client interface (CLI or webclient) sends a request to the orchestration layer (core engine).
- That the core engine delegates processing to specialized workflow modules (e.g., decoding pipelines or encoder/decoder tasks) and further interacts with low-level tasks within “wde/tasks”.
- Demonstrate that microservices communicate using asynchronous protocols, particularly through ZeroMQ, which minimizes serialization overhead.
- That many workflow components interact with caching mechanisms (such as kv_cache variants available in both benchmarks and wde/workflows). This supports efficient memory and storage usage.

##### Architectural Patterns and Principles
- Modular and Pluggable Architecture: Components like agents, tasks, and microservices are designed as independent modules that can be loaded on demand.
- Asynchronous Scheduling: Illustrate an orchestration flow in which asynchronous scheduling increases GPU utilization and reduces latency.
- Integration and Extensibility: Display the integration points for external services (vLLM, external APIs) and how performance/benchmarking modules are attached.

##### Relevant Technologies and Frameworks
- Python for orchestration and client interfaces.
- ZeroMQ for inter-process communication between microservices.
- Asynchronous frameworks (e.g., asyncio and gevent as suggested by test directories) supporting non-blocking IO.
- GPU-oriented execution components as evidenced by modules in wde/workflows/core/executor and wde/engine.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation/Interface Layer
    subgraph "Presentation/Interface Layer"
        CLI["Client Interface (CLI)"]:::interface
        WEB["Client Interface (Webserver)"]:::interface
    end

    %% Application/Orchestration Layer
    subgraph "Application/Orchestration Layer"
        ENG["Core Engine"]:::engine
        WF["Workflow Orchestrator"]:::engine
        subgraph "Agents Module"
            AG1["Agents (wde)"]:::engine
            AG2["Agents (app)"]:::engine
        end
    end

    %% Task & Workflow Modules
    subgraph "Task & Workflow Modules"
        TASK["Task Modules (Decoding,Encoding,Reranker,Retriever)"]:::task
    end

    %% Microservices Layer
    subgraph "Microservices Layer"
        MS["Microservices (HTTP,ZeroMQ)"]:::micro
    end

    %% Data & Caching Layer
    subgraph "Benchmarking & Profiling"
        DS["Deepseek v3"]:::cache
        OFF["Offloading KV Cache"]:::cache
        PER["Persistence KV Cache"]:::cache
        PRE["Prefix Caching"]:::cache
        REM["Remote KV Cache"]:::cache
        RET["Retriever Benchmark"]:::cache
    end

    %% Testing Infrastructure
    subgraph "Testing Infrastructure"
        TEST["Testing Infrastructure"]:::test
    end

    %% External Integrations
    subgraph "External Integrations"
        EXT["External Integrations (vLLM,OpenAI,Ollama)"]:::integration
    end

    %% Data Flow Connections
    CLI -->|"sends_request"| ENG
    WEB -->|"sends_request"| ENG
    
    ENG -->|"delegates_to"| WF
    ENG -->|"notifies"| AG1
    ENG -->|"notifies"| AG2
    WF -->|"triggers"| TASK

    ENG -->|"communicates_async"| MS
    WF -->|"communicates_async"| MS

    TASK -->|"utilizes"| DS
    TASK -->|"utilizes"| OFF
    TASK -->|"utilizes"| PER
    TASK -->|"utilizes"| PRE
    TASK -->|"utilizes"| REM
    TASK -->|"utilizes"| RET

    ENG -->|"logs_tests"| TEST
    WF -->|"logs_tests"| TEST

    MS -->|"integrates_with"| EXT
    AG1 -->|"integrates_with"| EXT
    AG2 -->|"integrates_with"| EXT

    %% Click Events
    click CLI "https://github.com/noooop/wde/tree/main/applications/chat_cli"
    click WEB "https://github.com/noooop/wde/tree/main/examples/webserver"
    click ENG "https://github.com/noooop/wde/tree/main/wde/engine"
    click WF "https://github.com/noooop/wde/tree/main/wde/workflows"
    click AG1 "https://github.com/noooop/wde/tree/main/wde/agents"
    click AG2 "https://github.com/noooop/wde/tree/main/applications/agents"
    click TASK "https://github.com/noooop/wde/tree/main/wde/tasks"
    click MS "https://github.com/noooop/wde/tree/main/wde/microservices"
    click DS "https://github.com/noooop/wde/tree/main/benchmarks/deepseek_v3"
    click OFF "https://github.com/noooop/wde/tree/main/benchmarks/offloading_KV_cache"
    click PER "https://github.com/noooop/wde/tree/main/benchmarks/persistence_kv_cache"
    click PRE "https://github.com/noooop/wde/tree/main/benchmarks/prefix_caching"
    click REM "https://github.com/noooop/wde/tree/main/benchmarks/remote_kv_cache"
    click RET "https://github.com/noooop/wde/tree/main/benchmarks/retriever"
    click TEST "https://github.com/noooop/wde/tree/main/tests"

    %% Styles
    classDef interface fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef engine fill:#A9DFBF,stroke:#27AE60,stroke-width:2px;
    classDef task fill:#F9E79F,stroke:#D4AC0D,stroke-width:2px;
    classDef micro fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef cache fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef test fill:#FAD7A0,stroke:#E67E22,stroke-width:2px;
    classDef integration fill:#FDEBD0,stroke:#D35400,stroke-width:2px;
```
