How noooop/wde works
Main Components
• Client Interfaces:
• Command-line interfaces (chat_cli and related agents)
• Webservers (examples/webserver) that are compatible with both OpenAI and Ollama.
• Core Engine/Workflow Components:
• The main engine orchestration found under the “wde” package (including wde/engine, wde/workflows, wde/agents, wde/tasks). These handle scheduling, session management, workflow definition, and the execution of tasks.
• Workflow modules subdivided into decoding, prefill-only, encode-only, and task-specific pipelines.
• Microservices Framework:
• The microservices module under “wde/microservices”, including entrypoints (HTTP, ZeroMQ-based endpoints for OpenAI and Ollama compatibility) and underlying frameworks (core communication, nameserver, and zero_manager components).
• Benchmark and Testing Infrastructure:
• Benchmarks (e.g., deepseek_v3, offloading_KV_cache, persistence_kv_cache, remote_kv_cache, prefix_caching) for performance testing and profiling.
• Extensive tests that cover core workflow, engine strategy and asynchronous scheduling.
• External Integrations:
• Integration with external APIs and tools like vLLM, and compatibility modules linking to third-party systems (e.g., Ollama, OpenAI).
Relationships and Data Flow
• How a client interface (CLI or webclient) sends a request to the orchestration layer (core engine).
• That the core engine delegates processing to specialized workflow modules (e.g., decoding pipelines or encoder/decoder tasks) and further interacts with low-level tasks within “wde/tasks”.
• Demonstrate that microservices communicate using asynchronous protocols, particularly through ZeroMQ, which minimizes serialization overhead.
• That many workflow components interact with caching mechanisms (such as kv_cache variants available in both benchmarks and wde/workflows). This supports efficient memory and storage usage.
Architectural Patterns and Principles
• Modular and Pluggable Architecture: Components like agents, tasks, and microservices are designed as independent modules that can be loaded on demand.
• Asynchronous Scheduling: Illustrate an orchestration flow in which asynchronous scheduling increases GPU utilization and reduces latency.
• Integration and Extensibility: Display the integration points for external services (vLLM, external APIs) and how performance/benchmarking modules are attached.
Relevant Technologies and Frameworks
• Python for orchestration and client interfaces.
• ZeroMQ for inter-process communication between microservices.
• Asynchronous frameworks (e.g., asyncio and gevent as suggested by test directories) supporting non-blocking IO.
• GPU-oriented execution components as evidenced by modules in wde/workflows/core/executor and wde/engine.