How synw/locallm works
Main Components
a. Core Packages (represented as main modules):
• @locallm/types (shared data types)
• @locallm/api (main API interface)
• @locallm/browser (browser-specific implementation)
• @locallm/evaluate (evaluation tools)
• @locallm/evaluations (evaluation implementations)
b. External Providers (represented as external services):
• Llama.cpp Server
• Koboldcpp Server
• Ollama Server
• Wllama (Browser-based)
┌─ Client Layer ─────────────────────────────────────┐
• Applications / Examples │
• API Layer ──────────────────────────────────────┐ │
• @locallm/api @locallm/browser │ │
• Provider Layer ───────────────────────────────┐ │ │
• LlamacppProvider KoboldcppProvider etc. │ │ │
• External Services Layer ────────────────────┐ │ │ │
• Llama.cpp Koboldcpp Ollama Wllama │ │ │ │
• ┘ │ │ │
• ┘ │ │
• ┘ │
• ┘
Key Relationships
a. Data Flow:
• Client → API Layer → Provider Layer → External Services
• Response flow in reverse direction
b. Dependencies:
• All packages depending on @locallm/types
• Provider-specific implementations connecting to their respective servers
Implementation Details
Use these symbols and conventions:
• Diamonds: Entry points
• Rectangles: Modules/packages
Additional Elements
Annotations for:
• API endpoints
• Configuration options
• Type definitions
• Evaluation paths
The evaluation system's integration
Keep the abstraction level focused on the architectural components while maintaining enough detail to show the distinct provider implementations and their relationships to external services.