ejnkns/hive

1. What this repo is

- hive is a local, OpenAI-compatible proxy daemon written in TypeScript for Node.js.

- Its core role is to sit between coding clients and multiple upstream LLM providers, then choose, retry, and fail over between provider:model nodes automatically.

2. Top-level runtime shape

- The main runtime appears to be centered around src/hive.ts and src/engine.ts, which likely bootstrap the proxy, telemetry, provider discovery, and dashboard/server pieces.

- src/hive/server-config.ts and src/hive/create-server/* define the local server boundary and route wiring.

3. Request path through the proxy

- Incoming OpenAI-style requests enter src/proxy.ts.

- The request is parsed, mutated, routed, and executed against a selected upstream model.

- The proxy returns a streamed response when successful, or normalizes upstream failures and retries other nodes when needed.

4. Routing and failover

- src/proxy/node-selector.ts is the decision point for picking the best provider:model based on current telemetry and session affinity.

- src/proxy/failover.ts handles retrying alternate nodes after failures, with a bounded retry loop.

- src/proxy/mutate-request.ts injects provider-specific auth/headers and adapts the client request to the chosen upstream.

5. Routing memory and health state

- src/proxy/routing-memory/* is the in-memory control plane for routing behavior.

- It includes a circuit breaker for temporarily disabling bad providers, feature discovery for remembering unsupported capabilities, and a session registry for sticking a session to a node when appropriate.

6. Provider catalog and discovery

- src/providers.ts, src/providers/registry.ts, and the individual provider files under src/providers/* define the supported upstreams.

- The repo supports multiple external model backends, including Groq, SambaNova, Google AI, NVIDIA NIM, GitHub Models, Cerebras, Mistral, OpenCode Zen, Ollama, oMLX, LM Studio, and others.

- src/providers/discovery.ts fetches available models from each provider’s /models endpoint at startup and caches the results.

7. Configuration and startup inputs

- Provider API keys and host overrides are loaded from .env or environment variables during initialization.

- src/hive/load-providers.ts and src/hive/manual-override.ts suggest two config layers: automatic provider loading plus an explicit override path for operator control.

- src/id.ts likely provides stable identity/session IDs used across routing and telemetry.

8. Telemetry as the feedback loop

- src/telemetry/* is the scoring and persistence subsystem that feeds routing decisions.

- It records request outcomes, stream characteristics, latency, throughput, and error classes, then turns them into node scores.

- src/telemetry/persist.ts writes cache data to disk, and the README says this lives under ~/.hive/telemetry-cache.json.

9. How scoring is computed

- src/telemetry/calculate-node-score/* breaks scoring into separate dimensions: latency, throughput, quality, reliability, and auth checks.

- The system also tracks windows, weights, and derived metrics, so routing can prefer fast and stable nodes rather than just “last successful” ones.

- Stream parsing and error classification live in src/telemetry/recorder/*, which helps distinguish real successes from truncated or failed streams.

10. Dashboard and operator visibility

- The browser UI lives under src/hive/ui/* and is served from src/hive/index.html.

- It is a lightweight Web Components dashboard showing provider status, scores, activity, logs, conversations, and stats.

- src/hive/create-server/assign-routes.ts likely exposes both the API surface and the dashboard assets from the same local server.

11. Logging and shared utilities

- src/hive/shared/logger/* provides centralized logging and banner output.

- src/utils/error-normalizer.ts and the proxy/telemetry helper modules are cross-cutting support code for keeping upstream failures comparable and actionable.

12. Repository boundaries to draw in a diagram

- One boundary is the client side: OpenCode or any OpenAI-compatible client sends requests to hive.

- A second boundary is the local hive daemon: proxy, routing memory, telemetry, provider registry, and dashboard server.

- A third boundary is the external provider layer: multiple third-party LLM APIs and local model servers.

- The key data loop is: request in → node selection → upstream call → result/error classification → telemetry update → future routing improvement.

13. Tooling and validation

- The project is TypeScript-based and includes tests alongside each major subsystem, indicating behavior is heavily verified at the routing, provider, telemetry, and proxy levels.

- vite.config.ts and the UI folder suggest the dashboard is built and served as a frontend bundle, while the daemon itself is a Node process.