How mingzilla/llm-client works
Project type and purpose
• This is a standalone Java client library module (llm-client) built with Spring WebFlux.
• Purpose: Provide a non-blocking, reactive interface to generic LLM (Large Language Model) HTTP APIs, with support for regular, JSON-streaming and SSE-streaming calls, error handling, and pre-flight validation.
File structure
• Top-level build files: build.gradle, settings.gradle, gradle wrapper, GitHub workflows.
• src/main/java/io/github/mingzilla/llmclient: core client class (LlmClient), domain models (LlmClientInput, LlmClientOutput, LlmClientError, etc.), JSON utilities, verifier and preflight exception.
• src/test/java/...: unit tests for every domain/model class and client methods.
• No “frontend” or “backend” directories because this is a library module.
• Configuration class (LlmClientConfig) provides WebClient bean and instantiates LlmClient.
README for technical stack and design principles
• Built on Spring WebFlux with Reactor Netty HttpClient.
• Fully reactive, non-blocking I/O.
• Separation of concerns: input builders, output models, JSON serialization, streaming.
• Uses Java records for immutable domain types.
• Error handling via LlmClientError and LlmClientPreflightException.
Main components
a. Consumer Application
• Any Spring/WebFlux service or controller that calls LlmClient.verifyAndSend / verifyAndStream / verifyAndStreamSse.
b. LlmClient Module
• Public API (methods: verifyAndSend, verifyAndStream, verifyAndStreamSse)
• Preflight Verifier (LlmClientVerifier)
• Input Builder (LlmClientInput / LlmClientInputBody / LlmClientMessage)
• JSON Utilities (LlmClientJsonUtil)
• Output Models (LlmClientOutput / LlmClientOutputChunk)
• Exception Handling (LlmClientPreflightException, LlmClientError)
c. Spring WebClient Layer
• WebClient bean configured via ReactorClientHttpConnector and Netty HttpClient
d. Network / HTTP Connector
• Reactor Netty HttpClient (connect timeout, response timeout)
e. External LLM API Provider
• Endpoints: /chat/json, /chat/stream, /chat/sse
• Returns JSON, chunked NDJSON, or SSE events
f. Supporting Libraries
• Spring WebFlux, Reactor Core, Netty, Jackson or other JSON parser
Relationships and data flows
• The Consumer Application invokes LlmClient.verifyAnd… methods.
• LlmClient internally calls the Preflight Verifier lambda; may throw LlmClientPreflightException on validation or rate limiting.
• If preflight passes, LlmClient uses Input Builder to construct an HTTP request payload.
• WebClient sends the request over Reactor Netty HttpClient to the External LLM API.
• Response is handled reactively:
• for non-streaming: WebClient retrieves full JSON → JSON Utilities parses into LlmClientOutput → returns Mono<LlmClientOutput>
• for JSON streaming: WebClient returns a Flux of LlmClientOutputChunk parsed by JSON Utilities
• for SSE streaming: WebClient returns Flux<ServerSentEvent<?>>
• The Consumer Application subscribes to the returned Mono/Flux.
Architectural patterns and design principles
• Reactive non-blocking pipeline (Publisher → Subscriber)
• Separation of concerns: input construction, HTTP transport, JSON parsing, error handling
• Immutability: use of Java records for domain types
• Preflight validation strategy via exceptions for early exit
• Bean configuration for testability and DI (LlmClientConfig)
Elements
• Consumer → LlmClient.verify…
• LlmClient → Preflight Verifier → (exception) or (continue)
• LlmClient → WebClient.request → Reactor Netty → External API
• External API → Reactor Netty → WebClient → JSON parser → LlmClientOutputChunk/Output → Consumer
• Annotations for protocols and formats: HTTP, JSON, SSE, NDJSON