How kagisearch/pyllms works
Project type and purpose
• PyLLMs is an open-source Python library (not a full-stack app) whose goal is to provide a unified, minimal interface for interacting with many LLM providers and running benchmarks.
• It exposes a simple API (llms.init, .complete, .benchmark) that hides the details of each provider’s HTTP/API calls.
File structure
• Top-level: setup.py, requirements.txt, README.md – standard Python package.
• llms/llms.py: the core factory and façade exposing init(), complete(), acomplete(), benchmark().
• llms/providers/: a plugin directory of provider modules.
• base_provider.py defines the Provider interface/abstract base.
• concrete modules (openai.py, anthropic.py, cohere.py, etc.) implement that interface.
• llms/results/result.py: standardizes response metadata (tokens, cost, latency).
README for additional insights
• Documentation describes sync/async, streaming, multi-model, chat-history, token counting, benchmarks.
Elements
• Clear labels: “Client Code,” “PyLLMs Core,” “Provider Factory,” “Provider (e.g., OpenAIProvider),” “Result Normalizer,” “External LLM Service.”
• From Client → Core → Factory → Provider → External Service → Provider → Result Normalizer → Core → Client