# agentjido/req_llm architecture

> Architecture diagram of the GitHub repository agentjido/req_llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/agentjido/req_llm
- Repository: https://github.com/agentjido/req_llm

## Overview

##### Project Type and Purpose
- ReqLLM is an Elixir library (not a standalone front-end/back-end app) that provides a unified, plugin-based interface for interacting with large language model (LLM) providers via HTTP.
- It builds on the Req HTTP client, adds typed data structures, streaming support, and a provider plugin architecture to simplify calls to OpenAI, Anthropic, Google, Groq, xAI, OpenRouter, and more.

##### High-Level Architectural Overview
- Core Library Modules (in `lib/req_llm/`)
- Context & Message builders (compose system/user prompts)
- Model metadata & schema definitions
- Provider plugin framework (registry, DSL, option translators)
- Capability modules (generation, embeddings, streaming, tool-calling)
- Response processing & usage extraction
- Utility modules (key management, codecs, JSON encoders)
- Mix Tasks (in `lib/mix/tasks/`):
- `model_sync`: auto-sync metadata from models.dev into JSON files under `priv/models_dev/`
- `stream_text`: a CLI helper for streaming text from the command line
- External Services:
- LLM Providers (via HTTP REST endpoints)
- models.dev API (for model metadata sync)

##### Main Components and Their Relationships
- Client API Layer (public functions like `generate_text/3`, `stream_text/3`, `embed_many/2`)
→ Builds a `ReqLLM.Context` struct and selects a `ReqLLM.Model`.
- Provider Plugin Layer
→ `ReqLLM.Provider.Registry` resolves the correct provider module (e.g., `ReqLLM.Providers.OpenAI`).
→ Uses a Provider DSL and `ContextCodec` to translate internal structs into provider-specific HTTP payloads.
- Req HTTP Pipeline
→ The provider module attaches middleware and executes `Req.request/1`.
→ Streaming calls produce a `Stream` of `ReqLLM.StreamChunk` entries.
- Response Handling
→ Raw JSON from HTTP response is decoded into Elixir structs (`Response`, `Usage`, content parts, etc.).
→ Cost and token usage are extracted into a `ReqLLM.Usage` struct.
→ Result is returned to the caller (either `{:ok, struct}` or raised on `!` variants).
- Model Metadata Management
→ `mix req_llm.models.sync` fetches metadata from models.dev, writes to `priv/models_dev/*.json`.
→ On startup (`ReqLLM.Application`), metadata JSON is loaded into memory for auto-sync and option validation.

##### Architectural Patterns and Design Principles
- Plugin Architecture: providers implement a common behavior (`ReqLLM.Plugin`) and register themselves, enabling easy extension.
- Composability: builds on Req’s middleware pipeline for tracing, instrumentation, and custom middleware injection.
- Typed Data Structures: every call, response, and intermediate piece is a well-defined struct that can be serialized.
- Separation of Concerns: split into context construction, provider translation, HTTP transport, and response decoding.
- Mix Tasks for operational concerns (metadata sync, CLI tooling).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Public API Layer
    subgraph "Public API Layer"
        API["Public API\n(generate_text/3,\nstream_text/3,\nembed_many/2)"]:::core
    end
    click API "https://github.com/agentjido/req_llm/blob/main/lib/req_llm.ex"

    %% Application & Startup
    subgraph "Application & Startup"
        App["ReqLLM.Application"]:::core
    end
    click App "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/application.ex"

    %% Context & Message Builders
    subgraph "Context & Message Builders"
        Context["Context Builder"]:::core
        Codec["Context Codec"]:::core
        Message["Message Builder"]:::core
        ContentPart["Content Part"]:::core
    end
    click Context "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/context.ex"
    click Codec "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/context/codec.ex"
    click Message "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/message.ex"
    click ContentPart "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/message/content_part.ex"

    %% Model Schema & Metadata
    subgraph "Model Schema & Metadata"
        SchemaNode["Schema"]:::core
        ModelNode["Model Metadata Struct"]:::core
    end
    click SchemaNode "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/schema.ex"
    click ModelNode "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/model.ex"

    %% Provider Plugin Framework
    subgraph "Provider Plugin Framework"
        ProviderBehaviour["Provider Behaviour"]:::plugin
        DSL["Provider DSL"]:::plugin
        Options["Option Translators"]:::plugin
        UtilsP["Provider Utils"]:::plugin
        Registry["Provider Registry"]:::plugin
    end
    click ProviderBehaviour "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/provider.ex"
    click DSL "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/provider/dsl.ex"
    click Options "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/provider/options.ex"
    click UtilsP "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/provider/utils.ex"
    click Registry "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/provider/registry.ex"

    %% Provider Implementations
    subgraph "Provider Implementations"
        Providers["Providers\n(lib/req_llm/providers/*)"]:::plugin
    end
    click Providers "https://github.com/agentjido/req_llm/tree/main/lib/req_llm/providers"

    %% Capability Modules
    subgraph "Capability Modules"
        CapCore["Capability Behaviour"]:::core
        Gen["Text Generation"]:::core
        Embed["Embeddings"]:::core
        StreamMod["Streaming Support"]:::core
        Tool["Tool-Calling"]:::core
    end
    click CapCore "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/capability.ex"
    click Gen "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/generation.ex"
    click Embed "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/embedding.ex"
    click StreamMod "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/stream_chunk.ex"
    click StreamMod "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/step/stream.ex"
    click Tool "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/tool.ex"

    %% Response Processing
    subgraph "Response Processing"
        Response["Response Struct"]:::core
        RespCodec["Response Codec"]:::core
        Usage["Usage Tracking"]:::core
    end
    click Response "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/response.ex"
    click RespCodec "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/response/codec.ex"
    click Usage "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/step/usage.ex"

    %% Utility Modules
    subgraph "Utility Modules"
        Keys["Key Management"]:::core
        Utils["General Utils"]:::core
    end
    click Keys "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/keys.ex"
    click Utils "https://github.com/agentjido/req_llm/blob/main/lib/req_llm/utils.ex"

    %% Mix Tasks & Metadata Store
    subgraph "Mix Tasks & Metadata Store"
        ModelSync["mix req_llm.models.sync"]:::task
        StreamText["mix stream_text"]:::task
        MetaStore["Metadata Store\n(priv/models_dev/*.json)"]:::store
    end
    click ModelSync "https://github.com/agentjido/req_llm/blob/main/lib/mix/tasks/model_sync.ex"
    click StreamText "https://github.com/agentjido/req_llm/blob/main/lib/mix/tasks/stream_text.ex"
    click MetaStore "https://github.com/agentjido/req_llm/tree/main/priv/models_dev/"

    %% External Services
    subgraph "External Services"
        LLM["LLM Provider APIs"]:::external
        ModelsDev["models.dev API"]:::external
    end

    %% Request/Response Flow
    API -->|"builds Context & selects Model"| Context
    API -->|"uses Schema & Model"| SchemaNode
    API --> Context
    Context -->|"compose messages"| Message
    Message --> ContentPart
    Context --> Codec

    Context -->|"resolve provider"| Registry
    Registry -->|"returns provider module"| ProviderBehaviour
    ProviderBehaviour --> DSL
    DSL --> Options
    DSL --> UtilsP
    ProviderBehaviour --> Providers

    ProviderBehaviour -->|"attach middleware"| ReqPipeline["Req HTTP Pipeline"]:::core

    ReqPipeline -->|"HTTP POST /chat"| LLM
    ReqPipeline -->|"stream StreamChunk"| StreamMod
    LLM -->|"JSON response"| ReqPipeline
    ReqPipeline -->|"raw JSON"| RespCodec
    RespCodec --> Response
    Response --> Usage
    Usage -->|"return {:ok, struct}"| API

    %% Metadata Sync Flow
    ModelSync -->|"fetch metadata"| ModelsDev
    ModelsDev -->|"JSON files"| MetaStore
    App -->|"load metadata"| MetaStore

    %% Styles
    classDef core fill:#D0E6FF,stroke:#0366d6,color:#0366d6
    classDef plugin fill:#E6FFED,stroke:#28a745,color:#28a745
    classDef external fill:#FFE6CC,stroke:#FF8C00,color:#FF8C00
    classDef task fill:#F0F0F0,stroke:#888,color:#333
    classDef store fill:#F0FFF0,stroke:#0A0,color:#0A0
```
