# kashinathj/local-llm-and-chatbot architecture

> Architecture diagram of the GitHub repository kashinathj/local-llm-and-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kashinathj/local-llm-and-chatbot
- Repository: https://github.com/kashinathj/local-llm-and-chatbot

## Overview

##### Project type + purpose (what you’re diagramming)
This is a **2-process, local-first “LLM-backed recipe assistant”**:
- **UI process**: Streamlit chat app (`src/bot.py`) running in the user’s browser via Streamlit server.
- **API process**: FastAPI backend (`src/api.py`) that exposes `POST /generate-recipe` and orchestrates LLM calls.
- **LLM providers (mode-dependent)**:
- **Local**: Ollama server running separately on the machine (Llama 3.1).
- **Cloud**: OpenAI API (GPT‑4o) reachable over the internet using `OPENAI_API_KEY` set in the **API server environment**.
- **Shared domain model**: Pydantic schema in `src/models.py` (Recipe JSON contract).
- **Prompting assets / behavior shaping**: few-shot dataset + Chef persona prompt in `src/utils.py`.
- **LangChain glue**: used in the API logic to call either `ChatOllama` or `ChatOpenAI` and to produce **structured JSON output** (or fallback JSON parsing for local mode).

This is not a monolith; it’s a small distributed system (even on localhost) with explicit boundaries (UI ↔ API ↔ LLM provider).

**Top-level**
- `requirements.txt`: runtime dependencies (call out FastAPI, Streamlit, LangChain, Pydantic, Uvicorn, etc.).
- `run_api.*`, `run_bot.*`, `run_streamlit.ps1`: “operational entrypoints” (useful for a deployment/runtime view).

**src/**
- `api.py` → **Backend service boundary** (FastAPI routes, request handling, LLM orchestration, env var checks, mode routing).
- `bot.py` → **Frontend boundary** (Streamlit UI, mode toggle, chat flow, calling the API, rendering recipe + nutrition sidebar).
- `utils.py` → **Prompting / few-shot dataset** (static assets influencing LLM output).
- `utils.py` + `api.py` together → “Prompt assembly + LLM invocation pipeline”.

- **Two modes**: `mode=local` vs `mode=openai` passed from UI → API.
- **Structured output**:
- OpenAI path uses `with_structured_output` (strong typing).
- Local path requires JSON extraction/parsing from LLM text (compatibility fallback).
- **API key location**: `OPENAI_API_KEY` is required **only in API process env**, not in the browser.
- **Ports/process separation**:
- FastAPI on `127.0.0.1:8001`
- Streamlit on `localhost:8501`
- **Healthcheck**: `GET /health` exists; include it as an operational endpoint.

##### Streamlit App Container (process)
- Runs `src/bot.py`
- Responsible for chat UI, mode selection, rendering recipe
##### FastAPI Service Container (process)
- Runs `src/api.py` via Uvicorn
- Exposes `/generate-recipe`, `/health`
##### Ollama Server (Local LLM Runtime) (external container but on same machine)
- Hosts `llama3.1`
##### OpenAI API (external SaaS)

- Browser ↔ Streamlit: user messages + UI rendering
- Streamlit → FastAPI: HTTP `POST /generate-recipe` with `{ingredients[], mode}`
- FastAPI → Ollama: (local mode) LangChain `ChatOllama` call
- FastAPI → OpenAI: (openai mode) LangChain `ChatOpenAI` call (requires `OPENAI_API_KEY`)
- FastAPI → Streamlit: Recipe JSON response
- Streamlit → Browser: formatted recipe + nutrition sidebar

**Annotate**: ports (`8501`, `8001`) and “localhost” vs “internet”.

- **API Routes / Controllers** (`src/api.py`)
- `/generate-recipe` handler
- `/health`
- Mode router: if `mode=openai` else `local`
- Error handling for missing/invalid key (OpenAI path)
- **Prompt Builder** (`src/utils.py`)
- Chef persona system prompt
- Few-shot dataset (10+ recipes)
- **Schema / Validation** (`src/models.py`)
- `Recipe` Pydantic model (response contract)
- **Structured Output Adapter**
- OpenAI: `with_structured_output(Recipe)`
- Local: “JSON parsing from model text” fallback

Routes → Orchestrator → Prompt Builder → LLM Client → (Structured Output Adapter) → Schema Validation → return JSON

This view is where you highlight the **architectural pattern**: “thin HTTP layer + orchestration pipeline + deterministic schema contract”.

Create **two sequences** (or one sequence with an `alt` block):

**Sequence: Generate Recipe**
##### `alt mode=openai`
- FastAPI uses `ChatOpenAI` + structured output
- OpenAI returns typed/structured response
##### `else mode=local`
- FastAPI uses `ChatOllama`
- Receives text, extracts JSON, validates to `Recipe`
##### Streamlit renders recipe (bold headers, numbered steps) + Nutrition Facts sidebar

Explicit failure branch:
- If `mode=openai` and API key missing/invalid → FastAPI returns error → Streamlit shows message and suggests Local mode.

##### Relationships & interactions to explicitly show (don’t omit)
These are the details that often get lost but matter here:
- **Trust boundaries**:
- Browser/UI does *not* hold secrets; API process holds `OPENAI_API_KEY`.
- **Contract boundary**:
- **Provider abstraction**:
- **Local runtime dependency**:
- Ollama must be running; call that out as a prerequisite note on the Ollama container.
- **Operational endpoints**:
- `/health` used for readiness checks and debugging.

**Streamlit (src/bot.py)**
- Chat UI (input, message history)
- Mode selector (Local/OpenAI)
- API Client (requests to FastAPI)
- Renderer (recipe formatting + nutrition sidebar)

**FastAPI (src/api.py)**
- Route: `/generate-recipe`
- Route: `/health`
- Request parsing (ingredients, mode)
- Prompt assembly (calls utils)
- LLM provider router
- OpenAI adapter (structured output)
- Ollama adapter (text → JSON parse)
- Pydantic validation (Recipe)
- Error mapper (key missing/invalid → user-friendly message)

**Shared**
- Recipe schema (`src/models.py`)
- Few-shot dataset + Chef prompt (`src/utils.py`)

**External**
- Ollama daemon + llama3.1 model
- OpenAI API

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% =========================
    %% View A: C4 L2 Containers
    %% =========================
    subgraph "C4 L2 — Container View (Local-first, 2-process)"
        direction TB

        User["User"]:::person
        Browser["Browser"]:::client

        subgraph "User Machine (localhost)"
            direction TB

            StreamlitProc["Streamlit App (Port 8501)"]:::firstparty
            FastAPIProc["FastAPI Service (Port 8001)"]:::firstparty

            Ollama["Ollama Server (Local LLM Runtime)\nPrereq: must be running\nModel: llama3.1"]:::localinfra
        end

        OpenAICloud["OpenAI API (SaaS)"]:::saas
    end

    User -->|"uses"| Browser
    Browser <-->|"UI_rendering + chat_input"| StreamlitProc

    StreamlitProc -->|"HTTP_POST_/generate-recipe {ingredients[], mode}"| FastAPIProc
    FastAPIProc -->|"HTTP_200 Recipe_JSON {name, ingredients, steps, prep_time_minutes, calories}"| StreamlitProc

    FastAPIProc -->|"mode=local: LangChain_ChatOllama prompt → completion"| Ollama
    FastAPIProc -->|"mode=openai: LangChain_ChatOpenAI prompt → completion (needs OPENAI_API_KEY)"| OpenAICloud

    %% Notes
    KeyNote["Secret Boundary: OPENAI_API_KEY\nstored only in FastAPI process env\n(not in Browser/Streamlit UI)"]:::note
    FastAPIProc -.->|"env_var"| KeyNote

    %% =========================
    %% View B: Component View (FastAPI internals)
    %% =========================
    subgraph "FastAPI Component View (src/api.py orchestrates pipeline)"
        direction TB

        subgraph "HTTP Layer"
            direction TB
            RouteGen["Route: POST /generate-recipe"]:::api
            RouteHealth["Route: GET /health"]:::api
        end

        subgraph "Orchestration Pipeline"
            direction TB
            Orchestrator["LLM Orchestrator / Mode Router\n(mode=local vs mode=openai)"]:::component
            PromptBuilder["Prompt Builder\n(Chef persona + few-shot examples)"]:::component
            ProviderIF["LLM Provider Interface"]:::component
        end

        subgraph "Provider Implementations"
            direction TB
            OpenAIAdapter["OpenAI Adapter\nChatOpenAI + with_structured_output(Recipe)"]:::component
            OllamaAdapter["Ollama Adapter\nChatOllama + text→JSON extraction/parsing"]:::component
        end

        subgraph "Contract + Validation"
            direction TB
            RecipeSchema["Recipe Schema (Pydantic)\n{name, ingredients, steps, prep_time_minutes, calories}"]:::contract
            ErrorHandling["Error Handling / Error Mapper\n(missing/invalid OPENAI_API_KEY etc.)"]:::component
        end

        RouteGen -->|"parse_request {ingredients[], mode}"| Orchestrator
        RouteHealth -->|"readiness/liveness"| Orchestrator

        Orchestrator -->|"build_prompt"| PromptBuilder
        PromptBuilder -->|"uses"| ProviderIF

        ProviderIF -->|"mode=openai"| OpenAIAdapter
        ProviderIF -->|"mode=local"| OllamaAdapter

        OpenAIAdapter -->|"typed_structured_output"| RecipeSchema
        OllamaAdapter -->|"json_parse_then_validate"| RecipeSchema

        Orchestrator -->|"on_error"| ErrorHandling
        ErrorHandling -->|"HTTP_error_response"| RouteGen

        RecipeSchema -->|"return_Recipe_JSON"| RouteGen
    end

    %% Link container FastAPI to its component view
    FastAPIProc -.->|"implements"| RouteGen
    FastAPIProc -.->|"implements"| RouteHealth

    %% Prompt builder uses shared assets
    PromptAssets["Prompting Assets (src/utils.py)\nChef persona + few-shot dataset"]:::firstparty
    PromptBuilder -->|"load_examples + persona"| PromptAssets

    %% Shared model used by API and UI assumptions
    SharedModel["Shared Domain Model (src/models.py)\nRecipe contract"]:::contract
    RecipeSchema -.->|"defined_in"| SharedModel
    StreamlitProc -.->|"renders_assuming_contract"| SharedModel

    %% =========================
    %% View C: Sequence (behavior)
    %% =========================
    subgraph "Sequence — Generate Recipe (alt: local vs openai)"
        direction TB

        S1["1) User enters ingredients in Streamlit chat"]:::step
        S2["2) Streamlit → FastAPI\nPOST /generate-recipe {ingredients[], mode}"]:::step
        S3["3) FastAPI builds prompt\n(Chef persona + few-shot)"]:::step
        ALT["4) alt mode=openai vs mode=local"]:::alt
        S4A["4a) mode=openai: ChatOpenAI\nwith_structured_output(Recipe)"]:::step
        S4B["4b) mode=local: ChatOllama\ntext→JSON extract/parse"]:::step
        S5["5) Validate to Recipe (Pydantic)"]:::step
        S6["6) FastAPI → Streamlit\n200 Recipe JSON"]:::step
        S7["7) Streamlit renders\nrecipe + nutrition sidebar"]:::step
        SFail["Failure: openai mode + missing/invalid key\nFastAPI returns error → Streamlit suggests Local mode"]:::warn

        S1 --> S2 --> S3 --> ALT
        ALT -->|"mode=openai"| S4A --> S5
        ALT -->|"mode=local"| S4B --> S5
        S5 --> S6 --> S7
        S4A -.->|"if_key_missing/invalid"| SFail
    end

    %% =========================
    %% Ops / Repo Artifacts
    %% =========================
    subgraph "Operational Entrypoints + Docs"
        direction TB
        Req["requirements.txt\n(FastAPI, Streamlit, LangChain, Pydantic, Uvicorn, etc.)"]:::deps
        RunApiBat["run_api.bat"]:::ops
        RunApiPs1["run_api.ps1"]:::ops
        RunBotBat["run_bot.bat"]:::ops
        RunStreamlitPs1["run_streamlit.ps1"]:::ops
        RunDoc["RUN_COMMANDS.md\n(commands + ports)"]:::doc
        Readme["README.md\n(modes, endpoints, behavior)"]:::doc

        Req -->|"installs_runtime_deps"| StreamlitProc
        Req -->|"installs_runtime_deps"| FastAPIProc

        RunApiBat -->|"starts"| FastAPIProc
        RunApiPs1 -->|"starts"| FastAPIProc
        RunBotBat -->|"starts"| StreamlitProc
        RunStreamlitPs1 -->|"starts"| StreamlitProc

        RunDoc -.->|"documents"| RunApiBat
        RunDoc -.->|"documents"| RunBotBat
        Readme -.->|"describes"| FastAPIProc
        Readme -.->|"describes"| StreamlitProc
    end

    %% =========================
    %% Click Events (from mapping)
    %% =========================
    click StreamlitProc "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/bot.py"
    click S1 "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/bot.py"
    click S2 "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/bot.py"
    click S7 "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/bot.py"

    click FastAPIProc "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click RouteGen "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click RouteHealth "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click Orchestrator "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click OpenAIAdapter "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click OllamaAdapter "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"
    click ErrorHandling "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/api.py"

    click SharedModel "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/models.py"
    click RecipeSchema "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/models.py"

    click PromptAssets "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/utils.py"
    click PromptBuilder "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/src/utils.py"

    click Req "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/requirements.txt"
    click RunApiBat "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/run_api.bat"
    click RunApiPs1 "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/run_api.ps1"
    click RunBotBat "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/run_bot.bat"
    click RunStreamlitPs1 "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/run_streamlit.ps1"
    click RunDoc "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/RUN_COMMANDS.md"
    click Readme "https://github.com/kashinathj/local-llm-and-chatbot/blob/main/README.md"

    %% =========================
    %% Styles
    %% =========================
    classDef person fill:#F3F4F6,stroke:#374151,color:#111827,stroke-width:1px
    classDef client fill:#E5E7EB,stroke:#4B5563,color:#111827,stroke-width:1px
    classDef firstparty fill:#DBEAFE,stroke:#1D4ED8,color:#0B1B3A,stroke-width:1.2px
    classDef api fill:#BFDBFE,stroke:#1E40AF,color:#0B1B3A,stroke-width:1.2px
    classDef component fill:#C7D2FE,stroke:#3730A3,color:#111827,stroke-width:1.1px
    classDef contract fill:#A7F3D0,stroke:#065F46,color:#064E3B,stroke-width:1.2px
    classDef localinfra fill:#DCFCE7,stroke:#166534,color:#052E16,stroke-width:1.2px
    classDef saas fill:#FED7AA,stroke:#9A3412,color:#7C2D12,stroke-width:1.2px
    classDef note fill:#FEF9C3,stroke:#A16207,color:#713F12,stroke-dasharray: 4 3
    classDef deps fill:#E5E7EB,stroke:#6B7280,color:#111827,stroke-width:1px
    classDef ops fill:#E0E7FF,stroke:#4338CA,color:#111827,stroke-width:1px
    classDef doc fill:#FCE7F3,stroke:#9D174D,color:#831843,stroke-width:1px
    classDef step fill:#FFFFFF,stroke:#6B7280,color:#111827,stroke-width:1px
    classDef alt fill:#F3F4F6,stroke:#374151,color:#111827,stroke-dasharray: 6 4
    classDef warn fill:#FFE4E6,stroke:#9F1239,color:#881337,stroke-width:1.2px
```
