# vishakhasadhwani/llm-deployment-demo architecture

> Architecture diagram of the GitHub repository vishakhasadhwani/llm-deployment-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vishakhasadhwani/llm-deployment-demo
- Repository: https://github.com/vishakhasadhwani/llm-deployment-demo

## Overview

This repository deploys a private, browser-based chat interface backed by a local vLLM model server. The documented workflow is: a user opens the Streamlit interface, submits a prompt, and receives a streamed model response. The README documents startup ordering and the GPU-backed inference service; the source excerpt shows the UI’s health/model checks and chat API calls. The entrypoint implementation was not sampled, so its startup relationship is supported by the README.

Main entry points: [`app.py`](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py) (Inference API client), [`entrypoint.sh`](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/entrypoint.sh) (Service startup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Chat frontend"]
  node_chat_ui["Streamlit chat UI<br/>[app.py]"]
  node_api_client["Inference API client<br/>[app.py]"]
  node_stream_parser["Stream parser<br/>[app.py]"]
end

subgraph group_inference["Inference runtime"]
  node_vllm["vLLM API server"]
  node_model["HuggingFace model"]
  node_gpu["NVIDIA GPU"]
end

subgraph group_startup["Service startup"]
  node_startup_script["Service startup<br/>[entrypoint.sh]"]
end

node_user(("User"))
node_browser(("Browser"))

node_user -->|"uses"| node_browser
node_browser -->|"opens"| node_chat_ui
node_chat_ui -->|"submits prompts"| node_api_client
node_api_client -->|"checks and requests"| node_vllm
node_vllm -->|"streams responses"| node_api_client
node_api_client -->|"parses stream"| node_stream_parser
node_stream_parser -->|"renders chunks"| node_chat_ui
node_startup_script -->|"starts and waits"| node_vllm
node_vllm -->|"loads"| node_model
node_vllm -->|"runs inference on"| node_gpu

click node_chat_ui "https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py"
click node_api_client "https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py"
click node_stream_parser "https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py"
click node_startup_script "https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/entrypoint.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat_ui,node_api_client,node_stream_parser,node_user,node_browser toneBlue
class node_vllm,node_model,node_gpu toneAmber
class node_startup_script toneMint
```

## Components

### Chat frontend

- [Streamlit chat UI](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py): `app.py`
- [Inference API client](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py): `app.py`
- [Stream parser](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/app.py): `app.py`

### Inference runtime

- **vLLM API server**
- **HuggingFace model**
- **NVIDIA GPU**

### Service startup

- [Service startup](https://github.com/vishakhasadhwani/llm-deployment-demo/blob/main/entrypoint.sh): `entrypoint.sh`

### Other components

- **User**
- **Browser**

## Connections

- User → Browser: uses (evidence: `README.md`)
- Browser → Streamlit chat UI: opens (evidence: `README.md`)
- Streamlit chat UI → Inference API client: submits prompts (evidence: `app.py`)
- Inference API client → vLLM API server: checks and requests (evidence: `app.py`)
- vLLM API server → Inference API client: streams responses (evidence: `app.py`)
- Inference API client → Stream parser: parses stream (evidence: `app.py`)
- Stream parser → Streamlit chat UI: renders chunks (evidence: `app.py`)
- Service startup → vLLM API server: starts and waits (evidence: `README.md`)
- vLLM API server → HuggingFace model: loads (evidence: `README.md`)
- vLLM API server → NVIDIA GPU: runs inference on (evidence: `README.md`)
