How microsoft/sample-app-aoai-chatgpt works
1. Overall shape
- This is a two-tier sample app: a Python backend exposes chat/API endpoints, and a React/TypeScript frontend provides the UI.
- The app is designed around Azure OpenAI, with multiple optional “data source” backends for retrieval-augmented chat.
2. Backend entrypoint and runtime
- app.py is the backend entrypoint, served in production through gunicorn.conf.py and python3 -m gunicorn app:app.
- The backend is the integration boundary for Azure OpenAI calls, auth checks, data-source querying, chat-history persistence, and security-related request handling.
3. Frontend presentation layer
- frontend/ is a Vite-based React SPA written in TypeScript.
- Main UI areas are the chat page, answer rendering, question input, and chat history panel under frontend/src/pages and frontend/src/components.
- frontend/src/state holds shared app state, while frontend/src/api centralizes client calls to the backend.
4. Static asset build pipeline
- Frontend build output is copied into static/, which is what the backend serves in deployed environments.
- This repo therefore has a clear source/build split: source lives in frontend/, deployable browser assets live in static/.
- Custom logos/favicon assets are expected to be placed in frontend/public and end up in static/assets after build.
5. Primary data flow: user chat to model response
- The user submits a question in the frontend.
- The frontend sends the request to the Python backend.
- The backend forwards the prompt to Azure OpenAI, optionally including retrieved context from a configured data source and/or prior chat history.
- The generated response, citations, and metadata come back to the frontend for rendering.
6. Retrieval/data-source boundary
- The app supports several mutually exclusive retrieval backends selected by environment variables:
- Azure AI Search
- Azure Cosmos DB Mongo vCore
- Elasticsearch
- Pinecone
- MongoDB
- Azure SQL Server
- Promptflow endpoint
- The retrieval layer is a major architectural boundary: the chat UI and core orchestration stay the same, while the backend swaps in different query/connectors depending on DATASOURCE_TYPE or related flags.
7. Chat history persistence
- When enabled, chat history is stored separately in Azure Cosmos DB using the AZURE_COSMOSDB_* settings.
- This is distinct from the document/retrieval data source: one Cosmos DB path is for conversation history, another may be used for document retrieval depending on configuration.
- The chat history UI in frontend/src/components/ChatHistory reflects this persisted conversation state.
8. Authentication and access control
- Authentication is optional but treated as important for production; the README notes chat can be blocked if no identity provider is configured.
- backend/auth contains auth helpers and sample user logic, indicating the backend can enforce or simulate user identity.
- Access control is also tied to data retrieval, including optional group-based filtering via fields like AZURE_SEARCH_PERMITTED_GROUPS_COLUMN.
9. Security and compliance hooks
- backend/security/ms_defender_utils.py suggests integration with Microsoft Defender for Cloud threat protection context.
- SANITIZE_ANSWER and related UI settings indicate a post-processing boundary where model output may be cleaned before display.
- The repo also includes backend/settings.py and shared utils, which likely centralize environment parsing and request/runtime configuration.
10. Offline/data-prep and indexing tooling
- The scripts/ directory is a substantial operational subsystem for preparing documents, chunking, embedding, and pushing content into target stores.
- There are separate prep paths for ACS, Cosmos, Pinecone, and batch indexing, which supports the repo’s multi-datasource architecture.
- data/employee_handbook.pdf is the sample corpus used by those pipelines.
11. Infrastructure and deployment
- infra/ contains Bicep modules for Azure resources: App Service hosting, AI/Cognitive Services, Cosmos DB, Search, Storage, and role assignments.
- infrastructure/deployment.json supports one-click deployment, while azure.yaml and README_azd.md indicate Azure Developer CLI support.
- This means the repo is deployable as a managed Azure application rather than a self-contained local demo only.
12. Validation and test boundaries
- tests/ includes unit tests for settings and utilities plus integration tests for datasource-related behavior.
- Frontend components also have targeted tests, e.g. answer parsing.
- CI workflows in .github/workflows validate static files, test updates, and app builds, reflecting the separation between frontend build artifacts, backend Python code, and infrastructure/configuration.
13. Useful diagram boundaries
- A useful architecture diagram should separate: browser/UI, Python web backend, Azure OpenAI, retrieval datasource(s), Cosmos DB for chat history, and Azure deployment/infrastructure.
- The most important arrows are user input to backend, backend to Azure OpenAI, backend to retrieval store, and backend to chat-history store.
Main entry points: app.py App Entrypoint · datastore.yml Datasource Switch · Chat.tsx Chat UI · main.bicep Infra Stack
Components 23
Frontend
React/TypeScript SPA and browser-facing build
- Chat UI · Main chat page and layout shell
frontend/src/pages/chat/Chat.tsx - Question Input · Prompt entry and submit control
frontend/src/components/QuestionInput/QuestionInput.tsx - Answer View · Renders answers and parsed citations
frontend/src/components/Answer/Answer.tsx - Chat History · Conversation list and history panel
frontend/src/components/ChatHistory/ChatHistoryPanel.tsx - App State · Shared client state and reducer flow
frontend/src/state/AppProvider.tsx - API Client · Centralized browser calls to backend APIs
frontend/src/api/api.ts
Backend
Python web app, auth, history, and security boundaries
- App Entrypoint · Backend application entrypoint
app.py - Settings · Environment parsing and runtime configuration
backend/settings.py - Auth Helpers · Identity and access-control helpers
backend/auth/auth_utils.py - Chat History · Conversation persistence in Cosmos DB
backend/history/cosmosdbservice.py - Defender Hooks · Security and threat-protection integration
backend/security/ms_defender_utils.py - Shared Utils · Common backend helpers and request utilities
backend/utils.py
Retrieval
Optional data-source connectors and document preparation
- Datasource Switch · Selects the active retrieval backend by configuration
scripts/datastore.yml - Azure Search · Search-based document retrieval
scripts/push_to_acs.py - Cosmos vCore · Mongo vCore document retrieval path
scripts/cosmos_mongo_vcore_data_preparation.py - Elasticsearch · Elasticsearch retrieval path
notebooks/AzureOpenAI_OnYourData_Elasticsearch.ipynb - Pinecone · Vector-store retrieval path
scripts/pinecone_data_preparation.py - Doc Prep · Chunking, embedding, and indexing pipelines
scripts/prepdocs.py
Platform
Azure deployment, infrastructure, and runtime assets
- Azure OpenAI · Chat completion and response generation
- History Store · Separate Cosmos DB store for chat transcripts
infra/core/database/cosmos/cosmos-account.bicep - Infra Stack · Azure resource composition and provisioning
infra/main.bicep - App Service · Production hosting target for the backend
infra/core/host/appservice.bicep - Search Service · Azure AI Search resource used by retrieval
infra/core/search/search-services.bicep
Connections 24
- Chat UI → Question Input: uses · Chat page embeds prompt entry
- Chat UI → Answer View: renders · Chat page displays generated answers
- Chat UI → Chat History: shows · Chat page includes conversation history
- Chat UI → App State: reads/writes · Page state is managed through shared app state
- App State → API Client: calls · State actions invoke backend APIs
- API Client → App Entrypoint: requests · Browser sends chat and history requests to backend
- App Entrypoint → Settings: loads · Entry point uses runtime settings
- App Entrypoint → Auth Helpers: checks · Backend validates identity and access
- App Entrypoint → Defender Hooks: applies · Security hooks run on requests and responses
- App Entrypoint → Shared Utils: uses · Shared helpers support runtime behavior
- App Entrypoint → Azure OpenAI: calls · Backend forwards prompts to Azure OpenAI
- App Entrypoint → Datasource Switch: queries · Backend selects and queries the active datasource
- Datasource Switch → Azure Search: can use · Search-backed retrieval option
- Datasource Switch → Cosmos vCore: can use · Cosmos vCore retrieval option
- Datasource Switch → Elasticsearch: can use · Elasticsearch retrieval option
- Datasource Switch → Pinecone: can use · Pinecone retrieval option
- Datasource Switch → Doc Prep: feeds · Prepared documents are indexed for retrieval
- App Entrypoint → Chat History: persists · Conversation state can be written to Cosmos DB
- Chat History → History Store: stores in · Chat transcripts are stored separately from retrieval data
- Infra Stack → App Service: provisions · Deployment stack includes application hosting
- Infra Stack → Search Service: provisions · Deployment stack includes search infrastructure
- Infra Stack → History Store: provisions · Deployment stack includes Cosmos DB storage
- App Service → App Entrypoint: runs · Production service hosts the Python backend
- Search Service → Azure Search: supports · Infrastructure backing for search retrieval