How azure-samples/ai-knowledge works
Project type and purpose
• This is a full-stack, desktop-style application (user runs it locally via Streamlit) with a Python/FastAPI backend.
• Purpose: provide an interface to deploy and orchestrate disconnected Azure AI service containers (OCR, summarization), integrate with Ollama LLMs and embeddings, perform RAG workflows (vectorize documents, query, summarize), all on the user’s machine.
File structure
• Top level:
• backend/ – FastAPI service entry point (main.py) and helper modules for interacting with Ollama, Azure Document Intelligence, language services.
• frontend/ – Streamlit app (app.py), AI provider abstractions (aiproviders/), UI components (ui/components/), state/UI coordination services (ui/services/), styling.
• tests/ – unit tests for Ollama service.
• requirements.txt, README.md, Docker setup expected externally (docker-compose described in README).
README for technical stack and architecture clues
• Built With: Python 3.12+, FastAPI, Streamlit, Docker containerized Azure Cognitive Services, Ollama, ChromaDB (vector store).
• Usage:
• Spin up two Azure containers (form-recognizer & textanalytics summarization).
• Pull Ollama models (LLM + embedding).
• Run FastAPI on port 8000 to expose endpoints for model listing, document processing, Q&A, summarization.
• Run Streamlit on port 8501 to render UI and call backend endpoints.
• .env holds container endpoints & keys.
• Core workflow:
User uploads document → chunk & embed → store in vector DB → process query via LLM + vector retrieval → display results.
Main components and interactions
a. Frontend (Streamlit)
• UI Components (chat_interface, document_viewer, header, question_suggestions)
• State Manager & UI Coordinator services for event flow
• Calls backend REST API (FastAPI) for:
• /get_models, /process_document, /ask_question, /summarize endpoints
b. Backend (FastAPI)
• Controllers (main.py) route incoming requests
• Helpers in backend/helpers:
• ollama_helper.py – calls local Ollama daemon for embeddings & completion
• doc_helper.py – splits PDFs/text into chunks
• language_helper.py – wraps Azure Text Analytics container calls for summarization
• Vector Store (ChromaDB) abstraction under frontend/aiproviders/vector_store.py (persist and query embeddings)
• Ollama Service abstraction under frontend/aiproviders/ollama_service.py
• Other provider abstractions in frontend/aiproviders/…
c. External Services / Containers
• Azure Form Recognizer container (OCR/document parsing)
• Azure Text Analytics Summarization container
• Ollama LLM daemon (running locally, exposes HTTP gRPC)
d. Data stores
• Local folder for storing models (C:/ExtractiveModel)
• ChromaDB (file-based) for vector embeddings
e. User’s machine hosts all components; Docker network “ocrvnet” for inter-container communication.
Architectural patterns and principles
• Separation of concerns: UI / orchestration (Streamlit) vs. business logic & service integration (FastAPI) vs. AI providers (helper layers) vs. persistence (vector store).
• Adapter pattern: aiproviders/ modules abstract away different AI backends (Azure containers, Ollama).
• Pipeline pattern: document ingestion → chunking → embedding → storage → retrieval → summarization/Q&A.
• User (actor)
• AI Provider Adapters (cluster): Ollama Service, Azure OCR, Azure Summarization
1. User → Streamlit UI → emits HTTP requests to FastAPI
2. FastAPI → Doc Helper → Azure OCR container (HTTP) → parsed text
3. FastAPI → Doc Helper → chunking → Ollama embedding adapter → ChromaDB (write)
4. FastAPI → Query endpoint → ChromaDB (read) → retrieved chunks → Ollama LLM adapter (completion)
5. FastAPI → Azure Summarization container → returns summary
6. Responses flow back to Streamlit → UI renders results to User
c. External container network: group OCR & Text Analytics under a “Docker Bridge Network (ocrvnet)” boundary.
d. Ports: 5000 (OCR), 5001 (Summarization), 11434 (Ollama default), 8000 (FastAPI), 8501 (Streamlit).