How retkowsky/azure-openai-demos works
Project Type and Purpose
– This repository is an open-source demo toolkit (“Azure AI Foundry GenAI demos”) comprised of Jupyter Notebooks illustrating dozens of Azure AI scenarios: OpenAI completions & embeddings, RAG patterns with Azure Cognitive Search, Document Intelligence, Speech Services, Computer Vision, multi-agent orchestration, SORA integrations, etc.
– It is not a monolithic application but a collection of modular demos that each read data (PDFs, audio, video, CSVs), load environment settings (azure.env), call one or more Azure services via REST/SDK, and write results locally.
File Structure Analysis
– Top-level: one folder per demo use case (e.g., “AI podcast generation”, “Agentic RAG”, “Autogen”, “sora”, “Neo4j”, etc.).
– Each demo folder contains:
• A Jupyter Notebook (.ipynb) implementing the scenario
• An azure.env file storing connection strings/keys for Azure services
• Example input assets (documents/, audio/, video/, data/)
• Optional output subfolders (outputs/, results/)
– A single README.md at the root provides an index of demos, feature descriptions, links, and new-content announcements.
Key Insights from README
– The README emphasizes that these are Azure AI Foundry / OpenAI Studio demos, organized by scenario, and uses services such as:
• Azure OpenAI (GPT-4, GPT-4o, GPT-4 Turbo, o-mini, o1, o3, phi-3 vision)
• Azure Cognitive Search (vector store)
• Azure Document Intelligence (Form Recognizer)
• Azure Speech Services (Speech-to-Text, Text-to-Speech, Whisper)
• Azure Computer Vision
• Azure AI Agent Service, AutoGen, SORA integrations
a. Main Components
• Developer Environment: Jupyter Notebook host (local or online) + VS Code + azure.env files
• GitHub Repository: version control of notebooks, configs, assets
• Azure AI Foundry / OpenAI Studio UI (portal for monitoring & management)
• Azure Services:
– Azure OpenAI Service (completions, chat, embeddings, fine-tuning)
– Azure Cognitive Search (indexing, vector search)
– Azure Document Intelligence Service
– Azure Speech Services (Transcription & Synthesis)
– Azure Computer Vision Service
– Azure AI Agent Service / AutoGen
– SORA integration service
• External Data Assets: PDFs, images, audio, video, CSVs in each demo folder
• Local Output Storage: outputs/, results/, generated media files
b. Relationships and Interactions
• Developer Environment loads azure.env → initializes Azure credentials
• Jupyter Notebook code → calls Azure OpenAI SDK/REST → OpenAI endpoints
• Notebooks optionally call:
– Azure Cognitive Search for retrieval (RAG patterns)
– Document Intelligence for form/document parsing
– Speech Services for transcription/synthesis
– Computer Vision for image analysis
– AI Agent Service or AutoGen orchestration endpoints
– SORA extension APIs for video→video or image→video transforms
• Azure services return JSON/media → Notebook writes to outputs/ local folder
c. Architectural Patterns
• Modular “demo per folder” pattern: each is a self-contained pipeline
• RAG pattern: Document Intelligence or Cognitive Search feed LLM
• Agent orchestration: Azure AI Agent Service / AutoGen multi-agent flows
• SDK/REST-based integration: environment-configured clients invoke Azure endpoints
d. Key Technologies
• Python & Jupyter
• Azure SDKs (azure-ai-openai, azure-search-documents, azure-ai-formrecognizer, azure-ai-speech, azure-ai-vision)
• Environment variable management (azure.env)
• GitHub for source control
• Azure AI Foundry / Studio for service provisioning & monitoring
Detail Balance
– Capture every critical integration (each Azure API called) but avoid over-splitting notebooks themselves.
– Represent azure.env as a single “Credentials & Configuration” component supplying secrets to all demo pipelines.
– Optionally, annotate common code patterns (e.g., “load env → init client → call endpoint → parse response”).