How opea-project/genaiexamples works
Project type and purpose
• GenAIExamples is an open-source, microservice-based sample suite showcasing Generative AI use cases (e.g., ChatQnA, DocSum, CodeGen).
• It leverages GenAIComps (collection of LLM, embedding, retrieval, reranking microservices), GenAIInfra (deployment connectors for Docker/Kubernetes), and GenAIEval (benchmarking).
File structure
• Top-level folders: one per sample (ChatQnA, AudioQnA, DocSum, etc.). Each contains:
• Core code (e.g., chatqna.py)
• docker_compose/ (per-hardware-compose files)
• kubernetes/ (Helm charts + GMC manifests)
• ui/ (React, Svelte, Vue, or Gradio frontends)
• tests/, benchmark/, docker_image_build/
• Root contains deployment scripts (deploy.py, benchmark.py), Docker image list, README, and Python requirements.
README for insights
• Describes architecture: GenAIComps microservices + GenAIInfra deployment + GenAIEval benchmarking
• Highlights deployment options: Python startup, Docker Compose, Kubernetes (Helm/GMC)
• Lists use cases and points to sample folders for detailed instructions.
Main components
a. Core microservices (from GenAIComps):
• LLM Service
• Embedding Service
• Retrieval Service
• Reranking Service
• (Optional) Fine-tuning, Summarization, Translator, etc.
b. Pipeline orchestrator
• In-process orchestrator or external GMC connector
c. API Gateway / Load Balancer
• Exposes a unified REST/WebSocket API to clients
d. Frontends (per sample UI)
• React/Svelte/Vue/Gradio clients
e. Data stores & external systems
• Vector DB (Milvus, Qdrant, Pinecone)
• Relational DB or Redis (for session state or caching)
• Object storage (for assets, audio/video)
f. Deployment & infrastructure
• Docker Compose network or Kubernetes cluster
• Helm charts or GMC microservices connector
• CI/CD workflows (GitHub Actions)
Relationships and interactions (data flow)
1. User → Frontend UI (React/Svelte/Gradio)
2. Frontend → API Gateway / Orchestrator
3. Orchestrator dispatches to:
• Retrieval Service (query vector store)
• LLM Service (generate response)
• Optional Reranking Service
4. Microservices read/write to vector DB, cache in Redis, store logs/metrics
5. Response flows back through orchestrator → Frontend → User
6. Benchmarking/monitoring via GenAIEval (metrics exported to Prometheus/Grafana)
Architectural patterns & design principles
• Microservices: each GenAI capability is its own Docker image + service
• Polyglot UI: multiple frontends can coexist behind the same API
• Infrastructure as code: Docker Compose + Helm charts + GMC manifests
• Clear separation of concerns: compute services (LLM, embedding) vs. storage services (vector DB) vs. UI
• Horizontal scalability: services can be replicated in Kubernetes
Elements
• Rectangles (microservices)
• Hexagons (API gateway)
• Cloud outline (Kubernetes cluster) vs. local Docker Compose block