How opea-project/genaicomps works
Overall architecture
• The project follows a microservice architecture.
• Each microservice (e.g., Embedding, Retriever, ASR, TTS, Dataprep, LLM, Reranking, etc.) is implemented as a containerized component with its own deployment configuration (Docker Compose and Kubernetes manifests are used throughout the “comps” directory).
• There is a ServiceOrchestrator (or “MegaService”) that composes these microservices into a cohesive solution, as shown in the README’s example.
• CI/CD workflows and deployment scripts are extensively set up within the “.github/workflows” area.
Main components
a. Front Door / API Gateway:
– The ChatQnAGateway, which receives incoming requests and directs them to the appropriate microservices.
b. Microservices:
– Embedding Service(s): Providing vector embedding functionalities.
– Retriever and Reranking Services: For retrieval-augmented generation, processing queries using LangChain and LlamaIndex integrations.
– ASR and TTS Services: For speech recognition and synthesis.
– Dataprep: Pre-processing and data ingestion pipelines.
– LLM (Chat and Text generation) Services: Depending on deployment, supporting inference engines like TGI, vLLM or Ray Serve.
– Other specialized components (e.g., agent, animation, chathistory, guardrails) as needed.
c. Service Orchestrator / MegaService:
– This component ties individual microservices together, creating customized workflow flows (e.g., connecting an embedding service to an LLM service).
d. Infrastructure Components:
– Container orchestration platforms (Docker, Kubernetes).
– Configuration management (YAML configuration files, Helm charts in the “kubernetes” directories).
– CI/CD pipelines for builds, tests, and deployments.
– Possible integration with external services (Redis, Qdrant, Milvus, etc.) based on the microservice choices.
e. Testing and Deployment:
– Include test cases and deployment workflows (as seen in the “tests” and “.github/workflows” directories).
Diagramming Relationships and Data Flow
– Name of the microservice (e.g., “Embedding on Gaudi2”, “ASR on Xeon CPU”, “LLM via Ray Serve”).
– Relevant technology or frameworks (e.g., LangChain, LlamaIndex, specific container images).
• API Gateway / ChatQnAGateway as the entry point.
– From the gateway to the microservices.
– From one microservice to another when composing a workflow.
– From microservices back to the gateway for response aggregation.
Architectural Patterns and Technologies
• Modularity and service isolation: Each component is built (and deployed) independently with its own Dockerfiles, Kubernetes manifests, and CI configurations.
• Container orchestration aspects by indicating that deployment artifacts exist in production-ready formats (Docker Compose and Kubernetes).
• Point out the usage of service decorators (such as the “register_microservice” decorator) to abstract microservice registration.
Final Advice
– Review the extensive directory structure in “comps/” to find all individual microservices; you might group them into categories if there are many.