How aws-samples/opensearch-intelligent-search-jp works
Project Type and Purpose
• This is a full-stack AWS-based sample implementation that deploys a Japanese hybrid search system leveraging OpenSearch. It uses CDK for infrastructure provisioning and includes a front-end UI (built with React/Vite) for interacting with the search service.
• The project demonstrates how to combine full-text search (with Sudachi analyzer) and vector search (using Amazon Bedrock’s embedding models) into a searchable system.
Main Components to Capture
a. Infrastructure Provisioning (CDK):
– The CDK package (in packages/cdk) is responsible for setting up AWS resources.
– It contains code for several AWS services (ECS tasks, Lambda functions, Cognito configuration, S3 buckets, and OpenSearch resources).
– Custom CDK constructs (e.g., for API, authentication, configuring OpenSearch, ECS for data ingestion) should be represented as a distinct “CDK Infrastructure Layer.”
b. Data Ingestion and Processing:
– ECS tasks (in packages/cdk/ecs/ingest-data) run containerized Python code that ingests sample data and creates OpenSearch indices.
– Several Lambda functions (delete, list, search documents found in packages/cdk/lambda) provide additional backend functionality.
– Include the direction of data flow: Data is passed from an ingestion process into OpenSearch, with Lambda functions offering read/write operations.
c. OpenSearch Cluster:
– This is the core search engine that supports hybrid search combining full-text and vector search.
– Indicate the use of the Sudachi plugin for text analysis and integration with embedding models from Amazon Bedrock.
d. Frontend UI:
– Located in the packages/ui directory, the UI is built using modern web technologies (React, TypeScript, Vite, Tailwind).
– It interacts with the backend—likely through APIs provided by Lambda functions or directly interfacing with OpenSearch—to display search results.
– Also, it handles user authentication (including Cognito sign-up functionality as indicated by the README).
e. External Services:
– Amazon Bedrock: used for generating embeddings via Titan or Cohere models.
– AWS Cognito: for self sign-up and user management as per project parameters.
– AWS services such as ECS, Lambda, and S3 which are provisioned via CDK.
– Optionally include deployment workflow aspects like the use of “run-ingest-ecs-task.sh” for data ingestion automation.
Relationships and Interactions
• ECS tasks ingest data and populate the search indices in OpenSearch.
• Lambda functions provide operational actions (delete, list, search) interacting with OpenSearch.
• External calls out to Amazon Bedrock for generating embeddings should be highlighted, showing the flow from the backend (or directly as part of the ingestion process) to these external services.
• The AWS CloudFormation/CDK stack acts as the orchestration layer deploying and managing all these resources.
Architectural Patterns & Design Principles
• Use Infrastructure-as-Code principles via AWS CDK.
• The project follows a separation of concerns:
– CDK code manages provisioning; the UI is completely independent and focuses on frontend interactions.
– Backend components (Lambda, ECS) are logically grouped per their functionality (data ingestion vs search operations).
• Emphasis on modular, reusable constructs (as seen in packages/cdk/lib/constructs).
• If applicable, indicate asynchronous processing where ECS tasks and Lambda functions operate independently from the user-triggered UI actions.
Specific Elements
– Clear labels for each component: “Frontend UI,” “CDK Infrastructure,” “ECS Data Ingestion,” “Lambda Functions (Delete/List/Search),” “OpenSearch Cluster,” “Amazon Bedrock Embedding Service,” “AWS Cognito.”
• From the UI to backend (API calls).
• From ECS tasks to OpenSearch (data ingestion).
• From Lambda functions to OpenSearch (read/write operations).
• From backend components to external services (embedding requests to Bedrock).