How looker-open-source/looker-explore-assistant works
Main Components
a. Frontend (Extension):
• Located under the "explore-assistant-extension" folder.
• Built with React/TypeScript, uses Webpack and Tailwind CSS.
• Includes components (Chat UI, AgentPage, various helper utilities, and Looker Embed/Extension SDK integrations).
• Manages user interactions such as natural language input, displaying questions history (stored in local storage), and visualizing responses.
b. Backend API:
• Under "explore-assistant-backend" folder.
• Contains Terraform configuration files to deploy GCP resources (Cloud Functions, BigQuery, GCS backends).
• Provides API endpoints to interact with Vertex AI for LLM-based query generation.
• Handles structured logging, cost tracking, and communicates with Looker’s connection and metadata.
c. Cloud Function:
• Standalone Python-based cloud function in "explore-assistant-cloud-function".
• Implements the logic that handles inquiries and passes them to the LLM (Vertex AI).
• Contains tests along with its requirements file.
d. Example and Training Data:
• Found in "explore-assistant-examples" and "explore-assistant-training" directories.
• Involves notebooks (.ipynb), JSON example files, and scripts to load/update example data.
• These examples help fine-tune the LLM with realistic Looker Explore queries.
e. Infrastructure-as-Code:
• Terraform files that configure GCP backends (GCS, BigQuery, Cloud Functions) in the backend directory.
Relationships and Interactions
• The Frontend extension interacts with Looker via its Extension SDK and also communicates with the Backend API.
• The Backend API (via Cloud Functions) processes natural language inputs, calls the Vertex AI LLM service, and returns the generated Looker query.
• Example components support training and fine-tuning of the LLM by providing representative Looker queries that are later stored (e.g., into BigQuery).
• Terraform scripts ensure that the required GCP infrastructure is deployed and configured correctly.
• The data flow starts from the user’s action on the extension (Chrome or Looker app), then the query is passed to the API (cloud function), which leverages Vertex AI, with results aggregated and possibly logged to BigQuery for cost tracking.
Architectural Patterns/Design Principles
• Separation of Concerns: Clear division between presentation (frontend), business logic/API (backend/cloud function), and infrastructure (Terraform).
• Microservice/Serverless: The backend is deployed as cloud functions on GCP.
• API-driven Communication: The extension communicates with the backend API which in turn calls the AI services.
• Extensibility: The examples and training module allow for continuous improvement and fine tuning of the model.
Relevant Technologies and Frameworks
• Frontend: React, TypeScript, Webpack, Tailwind CSS, Looker Extension/Embed SDK.
• Backend: Google Cloud Platform (Cloud Functions, BigQuery, GCS), Vertex AI.
• Infrastructure: Terraform.
• Testing, CI/CD: GitHub workflows in the .github directory.