How joon-solutions/looker-explore-assistant works
Project type and purpose
• This is a full-stack Looker Extension / API plugin that lets end users write natural-language prompts to generate Looker Explore queries.
• Frontend: a React + TypeScript single‐page application embedded in Looker via the Looker Extension SDK.
• Backend: Google Cloud–hosted services (Cloud Functions, Cloud Run) managed by Terraform, integrating with Vertex AI and BigQuery/Cloud SQL.
• Supporting tooling: example-generation scripts to populate BigQuery with prompt/response pairs, and a training notebook.
File structure
Top-level directories:
• explore-assistant-extension: React/TS front end, Redux slices, Webpack/Tailwind config, Looker Extension manifest
• explore-assistant-cloud-run: Python service (main.py) containerized for Cloud Run
• explore-assistant-backend: Terraform code to provision GCP resources (BigQuery datasets, Cloud Functions, Cloud Run, Cloud SQL)
• explore-assistant-examples: scripts to generate and load example prompt/response data into BigQuery
• explore-assistant-training: notebook and data for one-off model-training example generation
Key config files: Dockerfile and docker-compose.yml for cloud-run service, requirements.txt for Python dependencies, Terraform .tf files under terraform/*.
README for additional insights
• The extension queries Vertex AI to translate human text into a Looker Explore URL.
• User history is kept in browser localStorage; structured logs (input/output tokens) are streamed to BigQuery.
• BigQuery holds both example prompt/response pairs and the analytics logs.
• Deployment leverages Terraform (GCS backend, BigQuery, Cloud Functions, Cloud Run, Cloud SQL).
Main components & their interactions
a. User & Looker Client
• The user interacts with the Looker UI. The extension is loaded into Looker via the Embed/Extension SDK.
b. Frontend Extension
• React app (explore-assistant-extension) handles user input, history, and UI rendering.
• On prompt submission it calls the Backend API endpoint.
c. Backend API
• Cloud Run service (container: explore-assistant-cloud-run/main.py) or Cloud Function (terraform/cloud_function) exposes HTTP endpoints.
• Fetches example prompts from BigQuery and potentially Cloud SQL (for schema and state).
• Composes a one-shot prompt, calls Vertex AI REST API, receives generated Explore query.
• Logs request/response metadata (token counts, timestamps) into BigQuery or Cloud SQL.
d. Vertex AI
• Receives the composed prompt, returns the LLM output.
e. Data Stores
• BigQuery – host of example prompt/response pairs, log sink for structured logs.
• Cloud SQL – optional relational store for internal schema or other state.
f. Deployment & Provisioning
• Terraform modules under explore-assistant-backend/terraform create GCP resources.
• Cloud Storage backend for Terraform state.
Architectural patterns & design principles
• Clear separation of concerns: UI, API/service layer, data stores.
• Serverless compute (Cloud Run + Cloud Functions) for auto-scaling.
• Infrastructure as Code (Terraform) for reproducible deployments.
• Eventual consistency between logs and BigQuery sinks.
• One-shot prompt technique (no dedicated vector store for embeddings).
Essential elements
• Frontend: React/TS code, localStorage history, Looker Extension SDK
• Backend: Cloud Run and/or Cloud Function code, Python entrypoint, Dockerfile
• ML: Vertex AI service endpoint
• Data: BigQuery (examples + logs), Cloud SQL schema
• Infra as Code: Terraform modules, state backend