# googlecloudplatform/terraform-genai-knowledge-base architecture

> Architecture diagram of the GitHub repository googlecloudplatform/terraform-genai-knowledge-base, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/googlecloudplatform/terraform-genai-knowledge-base
- Repository: https://github.com/googlecloudplatform/terraform-genai-knowledge-base

## Overview

This Terraform-deployed knowledge-base solution turns uploaded documents into validated fine-tuning data and a deployed model. The main runtime path is document upload → event-driven webhook → OCR, indexing, and question/answer generation → Firestore-backed dataset export. README-described fine-tuning, human validation, model deployment, and prediction capabilities are included; their internal implementation details are not established by the sampled source.

Main entry points: [`webhook/main.py`](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py) (Event webhook), [`mim`](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/tree/main/mim) (Model fine-tuning), [`main.tf`](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/main.tf) (Cloud resources), [`notebooks/model-predictions.ipynb`](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/notebooks/model-predictions.ipynb) (Model predictions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_deploy["Cloud deployment"]
  node_terraform["Cloud resources<br/>[main.tf]"]
end

subgraph group_ingest["Document processing"]
  node_webhook["Event webhook<br/>[main.py]"]
  node_ocr["Document OCR<br/>[main.py]"]
  node_page_text["Page text<br/>[main.py]"]
  node_embeddings["Page embeddings<br/>[main.py]"]
  node_vector_index[("Vector Search")]
  node_qa_generation["QA generation<br/>[main.py]"]
  node_firestore[("Firestore records")]
end

subgraph group_training["Training and inference"]
  node_dataset_export["Dataset export<br/>[main.py]"]
  node_dataset_file["Tuning dataset"]
  node_human_review(("Human validation"))
  node_fine_tuning["Model fine-tuning"]
  node_model_registry[("Model Registry")]
  node_predictions["Model predictions"]
end

node_uploader(("Document uploader"))

node_terraform -.->|"deploys"| node_webhook
node_uploader -->|"triggers"| node_webhook
node_webhook -->|"requests OCR"| node_ocr
node_ocr -->|"returns text"| node_page_text
node_webhook -->|"requests embeddings"| node_embeddings
node_webhook -->|"indexes pages"| node_vector_index
node_webhook -->|"generates Q&amp;A"| node_qa_generation
node_webhook -->|"stores records"| node_firestore
node_webhook -->|"exports dataset"| node_dataset_export
node_dataset_export -->|"reads records"| node_firestore
node_dataset_export -->|"writes JSONL"| node_dataset_file
node_dataset_file -->|"supplies data"| node_fine_tuning
node_human_review -->|"validates data"| node_fine_tuning
node_fine_tuning -->|"deploys model"| node_model_registry
node_model_registry -.->|"serves predictions"| node_predictions

click node_terraform "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/main.tf"
click node_webhook "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_ocr "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_page_text "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_embeddings "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_qa_generation "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_dataset_export "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py"
click node_fine_tuning "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/tree/main/mim"
click node_predictions "https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/notebooks/model-predictions.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_terraform toneBlue
class node_webhook,node_ocr,node_page_text,node_embeddings,node_vector_index,node_qa_generation,node_firestore toneAmber
class node_dataset_export,node_dataset_file,node_human_review,node_fine_tuning,node_model_registry,node_predictions toneMint
class node_uploader toneIndigo
```

## Components

### Cloud deployment

- [Cloud resources](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/main.tf): `main.tf`

### Document processing

- [Event webhook](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- [Document OCR](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- [Page text](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- [Page embeddings](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- **Vector Search**
- [QA generation](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- **Firestore records**

### Training and inference

- [Dataset export](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/webhook/main.py): `webhook/main.py`
- **Tuning dataset**
- **Human validation**
- [Model fine-tuning](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/tree/main/mim): `mim`
- **Model Registry**
- [Model predictions](https://github.com/googlecloudplatform/terraform-genai-knowledge-base/blob/main/notebooks/model-predictions.ipynb): `notebooks/model-predictions.ipynb`

### Other components

- **Document uploader**

## Connections

- Cloud resources → Event webhook: deploys
- Document uploader → Event webhook: triggers (evidence: `README.md`)
- Event webhook → Document OCR: requests OCR (evidence: `webhook/main.py`)
- Document OCR → Page text: returns text (evidence: `webhook/main.py`)
- Event webhook → Page embeddings: requests embeddings (evidence: `webhook/main.py`)
- Event webhook → Vector Search: indexes pages (evidence: `webhook/main.py`)
- Event webhook → QA generation: generates Q&A (evidence: `webhook/main.py`)
- Event webhook → Firestore records: stores records (evidence: `webhook/main.py`)
- Event webhook → Dataset export: exports dataset (evidence: `webhook/main.py`)
- Dataset export → Firestore records: reads records (evidence: `webhook/main.py`)
- Dataset export → Tuning dataset: writes JSONL (evidence: `webhook/main.py`)
- Tuning dataset → Model fine-tuning: supplies data (evidence: `README.md`)
- Human validation → Model fine-tuning: validates data (evidence: `README.md`)
- Model fine-tuning → Model Registry: deploys model (evidence: `README.md`)
- Model Registry → Model predictions: serves predictions
