How googlecloudplatform/spanner-migration-tool works
Project Type & Purpose
• This is a full-stack, multi-module migration tool written in Go, with both a CLI and a browser-based UI (Angular).
• Purpose: ingest schema/data from various sources (MySQL, PostgreSQL, Oracle, SQL Server, DynamoDB, CSV, Cassandra), analyze (“assessment”), convert to Spanner schema & data, support minimal-downtime streaming migration, and expose an interactive UI.
Top-Level Components
a. User Interfaces
• CLI (main.go + cmd/) → user drives migration via terminal commands.
• UI (ui/ folder) → Angular SPA served during development; production UI served via separate web server.
b. Web API
• webv2/ → Go HTTP server exposing REST endpoints that back the Angular UI.
c. Core Engine
• assessment/ → code analyzers, embedding generation, report generation.
• conversion/ → schema and data transformation logic.
• streaming/ & reverse_replication/ → minimal downtime and change-data-capture pipelines.
• dao/ & accessors/ → abstracted clients for Spanner, Dataflow, Datastream, Storage, ResourceManager.
• sources/ → connectors for each source type (MySQL, PostgreSQL, Oracle, SQL Server, DynamoDB, CSV, Cassandra).
d. Shared Utilities
• common/, internal/, logger/, expressions_api/, import_data/, profiles/, performance/ → helpers, logging, additional tools.
External Dependencies & Services
• Source Databases / Dump Files
(MySQL, PostgreSQL, Oracle, SQL Server, DynamoDB, Cassandra, CSV files)
• Google Cloud Services
• Cloud Spanner
• Cloud Dataflow
• Cloud Datastream
• Google Cloud Storage
• Cloud Resource Manager
• Pub/Sub notifications (for streaming)