# udaythakare/migration-tool architecture

> Architecture diagram of the GitHub repository udaythakare/migration-tool, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/udaythakare/migration-tool
- Repository: https://github.com/udaythakare/migration-tool

## Overview

1. **Two-tier architecture**
   - The repo is split into a **Next.js frontend** under `frontend/` and a **Python migration engine** under `server/`.
   - The frontend is the operator console; the Python side performs schema analysis, mapping, and data movement.

2. **Frontend: interactive migration workspace**
   - `frontend/app/page.js` is the main dashboard entry point.
   - It handles schema import, mapping generation, workspace URL state, and local relationship parsing for table graphs.
   - `TableMappingCard.js` and `TableMappingDetail.js` are the main presentation/editing components for table-level and column-level mapping review.

3. **Frontend API layer as a control boundary**
   - `frontend/app/api/...` acts as the bridge between the UI and backend engine.
   - The routes are organized by domain:
     - `api/engine/[action]/route.js` for invoking migration engine actions
     - `api/mappings/route.js` for saving/loading mapping state
     - `api/projects/route.js` for project/workspace management
   - This layer keeps the browser UI decoupled from direct Python execution.

4. **Python engine orchestration**
   - `server/complete_migration.py` is the top-level orchestrator for running a migration end to end.
   - It likely coordinates extraction, mapping, ID remapping, transformation, and execution order.
   - `server/main.py` is the central service entrypoint for backend APIs or command-driven execution.

5. **Schema extraction and parsing**
   - `server/mysql_extractor.py` handles live source schema extraction from MySQL.
   - `server/data_parser.py` and `server/data_transformers.py` normalize incoming table/column data and prepare it for migration.
   - `server/parse_legacy_schema` is also described in the README as the upload path for `.sql` schema files used by the UI.

6. **Mapping engine: deterministic first, AI-assisted second**
   - `server/hybrid_mapper.py` appears to be the main mapping strategy, combining heuristics with model-based assistance.
   - `server/simple_mapper.py` and `server/ai_schema_mapper.py` suggest a fallback/alternate mapping pipeline.
   - `server/gemini_schema_mapper.py` and `server/groq_schema_mapper.py` represent model-specific schema matching backends.
   - The architecture is centered on producing table/column correspondences between source MySQL and target PostgreSQL schemas.

7. **Dependency ordering and conflict handling**
   - The migration is not a flat copy; it needs a safe execution order.
   - The README and filenames indicate special handling for parent/child table ordering, circular dependencies, and conflict resolution.
   - `server/id_remapper.py` handles primary key and foreign key translation so relational integrity survives the move.

8. **Remote target execution**
   - `server/connect_vps.py` is the remote infrastructure boundary for connecting to a VPS-hosted target database.
   - The README mentions secure remote access and retry/timeout behavior, so the engine is designed to operate against non-local infrastructure.
   - This is an important boundary because migration execution depends on networked database access, not just local files.

9. **Persistent mapping and migration artifacts**
   - Mapping state is stored in JSON artifacts such as `server/mapping.json` and `server/mappings/v1.0.0.json`.
   - The frontend also has `.local-data/migration-store.json`, which suggests local workspace persistence for the UI.
   - These artifacts are the handoff point between planning in the UI and execution in the Python engine.

10. **Operational inputs and test fixtures**
   - The SQL files (`server/lagacy.sql`, `server/migrate.sql`, `server/vps_schema.sql`) look like fixtures or reference schemas used during testing and migration.
   - The `server/test_*.py` files indicate the engine is validated at multiple layers: parsing, grouping, endpoint behavior, AI mapping, and migration flow.
   - `server/cli_map.py` and `server/debug_regex.py` suggest there are command-line and debugging utilities for schema analysis outside the main UI flow.

11. **External services that materially affect the design**
   - The system is built around migration from **MySQL source** to **PostgreSQL target**.
   - AI assistance is provided through **Gemini** and possibly **Groq** for schema interpretation.
   - The runtime stack therefore spans browser UI, Node/Next.js API routes, Python migration services, database connectivity, and AI model calls.

12. **End-to-end data flow**
   - A user loads or connects a source schema in the frontend.
   - The frontend sends the schema/request through its API routes to the Python engine.
   - The engine extracts and parses schemas, computes mappings, optionally consults AI, persists the mapping, and then migrates data in dependency-safe order with ID remapping and conflict handling.
   - Results and mapping state are then surfaced back in the dashboard for review and further edits.

Main entry points: [`server/complete_migration.py`](https://github.com/udaythakare/migration-tool/blob/master/server/complete_migration.py) (Orchestrator), [`frontend/app/page.js`](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/page.js) (Workspace), [`server/hybrid_mapper.py`](https://github.com/udaythakare/migration-tool/blob/master/server/hybrid_mapper.py) (Hybrid Mapper), [`server/main.py`](https://github.com/udaythakare/migration-tool/blob/master/server/main.py) (Backend Entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_frontend_page["Workspace<br/>Next.js dashboard<br/>[page.js]"]
  node_frontend_card["Table Card<br/>UI component"]
  node_frontend_detail["Mapping Detail<br/>UI component"]
  node_frontend_engine_api["Engine API<br/>Next.js route<br/>[route.js]"]
  node_frontend_mappings_api["Mappings API<br/>Next.js route<br/>[route.js]"]
  node_frontend_projects_api["Projects API<br/>Next.js route<br/>[route.js]"]
end

subgraph group_server["Python Engine"]
  node_server_main["Backend Entry<br/>Python service<br/>[main.py]"]
  node_server_complete["Orchestrator<br/>migration runner"]
  node_server_extractor["MySQL Extractor<br/>schema extractor<br/>[mysql_extractor.py]"]
  node_server_parser["Schema Parser<br/>normalizer<br/>[data_parser.py]"]
  node_server_transform["Transformers<br/>data shaping"]
  node_server_hybrid["Hybrid Mapper<br/>mapping engine<br/>[hybrid_mapper.py]"]
  node_server_simple["Simple Mapper<br/>mapping fallback<br/>[simple_mapper.py]"]
  node_server_ai["AI Mapper<br/>schema matching"]
  node_server_id["ID Remapper<br/>relational remapping<br/>[id_remapper.py]"]
  node_server_vps["VPS Connect<br/>remote execution<br/>[connect_vps.py]"]
  node_server_cli["CLI Tools<br/>utility entrypoint<br/>[cli_map.py]"]
end

subgraph group_artifacts["Artifacts"]
  node_frontend_store["Local Store<br/>browser persistence"]
  node_server_mapping_artifact["Mapping JSON<br/>artifact<br/>[mapping.json]"]
  node_server_versioned_mapping["Versioned Maps<br/>artifact<br/>[v1.0.0.json]"]
end

node_frontend_page -->|"renders"| node_frontend_card
node_frontend_page -->|"opens"| node_frontend_detail
node_frontend_page -->|"requests"| node_frontend_engine_api
node_frontend_page -->|"loads/saves"| node_frontend_mappings_api
node_frontend_page -->|"manages"| node_frontend_projects_api
node_frontend_page -->|"persists"| node_frontend_store
node_frontend_mappings_api -->|"forwards"| node_server_main
node_frontend_engine_api -->|"invokes"| node_server_main
node_frontend_projects_api -->|"coordinates"| node_server_main
node_server_main -->|"orchestrates"| node_server_complete
node_server_complete -->|"extracts"| node_server_extractor
node_server_complete -->|"parses"| node_server_parser
node_server_complete -->|"prepares"| node_server_transform
node_server_complete -->|"maps"| node_server_hybrid
node_server_hybrid -->|"falls back"| node_server_simple
node_server_hybrid -->|"assists"| node_server_ai
node_server_complete -->|"remaps"| node_server_id
node_server_complete -->|"executes"| node_server_vps
node_server_complete -->|"writes"| node_server_mapping_artifact
node_server_mapping_artifact -->|"versions"| node_server_versioned_mapping
node_server_cli -->|"uses"| node_server_parser
node_server_cli -->|"uses"| node_server_hybrid

click node_frontend_page "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/page.js"
click node_frontend_card "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/TableMappingCard.js"
click node_frontend_detail "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/TableMappingDetail.js"
click node_frontend_engine_api "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/engine/[action]/route.js"
click node_frontend_mappings_api "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/mappings/route.js"
click node_frontend_projects_api "https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/projects/route.js"
click node_frontend_store "https://github.com/udaythakare/migration-tool/blob/master/frontend/.local-data/migration-store.json"
click node_server_main "https://github.com/udaythakare/migration-tool/blob/master/server/main.py"
click node_server_complete "https://github.com/udaythakare/migration-tool/blob/master/server/complete_migration.py"
click node_server_extractor "https://github.com/udaythakare/migration-tool/blob/master/server/mysql_extractor.py"
click node_server_parser "https://github.com/udaythakare/migration-tool/blob/master/server/data_parser.py"
click node_server_transform "https://github.com/udaythakare/migration-tool/blob/master/server/data_transformers.py"
click node_server_hybrid "https://github.com/udaythakare/migration-tool/blob/master/server/hybrid_mapper.py"
click node_server_simple "https://github.com/udaythakare/migration-tool/blob/master/server/simple_mapper.py"
click node_server_ai "https://github.com/udaythakare/migration-tool/blob/master/server/ai_schema_mapper.py"
click node_server_id "https://github.com/udaythakare/migration-tool/blob/master/server/id_remapper.py"
click node_server_vps "https://github.com/udaythakare/migration-tool/blob/master/server/connect_vps.py"
click node_server_cli "https://github.com/udaythakare/migration-tool/blob/master/server/cli_map.py"
click node_server_mapping_artifact "https://github.com/udaythakare/migration-tool/blob/master/server/mapping.json"
click node_server_versioned_mapping "https://github.com/udaythakare/migration-tool/blob/master/server/mappings/v1.0.0.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_page,node_frontend_card,node_frontend_detail,node_frontend_engine_api,node_frontend_mappings_api,node_frontend_projects_api toneBlue
class node_server_main,node_server_complete,node_server_extractor,node_server_parser,node_server_transform,node_server_hybrid,node_server_simple,node_server_ai,node_server_id,node_server_vps,node_server_cli toneAmber
class node_frontend_store,node_server_mapping_artifact,node_server_versioned_mapping toneMint
```

## Components

### Frontend

Next.js operator console and local workspace state

- [Workspace](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/page.js): Next.js dashboard · Main migration workspace, schema import, and mapping review entrypoint · `frontend/app/page.js`
- [Table Card](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/TableMappingCard.js): UI component · Table-level mapping review and editing · `frontend/app/TableMappingCard.js`
- [Mapping Detail](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/TableMappingDetail.js): UI component · Column-level mapping detail editor · `frontend/app/TableMappingDetail.js`
- [Engine API](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/engine/[action]/route.js): Next.js route · Browser-to-engine command boundary · `frontend/app/api/engine/[action]/route.js`
- [Mappings API](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/mappings/route.js): Next.js route · Loads and saves mapping state for the workspace · `frontend/app/api/mappings/route.js`
- [Projects API](https://github.com/udaythakare/migration-tool/blob/master/frontend/app/api/projects/route.js): Next.js route · Workspace and project management boundary · `frontend/app/api/projects/route.js`

### Python Engine

Schema analysis, mapping, and migration execution

- [Backend Entry](https://github.com/udaythakare/migration-tool/blob/master/server/main.py): Python service · Central backend entrypoint for commands and API execution · `server/main.py`
- [Orchestrator](https://github.com/udaythakare/migration-tool/blob/master/server/complete_migration.py): migration runner · End-to-end migration coordinator · `server/complete_migration.py`
- [MySQL Extractor](https://github.com/udaythakare/migration-tool/blob/master/server/mysql_extractor.py): schema extractor · Live source schema extraction from MySQL · `server/mysql_extractor.py`
- [Schema Parser](https://github.com/udaythakare/migration-tool/blob/master/server/data_parser.py): normalizer · Parses and normalizes incoming table and column data · `server/data_parser.py`
- [Transformers](https://github.com/udaythakare/migration-tool/blob/master/server/data_transformers.py): data shaping · Prepares rows and schema structures for migration · `server/data_transformers.py`
- [Hybrid Mapper](https://github.com/udaythakare/migration-tool/blob/master/server/hybrid_mapper.py): mapping engine · Primary heuristic plus AI-assisted schema matching · `server/hybrid_mapper.py`
- [Simple Mapper](https://github.com/udaythakare/migration-tool/blob/master/server/simple_mapper.py): mapping fallback · Deterministic alternate mapping path · `server/simple_mapper.py`
- [AI Mapper](https://github.com/udaythakare/migration-tool/blob/master/server/ai_schema_mapper.py): schema matching · AI-assisted schema correspondence layer · `server/ai_schema_mapper.py`
- [ID Remapper](https://github.com/udaythakare/migration-tool/blob/master/server/id_remapper.py): relational remapping · Primary and foreign key remapping for relational integrity · `server/id_remapper.py`
- [VPS Connect](https://github.com/udaythakare/migration-tool/blob/master/server/connect_vps.py): remote execution · Remote target database connection and retry boundary · `server/connect_vps.py`
- [CLI Tools](https://github.com/udaythakare/migration-tool/blob/master/server/cli_map.py): utility entrypoint · Command-line schema analysis and mapping helpers · `server/cli_map.py`

### Artifacts

Persistent mapping and schema files used across runs

- [Local Store](https://github.com/udaythakare/migration-tool/blob/master/frontend/.local-data/migration-store.json): browser persistence · Client-side persisted migration workspace state · `frontend/.local-data/migration-store.json`
- [Mapping JSON](https://github.com/udaythakare/migration-tool/blob/master/server/mapping.json): artifact · Persisted mapping state shared across planning and execution · `server/mapping.json`
- [Versioned Maps](https://github.com/udaythakare/migration-tool/blob/master/server/mappings/v1.0.0.json): artifact · Versioned mapping snapshots · `server/mappings/v1.0.0.json`

## Connections

- Workspace → Table Card: renders · Dashboard drives table-level review
- Workspace → Mapping Detail: opens · Dashboard drills into column-level mappings
- Workspace → Engine API: requests · Sends schema analysis and migration actions
- Workspace → Mappings API: loads/saves · Syncs mapping state with persistence
- Workspace → Projects API: manages · Handles workspace/project state
- Workspace → Local Store: persists · Local workspace persistence
- Mappings API → Backend Entry: forwards · API boundary into Python backend
- Engine API → Backend Entry: invokes · Executes backend actions by route parameter
- Projects API → Backend Entry: coordinates · Project-level backend operations
- Backend Entry → Orchestrator: orchestrates · Primary backend entry delegates full migrations
- Orchestrator → MySQL Extractor: extracts · Reads source schema from MySQL
- Orchestrator → Schema Parser: parses · Normalizes schema and input payloads
- Orchestrator → Transformers: prepares · Shapes data for execution
- Orchestrator → Hybrid Mapper: maps · Produces table and column correspondences
- Hybrid Mapper → Simple Mapper: falls back · Uses deterministic mapping when needed
- Hybrid Mapper → AI Mapper: assists · Consults model-backed mapping logic
- Orchestrator → ID Remapper: remaps · Preserves relational integrity during move
- Orchestrator → VPS Connect: executes · Targets remote PostgreSQL infrastructure
- Orchestrator → Mapping JSON: writes · Persists migration plan and resolved mappings
- Mapping JSON → Versioned Maps: versions · Stores snapshot variants of mapping state
- CLI Tools → Schema Parser: uses · CLI helper for schema inspection
- CLI Tools → Hybrid Mapper: uses · CLI helper for mapping experiments
