How sp-today/cloud-migration-factory-on-aws works
1. This repository is an AWS migration platform, not a single app. The architecture is built around a CloudFormation-deployed control plane that provisions CloudFront, API Gateway, Lambda, Cognito, S3, DynamoDB, and optional WAF layers. The root template and nested templates in deployment/CFN-templates split the solution into feature-specific stacks.
2. The user-facing layer is a React/Vite frontend in source/frontend. It is packaged as a static web app, served through AWS frontend infrastructure, and configured at runtime by public/env.js. The frontend talks to backend APIs through typed API clients and feature hooks, with route separation for pre-auth and authenticated flows.
3. Authentication and user administration are centralized around Amazon Cognito. Backend Lambdas such as lambda_auth, lambda_login, lambda_user_admin, lambda_cognito_user_update, and lambda_cognito_group_update manage login, password flows, group membership, and permission synchronization. This is the main identity boundary for the whole system.
4. The core application logic lives in Python Lambda functions under source/backend/lambda_functions. These functions implement the migration factory’s domain model: applications, servers, databases, waves, pipelines, tasks, roles, policies, schemas, and notification workflows. The repository is organized as many small domain Lambdas instead of a single monolith.
5. DynamoDB is the primary backend data store for the solution’s metadata and operational state. Lambdas such as lambda_manage_entities, lambda_schema, lambda_items, lambda_waves, lambda_pipeline_stream, and lambda_task_orchestrator read and mutate this data, enforce entity relationships, and coordinate workflow state. This gives the system a metadata-driven architecture.
6. The model is schema-driven. Default schema and policy data live in lambda_defaultschema, and backend helpers under lambda_manage_entities and lambda_layer_items handle validation, relationships, and item-level rules. The frontend mirrors this with schema-aware forms and validation steps so that user edits align with backend constraints.
7. Data import/export is a major flow. The frontend has import wizards, validation pipelines, and spreadsheet parsing utilities; backend Lambdas such as lambda_import_data_from_s3, lambda_upload_data_entities, and lambda_import_export_pipeline_templates process uploaded files, transform them into entities, and persist them. S3 is the staging boundary for bulk data exchange.
8. Remote automation is handled as packaged scripts plus orchestration Lambdas. Under source/integrations/mgn and source/integrations/automation_packages, the repo ships automation scripts for prerequisite checks, agent install, file copy, user management, verification, and post-cutover validation. These are paired with Lambda handlers like lambda_mgn, lambda_mgn_launch, and lambda_mgn_template to drive migration execution through AWS MGN/SSM-style workflows.
9. Systems management and script execution have their own subsystem. The lambda_ssm* family manages SSM jobs, scripts, outputs, and socket-based interactions, while lambda_ssm_load_scripts validates packaged automation content. This creates a separate operational channel for running and monitoring remote commands and jobs.
10. Migration tracker support is another distinct integration path. The source/integrations/migration-tracker Glue scripts and related Lambdas such as lambda_migrationtracker_glue_execute and lambda_migrationtracker_glue_scriptcopy extract migration data for reporting and analytics. This is a reporting/ETL boundary separate from the live operational metadata store.
11. Real-time or session-oriented features are handled by socket/auth Lambdas. lambda_genai_sockets, lambda_genai_sockets_auth, lambda_genai_sockets_handler, and lambda_ssm_socket indicate websocket-style backchannels for interactive experiences, including the repository’s GenAI-oriented helper flow and live operational updates.
12. Shared code is split into Lambda layers and common integration utilities. The layers under source/backend/lambda_layers provide reusable constants, logging, pipeline helpers, policy logic, and item/query utilities; the integration side has source/integrations/common/mfcommon.py for shared factory access helpers. This keeps cross-cutting logic out of individual Lambdas.
13. Notifications and downstream effects are explicit subsystems. lambda_notifications, lambda_email_notification, and lambda_ui_notification propagate status changes back to users, while pipeline and task Lambdas emit execution outputs. The result is an event-driven feel even though the persistence is mostly DynamoDB-centric.
14. Build and packaging are part of the architecture. deployment/build-s3-dist.sh assembles frontend artifacts, Lambda ZIPs, layers, and integration packages into regional and global S3 distributions, which are then referenced by the CloudFormation stacks. The repo is designed to be deployed as a versioned solution bundle rather than run directly from source.
Main entry points: lambda_functions Domain Lambdas · frontend Web console · api_clients API clients · build-s3-dist.sh Build bundle
Components 19
Deployment
CloudFormation stacks and build packaging for the solution bundle.
- Root stack · Top-level CloudFormation template that composes the deployed control plane.
deployment/CFN-templates/aws-cloud-migration-factory-solution.template - Feature stacks · Feature-specific nested templates for automation, MGN, replatform, tracker, WAF, target account, and credential manager.
deployment/CFN-templates - Build bundle · Assembles frontend assets, Lambda ZIPs, layers, and integration packages for distribution.
deployment/build-s3-dist.sh
Frontend
React/Vite console and its client-side UX layers.
- Web console · Static browser application for operators and users.
source/frontend - Routes · Separates pre-authenticated and authenticated application flows.
source/frontend/src/AuthenticatedRoutes.tsx - API clients · Client wrappers for login, admin, user, tools, and GenAI socket APIs.
source/frontend/src/api_clients - Schema UI · Schema-aware forms, validation, and entity editing flows.
source/frontend/src/components/ui_attributes - Import flow · Spreadsheet parsing, validation, and import wizards for data onboarding.
source/frontend/src/components/data-source - Planning tools · Views for waves, pipelines, tasks, and migration planning.
source/frontend/src/components/wave_planning
Backend Lambdas
Python Lambda services for identity, metadata, workflows, and operations.
- Identity · Login, password, user admin, and group synchronization around Amazon Cognito.
source/backend/lambda_functions/lambda_auth/lambda_auth.py - Entity store · Schema, entities, items, relationships, and validation against metadata state.
source/backend/lambda_functions/lambda_manage_entities/lambda_manage_entities.py - Domain Lambdas · Core application Lambdas for apps, servers, databases, waves, policies, roles, tasks, pipelines, and notifications.
source/backend/lambda_functions - Import backend · Transforms uploaded files into entities and persists imported metadata.
source/backend/lambda_functions/lambda_import_data_from_s3/lambda_import_data_from_s3.py - SSM channel · Jobs, scripts, outputs, and socket-style interactions for remote command execution.
source/backend/lambda_functions/lambda_ssm/lambda_ssm.py
Integrations
Packaged automation scripts and external system adapters.
- MGN automation · Launch and template handlers that drive AWS MGN-based automation workflows.
source/integrations/mgn/lambdas/lambda_mgn.py - Tracker ETL · Glue script execution and script-copy handlers for migration tracker reporting.
source/integrations/migration-tracker/GlueScript - Credential mgr · Integration lambdas for creating, reading, updating, listing, and deleting secrets.
source/integrations/credential_manager/lambdas/CredentialManager.py
Shared Code
Reusable layers and common helpers.
- Lambda layers · Reusable logging, constants, pipeline, policy, item, and utility code.
source/backend/lambda_layers - Common utils · Shared factory helper module used by integration packages and lambdas.
source/integrations/common/mfcommon.py
Connections 27
- Root stack → Feature stacks: composes · Root CloudFormation stack assembles the feature-specific nested stacks.
- Build bundle → Root stack: publishes · Build output is deployed and referenced by the CloudFormation templates.
- Build bundle → Web console: bundles · Frontend static assets are built into the solution distribution.
- Build bundle → Domain Lambdas: bundles · Lambda artifacts are packaged for deployment.
- Web console → Routes: uses · Application is split into pre-authenticated and authenticated route sets.
- Web console → API clients: calls · UI talks to backend services through typed clients.
- Web console → Schema UI: renders · Schema-aware forms drive entity editing and validation.
- Web console → Import flow: guides · Import wizards manage spreadsheet and batch ingestion flows.
- Web console → Planning tools: supports · Planning views cover waves, pipelines, tasks, and execution status.
- API clients → Identity: auth APIs · Login and user administration endpoints back the authenticated console.
- API clients → Domain Lambdas: domain APIs · Main application operations are served by backend Lambdas.
- API clients → SSM channel: ops APIs · Remote execution and script management are exposed through SSM services.
- API clients → MGN automation: automation APIs · Migration execution workflows connect to MGN handlers.
- Import flow → Import backend: uploads · Validated files are staged to S3 and processed server-side.
- Schema UI → Entity store: validates against · Frontend editing mirrors backend schema and relationship rules.
- Identity → Credential mgr: shares identity · Authentication and secret-managed access are part of the security boundary.
- Domain Lambdas → Entity store: reads/writes · Domain Lambdas persist and query metadata state in DynamoDB.
- Domain Lambdas → Lambda layers: reuses · Domain services depend on shared layers for common behavior.
- Domain Lambdas → SSM channel: orchestrates · Task and pipeline services trigger remote execution workflows.
- Domain Lambdas → MGN automation: drives · Migration domain services coordinate MGN-based execution.
- Domain Lambdas → Tracker ETL: reports to · Operational events can feed tracker extraction and analytics.
- Domain Lambdas → Identity: authorizes · User and group management enforce the Cognito boundary.
- Import backend → Entity store: persists · Imported records are transformed into managed entities.
- Import backend → Lambda layers: reuses · Import pipeline uses shared utilities and pipeline helpers.
- SSM channel → Lambda layers: reuses · SSM handlers rely on shared constants, logging, and pipeline helpers.
- MGN automation → Common utils: uses · Migration automation packages share common integration helpers.
- Tracker ETL → Common utils: uses · Tracker ETL scripts rely on shared integration helpers.