How siemens/snap-to-bucket works
Project type and purpose
• This is a Python‐based CLI tool (open‐source utility) that runs on an EC2 instance.
• Its goal is to migrate AWS EBS snapshots into an S3 bucket (and optionally restore them).
File structure
• src/snap_to_bucket/run.py provides the CLI entrypoint.
• src/snap_to_bucket/runner/snap_handler.py encapsulates the overall workflow.
• src/snap_to_bucket/handlers/ holds three handler classes:
• ec_2_handler.py – uses boto3 to list/attach/detach EBS volumes & snapshots.
• fs_handler.py – handles mounting, partitioning, tar/gzip, splitting.
• s3_handler.py – uses boto3 to upload/download objects from S3.
• Configuration/deployment info is in Pipfile, setup.py, pyproject.toml.
README for additional insights
• The tool runs on an EC2 instance with an IAM role permitting EC2 and S3 operations.
• CLI flags control tag selection, S3 bucket, volume type, gzip, splitting, delete-on-transfer, restore mode.
A. CLI Layer
• snap2bucket command (entrypoint)
B. Orchestration Layer
• SnapHandler runner
C. Handler Components
1. EC2Handler – calls AWS EC2 APIs via boto3
2. FSHandler – local OS calls for mount/umount, tar, gzip, split
3. S3Handler – S3 API calls via boto3
D. External Services
• AWS EC2 snapshot & volume service
• AWS S3 (bucket)
E. Local Environment
• Linux file system (e.g., /mnt/snaps) on the EC2 instance
Relationships & data flows
• CLI → SnapHandler: parses options
• SnapHandler → EC2Handler:
• Snapshots by tag
• Create volume from snapshot
• Attach volume
• EC2Handler → FSHandler:
• Mount the attached device
• Tar (± gzip) and split into chunks
• FSHandler → S3Handler:
• Upload chunk(s) to S3
• Optionally delete snapshots via EC2Handler
• For restore mode, reverse: S3Handler → FSHandler → EC2Handler
Architectural patterns and principles
• Separation of concerns: each handler has a single responsibility.
• Plugin‐style extensibility: adding a new handler for e.g., another cloud provider.
• CLI orchestration + dependency injection: runner invokes handlers in a defined sequence.