How abendy/macos-to-s3 works
Project type and purpose
• This is a command‑line backup automation tool for macOS that uses BorgBackup for encrypted archiving and Amazon S3 as remote storage. It is neither a multi‑tier web app nor a service mesh, but a standalone orchestrator (borg‑backup.sh) that ties together local configuration, key management, Borg, and AWS S3.
File structure
• Top‑level files:
• .env.sample → environment variable template for AWS/Borg settings
• borg‑backup.sh → the orchestration script driving all backup operations
• requirements.txt → Python dependencies (if borg‑backup.sh or hooks call any Python modules)
• etc/ directory: sample includes/excludes patterns for controlling which files get backed up
• Aux files: .gitignore, README.md
• No “frontend” or “backend” folders; project is essentially one component (the script) plus config and key directories.
README for additional insights
• Describes setup of virtualenv, creation of Borg cache/security dirs, GPG/pass integration for encryption keys, AWS CLI configuration, IAM user setup, and borg init/create/list/extract commands.
a. Main components
1. macOS Host (Client)
– Operating system where backups originate and the script runs
2. Configuration Layer
– .env file (AWS credentials, bucket/path)
– etc/backup.includes & etc/backup.excludes
3. Key Management
– GPG suite & pass (password‑store) for Borg encryption passphrases
4. Orchestration Script
– borg‑backup.sh (invokes environment, GPG/pass, borg commands)
5. BorgBackup
– Local cache/security dirs (/usr/local/var/lib/borg/{cache,security})
– Local repository (optionally) or remote repo URL pointing at S3
6. AWS S3 (External Service)
– S3 bucket, accessed using AWS CLI credentials stored in .env
b. Relationships & data flows
– Script loads .env → exports AWS & Borg env vars
– Script invokes GPG/pass → retrieves encryption keys/passphrase
– Script invokes borg init/create → writes archives into “repository” (could be local FS or remote via S3 plugin/SSH)
– Borg writes to cache/security dirs for performance & key storage
– Borg (or separate AWS CLI push, if implemented) transfers archive data to S3 bucket
– For restore/list, script calls borg list/extract/mount against the same repo URL
c. Architectural patterns & design principles
– Separation of concerns: config (env + includes/excludes), key management, orchestration logic, storage
– Declarative patterns: sample files (.env.sample, backup.includes.sample) → user‑provided config
– Fail‑fast design: script should validate env vars & key accessibility before starting backup
– Encryption‑by‑default: all archives are Borg‑encrypted using GPG/pass integration
d. Key technologies, frameworks, libraries
– Bash (or POSIX shell) scripting
– Python (only if borg‑backup.sh uses Python modules from requirements.txt)
– BorgBackup (archive tool with encryption & deduplication)
– GPG & pass password‑store for key management
– AWS CLI / IAM policies for S3 access
Detail, not over‑abstraction
• Both read (restore/list) and write (create/backup) paths if possible