How fullerzz/borgboi works
Project Type and Purpose
• Type: Command-line utility/back-end tool.
• Purpose: To wrap and automate common Borg backup operations (e.g., creating archives, pruning, compacting) and to sync backup archives with an S3 bucket. It also includes functionalities for displaying logs, validating repository state, and potentially managing backup metadata through DynamoDB.
Analyzing the File Structure
• Top-Level Organization:
• Configuration and project management files (.editorconfig, .github workflows, pyproject.toml, etc.).
• Source code in the src/borgboi directory which includes the main modules (cli.py, orchestrator.py, backups.py, dynamodb.py, rich_utils.py, validator.py).
• Terraform directory contains infrastructure-as-code setups for AWS resources (dynamodb.tf, s3.tf, iam.tf, etc.), representing the external system provisioning.
• Tests directory for unit and integration tests.
• Implicit Architectural Patterns:
• The CLI layer (cli.py) is the entry point.
• The orchestrator (orchestrator.py) likely acts as a mediator/controller that coordinates various backup operations.
• The backups.py module encapsulates core backup functionalities (creating archives, pruning, etc.).
• Dynamodb.py provides integration with AWS DynamoDB, likely for tracking state or metadata.
• Rich_utils.py suggests that terminal output is enhanced for usability (for example, using colorized logs or formatted outputs).
• External Dependencies:
• Borg (the underlying backup tool).
• AWS services (S3, DynamoDB, IAM) configured via Terraform.
Key Components
a. Internal Components:
• CLI Component: Represented by src/borgboi/cli.py where user commands like “daily-backup” are parsed.
• Orchestrator: src/borgboi/orchestrator.py that manages workflow and business logic, coordinating between different modules.
• Backup Module: src/borgboi/backups.py that wraps core Borg backup operations.
• DynamoDB Integration: src/borgboi/dynamodb.py handling interactions with DynamoDB.
• Validation & Utility Components: src/borgboi/validator.py and rich_utils.py used for error-checking and enriched terminal output.
b. External Components:
• Borg Backup Tool: An external dependency where Borg commands are executed.
• AWS S3: Representing the S3 bucket where the backup repository and archives are synced.
• AWS DynamoDB: Represents the database for backup metadata/state.
• Terraform Infrastructure: A deployed infrastructure seen in the terraform directory (s3.tf, dynamodb.tf, iam.tf) which defines and manages the external resources.
Relationships and Interactions
• The CLI module is the user’s entry point; once a command is issued (e.g., “daily-backup”), it calls the orchestrator.
• The Orchestrator coordinates with:
• The backups module to execute Borg backup operations.
• The dynamodb integration module for updating or retrieving backup metadata.
• The backups module wraps and invokes the underlying Borg commands.
• Post backup operations, synchronization logic ensures that the backup repository is updated with the S3 bucket.
• Terraform provisions and maintains the external AWS infrastructure, which underpins the S3 and DynamoDB services.
Architectural Patterns & Design Principles
• Layered Architecture:
• Presentation/CLI Layer -> Orchestrator (Business Logic) -> Service Modules (Backups, DynamoDB, etc.)
• Separation of Concerns:
• Each Python module handles a specific function (CLI handling, backup operations, AWS interactions, validations).
• Infrastructure as Code:
• Using Terraform to provision, manage, and version the external cloud resources.
• External Dependency Integration:
• Clear demarcation between the internal application logic and external services (Borg, AWS).