How dibbed/redis-mongo-backup-tool works
Project Type and Purpose
• This is a command-line backup tool (open-source utility) written in Python.
• Its goal is to back up, restore, and clear data in two external systems: Redis keyspaces (via the Redis client library) and MongoDB databases (by shelling out to the official mongodump/mongorestore tools).
• It’s designed for modularity, environment-based configuration, and a simple entrypoint.
File Structure
• Top-level:
• main.py – CLI entrypoint that parses commands and dispatches to subcommands
• requirements.txt – Python dependencies (e.g., redis, click/argparse)
• .gitignore, LICENSE, README.md
• backup_tool/ module:
• config.py – reads environment variables or defaults (connection URIs, backup directory, log settings)
• redis_manager.py – contains functions to backup, restore, clear keys matching a pattern
• mongo_manager.py – wraps mongodump/mongorestore shell commands for a named database
• utils.py – logging setup, error handling, common helpers
Extract Architectural Insights from README
• The README outlines a clear separation between Redis operations (in-process via Python) and MongoDB operations (via external CLI tools).
• It stresses environment-driven configuration and a modular codebase with one module per concern.
• No persistent server component—this is a stateless CLI utility that interacts only with:
• External Redis server
• External MongoDB Database Tools
• Local filesystem (for backup files)
Components and Relationships
a. Main Components
1. CLI Entrypoint (main.py)
2. Config Manager (backup_tool/config.py)
3. Redis Manager (backup_tool/redis_manager.py)
4. Mongo Manager (backup_tool/mongo_manager.py)
5. Utils (logging & error handling)
6. External Systems:
– Redis Server (connected via redis://…)
– MongoDB Tools (mongodump/mongorestore on PATH)
7. Local Filesystem (backup directory)
b. Relationships & Data Flow
• User runs python main.py <service> <action> [flags] → CLI Entrypoint
• Entrypoint loads environment variables via Config Manager → builds connection URIs, paths
• CLI passes control to either Redis Manager or Mongo Manager
– Redis Manager uses Python Redis client to connect → reads/writes keys → writes JSON to Local Filesystem or reads JSON from it
– Mongo Manager shells out to mongodump/mongorestore → reads/writes BSON files in Local Filesystem
• All operations log through Utils
c. Architectural Patterns & Principles
• Single Responsibility: each manager handles one external system
• Separation of Concerns: config, business logic, and utilities are decoupled
• Statelessness: CLI does not maintain long-lived state beyond file I/O
• Extensibility: new managers (e.g., for another database) can be added under backup_tool/
d. Key Technologies & Frameworks
• Python 3.9+
• Redis-py (for Redis operations)
• System shell calls to MongoDB Database Tools
• Standard library modules for JSON, subprocess, and logging
Detailed Coverage of Architectural Elements
• Do not omit the Utils module: it’s central for logging and error handling
• Configuration defaults vs. overrides via CLI flags
• Two distinct backup paths:
1. Redis: Pattern → JSON file in backups dir
2. MongoDB: Database name → filesystem directory of BSON dumps
• Call out external dependencies: Redis server, MongoDB tools