How mateusfreira/nun-db works
Main Components
• Network Layer: Detail how incoming requests are handled. The “src/lib/network” folder includes http_ops, tcp_ops, and ws_ops modules. This layer is where incoming text-based commands from clients are received.
• Request Processing: Modules like “parse_request.rs” and “process_request.rs” indicate that raw requests are parsed and then dispatched to the appropriate commands.
• Database Core Operations: This includes core modules like db_ops, replication_ops, election_ops, command_line handling, and consensus_ops. These modules implement the actual database commands (CRUD operations, snapshots, replication, increment, etc.) and consensus/leader election.
• Storage Layer: Located in “src/lib/storage” (with submodules for disk and S3-based storage) and possibly “disk_ops” in addition to “memory repository” sections. These represent how the actual data is persisted either to disk or possibly S3.
• Monitoring and Configuration: The “monitoring.rs” and “configuration.rs” files manage the health of the system and the overall settings (like port numbers, replication addresses, timeout intervals, etc., as detailed in the README).
• External Tools and Connectors: The Docker configurations (Dockerfile, docker-compose.yml) and CI workflows indicate automated builds, testing, and deployment aspects.
Relationships and Data Flow
• Inside the processing pipeline, label that parsed requests are dispatched to “db_ops”, “replication_ops” or “election_ops”, depending on the command.
• Data flow from these modules to the storage layer. For example, a “set” operation should show data moving from the command processor to the memory repository and then persisting via “disk_ops” or “s3” modules.
• Also, include feedback loops indicating replication and consensus operations. For instance, the replication_ops module interacts with network modules (for joining other nodes) and storage (for maintaining consistency).
Architectural Patterns and Design Principles
• Modular Design: Separate concerns (network handling, request processing, core database logic, storage, and monitoring) that are clearly segregated in different folders.
• Command Pattern: The presence of a command_line directory suggests a modular design where each database command (create-db, use-db, set, snapshot, etc.) is self-contained and can be managed independently.
• Leaderless and Replication Model: Some modules like consensus_ops, election_ops, and replication_ops reveal that the architecture supports distributed consensus and leader election strategies.
• Safety and Scalability: The design details (like using Rust and providing configurable options via environment variables) indicate an emphasis on memory safety, performance, and future scalability.
Relevant Technologies and Frameworks
• Rust: The entire codebase is in Rust, as suggested by Cargo.toml and GitHub workflow files (e.g. rust.yml).
• Docker: The Dockerfile(s) and docker-compose are used for container deployment highlighting how the system can be packaged and run in production.
• CI/CD: GitHub workflows for testing and building suggest that the system is designed for robust deployment cycles.