# iamd3vil/blobasaur architecture

> Architecture diagram of the GitHub repository iamd3vil/blobasaur, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iamd3vil/blobasaur
- Repository: https://github.com/iamd3vil/blobasaur

## Overview

Blobasaur is a Redis-protocol-compatible sharded blob store. Redis clients send commands to the TCP server, which parses and handles requests, routes storage work across shard state and SQLite, and returns RESP replies. The product also includes cluster discovery and redirects, shard migration and vacuum operations, configurable compression, TTL cleanup, and Prometheus-compatible monitoring. Edges follow explicit README statements or the shown source references; several internal details are unsampled, so uncertain links are omitted.

Main entry points: [`src/server.rs`](https://github.com/iamd3vil/blobasaur/blob/main/src/server.rs) (TCP command server), [`src/compression/mod.rs`](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/mod.rs) (Compression selection), [`src/app_state.rs`](https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs) (Application state), [`src/shard_manager.rs`](https://github.com/iamd3vil/blobasaur/blob/main/src/shard_manager.rs) (Shard write workers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_request["Redis request serving"]
  node_config["Configuration<br/>[config.rs]"]
  node_protocol["RESP protocol<br/>[protocol.rs]"]
  node_tcp["TCP command server<br/>[server.rs]"]
end

subgraph group_storage["Sharded storage"]
  node_appstate["Application state<br/>[app_state.rs]"]
  node_sharding["Shard selection<br/>[app_state.rs]"]
  node_writeworkers["Shard write workers<br/>[shard_manager.rs]"]
  node_sqlite[("SQLite shard databases<br/>[app_state.rs]")]
  node_compression["Compression selection<br/>[mod.rs]"]
  node_brotli["Brotli<br/>[brotli.rs]"]
  node_gzip["Gzip<br/>[gzip.rs]"]
  node_lz4["LZ4<br/>[lz4.rs]"]
  node_none["No compression<br/>[none.rs]"]
  node_zstd["Zstandard<br/>[zstd.rs]"]
end

subgraph group_cluster["Cluster operations"]
  node_cluster_manager["Cluster discovery<br/>[cluster.rs]"]
end

subgraph group_maintenance["Maintenance"]
  node_migration["Shard migration<br/>[migration.rs]"]
  node_vacuum_cli["Vacuum orchestration<br/>[main.rs]"]
  node_vacuum_shards["Shard vacuum<br/>[shard_manager.rs]"]
end

subgraph group_observability["Monitoring"]
  node_metrics["Metrics collection<br/>[metrics.rs]"]
  node_http["Metrics and health HTTP<br/>[http_server.rs]"]
end

node_clients(("Redis clients"))
node_prometheus(("Prometheus scraper"))

node_clients -->|"sends commands"| node_tcp
node_tcp -->|"parses and encodes"| node_protocol
node_tcp -->|"uses state"| node_appstate
node_tcp -.->|"routes requests"| node_sharding
node_appstate -->|"creates hash ring"| node_sharding
node_appstate -->|"opens shard pools"| node_sqlite
node_writeworkers -->|"writes batches"| node_sqlite
node_tcp -.->|"dispatches storage work"| node_writeworkers
node_appstate -->|"initializes compressor"| node_compression
node_compression -->|"selects algorithm"| node_brotli
node_compression -->|"selects algorithm"| node_gzip
node_compression -->|"selects algorithm"| node_lz4
node_compression -->|"selects algorithm"| node_none
node_compression -->|"selects algorithm"| node_zstd
node_config -->|"validates settings"| node_compression
node_tcp -->|"requests redirects"| node_cluster_manager
node_appstate -->|"initializes cluster"| node_cluster_manager
node_vacuum_cli -->|"connects to nodes"| node_tcp
node_tcp -->|"dispatches vacuum"| node_vacuum_shards
node_vacuum_cli -->|"runs migration command"| node_migration
node_tcp -->|"records operations"| node_metrics
node_writeworkers -->|"records shard activity"| node_metrics
node_http -->|"exports metrics"| node_metrics
node_prometheus -->|"scrapes endpoint"| node_http

click node_config "https://github.com/iamd3vil/blobasaur/blob/main/src/config.rs"
click node_protocol "https://github.com/iamd3vil/blobasaur/blob/main/src/redis/protocol.rs"
click node_tcp "https://github.com/iamd3vil/blobasaur/blob/main/src/server.rs"
click node_appstate "https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs"
click node_sharding "https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs"
click node_writeworkers "https://github.com/iamd3vil/blobasaur/blob/main/src/shard_manager.rs"
click node_sqlite "https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs"
click node_compression "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/mod.rs"
click node_brotli "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/brotli.rs"
click node_gzip "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/gzip.rs"
click node_lz4 "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/lz4.rs"
click node_none "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/none.rs"
click node_zstd "https://github.com/iamd3vil/blobasaur/blob/main/src/compression/zstd.rs"
click node_cluster_manager "https://github.com/iamd3vil/blobasaur/blob/main/src/cluster.rs"
click node_migration "https://github.com/iamd3vil/blobasaur/blob/main/src/migration.rs"
click node_vacuum_cli "https://github.com/iamd3vil/blobasaur/blob/main/src/main.rs"
click node_vacuum_shards "https://github.com/iamd3vil/blobasaur/blob/main/src/shard_manager.rs"
click node_metrics "https://github.com/iamd3vil/blobasaur/blob/main/src/metrics.rs"
click node_http "https://github.com/iamd3vil/blobasaur/blob/main/src/http_server.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_protocol,node_tcp toneBlue
class node_appstate,node_sharding,node_writeworkers,node_sqlite,node_compression,node_brotli,node_gzip,node_lz4,node_none,node_zstd,node_clients toneAmber
class node_cluster_manager toneMint
class node_migration,node_vacuum_cli,node_vacuum_shards toneRose
class node_metrics,node_http,node_prometheus toneIndigo
```

## Components

### Redis request serving

- [Configuration](https://github.com/iamd3vil/blobasaur/blob/main/src/config.rs): `src/config.rs`
- [RESP protocol](https://github.com/iamd3vil/blobasaur/blob/main/src/redis/protocol.rs): `src/redis/protocol.rs`
- [TCP command server](https://github.com/iamd3vil/blobasaur/blob/main/src/server.rs): `src/server.rs`

### Sharded storage

- [Application state](https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs): `src/app_state.rs`
- [Shard selection](https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs): `src/app_state.rs`
- [Shard write workers](https://github.com/iamd3vil/blobasaur/blob/main/src/shard_manager.rs): `src/shard_manager.rs`
- [SQLite shard databases](https://github.com/iamd3vil/blobasaur/blob/main/src/app_state.rs): `src/app_state.rs`
- [Compression selection](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/mod.rs): `src/compression/mod.rs`
- [Brotli](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/brotli.rs): `src/compression/brotli.rs`
- [Gzip](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/gzip.rs): `src/compression/gzip.rs`
- [LZ4](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/lz4.rs): `src/compression/lz4.rs`
- [No compression](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/none.rs): `src/compression/none.rs`
- [Zstandard](https://github.com/iamd3vil/blobasaur/blob/main/src/compression/zstd.rs): `src/compression/zstd.rs`

### Cluster operations

- [Cluster discovery](https://github.com/iamd3vil/blobasaur/blob/main/src/cluster.rs): `src/cluster.rs`

### Maintenance

- [Shard migration](https://github.com/iamd3vil/blobasaur/blob/main/src/migration.rs): `src/migration.rs`
- [Vacuum orchestration](https://github.com/iamd3vil/blobasaur/blob/main/src/main.rs): `src/main.rs`
- [Shard vacuum](https://github.com/iamd3vil/blobasaur/blob/main/src/shard_manager.rs): `src/shard_manager.rs`

### Monitoring

- [Metrics collection](https://github.com/iamd3vil/blobasaur/blob/main/src/metrics.rs): `src/metrics.rs`
- [Metrics and health HTTP](https://github.com/iamd3vil/blobasaur/blob/main/src/http_server.rs): `src/http_server.rs`

### Other components

- **Redis clients**
- **Prometheus scraper**

## Connections

- Redis clients → TCP command server: sends commands (evidence: `README.md`)
- TCP command server → RESP protocol: parses and encodes (evidence: `src/server.rs`)
- TCP command server → Application state: uses state (evidence: `src/server.rs`)
- TCP command server → Shard selection: routes requests
- Application state → Shard selection: creates hash ring (evidence: `src/app_state.rs`)
- Application state → SQLite shard databases: opens shard pools (evidence: `src/app_state.rs`)
- Shard write workers → SQLite shard databases: writes batches (evidence: `src/shard_manager.rs`)
- TCP command server → Shard write workers: dispatches storage work (evidence: `src/server.rs`)
- Application state → Compression selection: initializes compressor (evidence: `src/app_state.rs`)
- Compression selection → Brotli: selects algorithm (evidence: `src/compression/mod.rs`)
- Compression selection → Gzip: selects algorithm (evidence: `src/compression/mod.rs`)
- Compression selection → LZ4: selects algorithm (evidence: `src/compression/mod.rs`)
- Compression selection → No compression: selects algorithm (evidence: `src/compression/mod.rs`)
- Compression selection → Zstandard: selects algorithm (evidence: `src/compression/mod.rs`)
- Configuration → Compression selection: validates settings (evidence: `src/config.rs`)
- TCP command server → Cluster discovery: requests redirects (evidence: `src/server.rs`)
- Application state → Cluster discovery: initializes cluster (evidence: `src/app_state.rs`)
- Vacuum orchestration → TCP command server: connects to nodes (evidence: `src/main.rs`)
- TCP command server → Shard vacuum: dispatches vacuum (evidence: `src/server.rs`)
- Vacuum orchestration → Shard migration: runs migration command (evidence: `src/main.rs`)
- TCP command server → Metrics collection: records operations (evidence: `src/server.rs`)
- Shard write workers → Metrics collection: records shard activity (evidence: `src/shard_manager.rs`)
- Metrics and health HTTP → Metrics collection: exports metrics (evidence: `src/http_server.rs`)
- Prometheus scraper → Metrics and health HTTP: scrapes endpoint (evidence: `README.md`)
