# avinassh/py-caskdb architecture

> Architecture diagram of the GitHub repository avinassh/py-caskdb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/avinassh/py-caskdb
- Repository: https://github.com/avinassh/py-caskdb

## Overview

##### Main Components Identification
The system has three primary layers that should be prominently featured:

a) Memory Layer (KeyDir)
- As a top-level component
- Shows the in-memory hash table storing key-to-position mappings
- Indicated by memory_store.py implementation

b) Disk Layer
- As a bottom-level component
- Shows the log-structured storage format
- Indicated by disk_store.py implementation

c) Format Layer
- As a middle layer
- Shows the serialization/deserialization logic
- Indicated by format.py implementation

##### Specific Implementation Details

Technical aspects:
- KeyDir structure in memory
- File handle management
- Row pointer structure
- Serialization format

##### Additional Elements

Supporting components:
- Version information
- File format compatibility notes
- Key limitations (from README)

**[Client Interface]**
- ↕
**[Memory Store (KeyDir)]**
- ↕
**[Format Layer]**
- ↕
**[Disk Store]**
- ↕
**[Physical Storage]**

##### Important Notes

- Log-structured nature of the storage
- How the KeyDir maps to disk positions
- Single-file storage design
- Platform-independent nature of the file format

##### Technical Constraints to Highlight

- All keys must fit in memory
- Single file storage limitation
- No range scan support
- Startup time considerations

## Diagram (Mermaid)

```mermaid
graph TB
    %% Main Components
    Client(["Client Interface"])
    
    subgraph Memory["Memory Layer"]
        KeyDir["KeyDir Hash Table"]:::memory
        MemoryStore["Memory Store"]:::memory
    end
    
    subgraph Format["Format Layer"]
        FormatModule["Format Module"]:::format
        Header["Header Structure<br/>(12 bytes)"]:::format
        DataSection["Data Section<br/>(Variable Length)"]:::format
    end
    
    subgraph Disk["Disk Layer"]
        DiskStore["Disk Store"]:::disk
        LogStorage["Log-Structured<br/>Storage"]:::disk
    end
    
    %% Relationships - Write Path
    Client -->|Write| MemoryStore
    MemoryStore -->|Update KeyDir| KeyDir
    MemoryStore -->|Serialize| FormatModule
    FormatModule -->|Format Data| Header
    FormatModule -->|Format Data| DataSection
    Header & DataSection -->|Write to Disk| DiskStore
    DiskStore -->|Append| LogStorage

    %% Relationships - Read Path
    Client -.->|Read| MemoryStore
    MemoryStore -.->|Lookup| KeyDir
    KeyDir -.->|Get Position| DiskStore
    DiskStore -.->|Read Data| FormatModule
    FormatModule -.->|Deserialize| Client

    %% Subcomponent Details
    Header -->|Contains| TimestampSize["Timestamp<br/>(4 bytes)"]
    Header -->|Contains| KeySize["Key Size<br/>(4 bytes)"]
    Header -->|Contains| ValueSize["Value Size<br/>(4 bytes)"]

    %% Styling
    classDef memory fill:#d0e8ff,stroke:#333,stroke-width:2px
    classDef format fill:#fff3b0,stroke:#333,stroke-width:2px
    classDef disk fill:#d0ffd0,stroke:#333,stroke-width:2px
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px

    %% Click Events
    click MemoryStore "https://github.com/avinassh/py-caskdb/blob/master/src/caskdb/memory_store.py"
    click DiskStore "https://github.com/avinassh/py-caskdb/blob/master/src/caskdb/disk_store.py"
    click FormatModule "https://github.com/avinassh/py-caskdb/blob/master/src/caskdb/format.py"
    click KeyDir "https://github.com/avinassh/py-caskdb/blob/master/src/caskdb/memory_store.py"
    click LogStorage "https://github.com/avinassh/py-caskdb/blob/master/src/caskdb/disk_store.py"

    %% Legend
    subgraph Legend
        L1["Memory Components"]:::memory
        L2["Format Components"]:::format
        L3["Disk Components"]:::disk
        L4["--> Write Flow"]
        L5["-.-> Read Flow"]
    end
```
