# jajir/hestiastore architecture

> Architecture diagram of the GitHub repository jajir/hestiastore, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jajir/hestiastore
- Repository: https://github.com/jajir/hestiastore

## Overview

##### Project type and purpose
- HestiaStore is an embeddable, Java-based key-value storage engine library (no UI or network layer).
- It provides a lightweight API (Index and Directory abstractions) and implements on-disk/in-memory storage, optional write-ahead logging, caches, bloom filters, segments, and pluggable data file formats.

##### File structure
- Top-level: pom.xml, docs/, src/ (main, test, integration-test).
- src/main/java/org/hestiastore/index: Core API (Index, IndexConfiguration, Directory).
- Subpackages implement storage modules:
– directory: abstraction for MemDirectory, FsDirectory, ZipDirectory, etc.
– datatype: key/value serialization (TypeDescriptor, Convertor, VarLengthReader/Writer).
– chunkstore, chunkpairfile, datablockfile, sorteddatafile, unsorteddatafile: various on-disk file formats.
– segment: segment builder, compaction, splitting, searchers, consistency checks.
– bloomfilter: optional probabilistic membership check.
– cache: LRU and unique caches for segments or keys.
– log: write-ahead logging (LogWriter, LogFilesManager).
– properties: persisting index properties and configuration.
– scarceindex: alternative minimal index strategy.
– sst: high-level “Sorted String Table” index interface and state machine.

##### README for additional insights
- Describes features: in-memory or file-backed indexes, optional WAL, user-defined types, thread-safety toggle.
- Links to architecture docs, usage, configuration, logging guides.

##### Elements
- Labels: name each module/package clearly (e.g., “segment”, “chunkstore”).
– API (blue)
– Engine/core (green)
– Storage/IO (orange)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top-level client
    ClientApp(["Client Application"]):::external

    %% Public API Layer
    subgraph "Public API" 
        direction TB
        IndexAPI["Index API Layer"]:::api
        PublicCore["Public API Core"]:::api
    end

    %% Engine/Core Layer
    subgraph "Engine/Core" 
        direction TB
        SegmentMgr["Segment Manager"]:::core
        Caching["Caching"]:::core
        Bloom["Bloom Filter"]:::core
        Logging["Write-Ahead Logging (WAL)"]:::optional
        Datatype["Datatype Serialization"]:::core
        Properties["Properties Store"]:::core
        Scarce["ScarceIndex (Alternative)"]:::optional
    end

    %% Storage/IO Layer
    subgraph "Storage / I/O" 
        direction TB
        DirAbst["Directory Abstraction"]:::storage
        UnsortedDF["UnsortedDataFile"]:::storage
        SortedDF["SortedDataFile"]:::storage
        CPF["ChunkPairFile"]:::storage
        CS["ChunkStore"]:::storage
        DBF["DataBlockFile"]:::storage
    end

    %% Connections
    ClientApp -->|"invoke Index.create(), put(), get(), delete()"| IndexAPI
    IndexAPI -->|"delegates to"| PublicCore
    IndexAPI -->|"opens"| DirAbst
    IndexAPI -->|"routes to"| SegmentMgr

    SegmentMgr -->|"writes/reads from"| UnsortedDF
    SegmentMgr -->|"writes/reads from"| SortedDF
    SegmentMgr -->|"writes/reads from"| CPF
    SegmentMgr -->|"writes/reads from"| CS
    SegmentMgr -->|"writes/reads from"| DBF

    UnsortedDF -->|"file I/O via"| DirAbst
    SortedDF -->|"file I/O via"| DirAbst
    CPF -->|"file I/O via"| DirAbst
    CS -->|"file I/O via"| DirAbst
    DBF -->|"file I/O via"| DirAbst

    SegmentMgr -->|"check cache"| Caching
    SegmentMgr -->|"use bloom filter"| Bloom

    IndexAPI -->|"optional WAL append"| Logging
    IndexAPI -->|"after commit flush"| Properties

    %%% Optional alternative path
    IndexAPI -->|"alternative index"| Scarce

    %% Click Events
    click IndexAPI "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index"
    click PublicCore "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index"
    click DirAbst "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/directory"
    click SegmentMgr "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/segment"
    click UnsortedDF "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/unsorteddatafile"
    click SortedDF "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/sorteddatafile"
    click CPF "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/chunkpairfile"
    click CS "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/chunkstore"
    click DBF "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/datablockfile"
    click Caching "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/cache"
    click Bloom "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/bloomfilter"
    click Logging "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/log"
    click Datatype "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/datatype"
    click Properties "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/properties"
    click Scarce "https://github.com/jajir/hestiastore/tree/main/src/main/java/org/hestiastore/index/scarceindex"

    %% Styles
    classDef api fill:#ADD8E6,stroke:#333;
    classDef core fill:#90EE90,stroke:#333;
    classDef storage fill:#FFA500,stroke:#333;
    classDef optional fill:#D3D3D3,stroke:#333,stroke-dasharray:5 5;
    classDef external fill:#FFFFE0,stroke:#333;
```
