# fmartinez09/java-wal architecture

> Architecture diagram of the GitHub repository fmartinez09/java-wal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fmartinez09/java-wal
- Repository: https://github.com/fmartinez09/java-wal

## Overview

##### Project Type and Purpose
- This is a standalone Java library/tool implementing a WAL storage engine (not a web app).
- Purpose: provide durable, append-only logging with recovery, segmentation, checksums, and an iterator API.

##### File Structure
- src/main/java/LogFile.java – core class handling append, read, fsync, page buffering, fragmentation, and segment rotation.
- src/main/java/Record.java – encapsulates a payload + metadata (length, CRC).
- src/test/java/TestLog.java – unit tests for append/read/recovery behavior.
- pom.xml – Maven build, declares dependencies (e.g., for CRC or logging).

##### README for Architectural Insights
- Binary record format: length header, data, CRC32 footer.
- Page buffering & configurable padding, fsync policies.
- Record fragmentation logic (FULL/FIRST/MIDDLE/LAST).
- Recovery scan on startup that truncates incomplete/corrupt pages.
- Segment rotation scheme (wal-000001.log, wal-000002.log, …).
- Iterator API for replay.

##### Main Components to Illustrate
- Client API Layer
– Exposes append(Record) and readNext() / iterator operations.
- WAL Core / Manager
– Coordinates page buffering, fragmentation, CRC32 calculation, and segment rotation.
– Implements fsync policy (Always / Never / Every N records / Time-based).
- Page Buffer & Serializer
– Serializes Record → [len][data][crc].
– Splits/fragments large records across fixed-size pages.
- File I/O & Segmentation
– Writes buffers to disk files named wal-000001.log, etc.
– Reads from appropriate segment during replay.
- Recovery Module
– On open, scans all segments, validates checksums, truncates incomplete pages/records.
- Underlying File System (OS)
– Durable storage; fsync calls for persistence guarantees.

##### Relationships & Data Flow
1. Client calls append() → WAL Manager
2. WAL Manager serializes via Page Buffer → applies fragmentation
3. Buffer flushed to File I/O layer → OS cache → optionally fsync
4. On readNext() or iterator, File I/O → Page Buffer → deserializer → Record to client
5. At startup/close, Recovery Module inspects segments → validates CRC → truncates if needed

##### Architectural Patterns & Principles
- Layered Architecture:
– API Layer → Core/Manager → I/O Layer → OS
- Single Responsibility:
– LogFile.java handles orchestration; Record.java handles data encapsulation.
- Pluggable Policies:
– FsyncPolicy interface (Always, Never, EveryN, TimeBased).
- Modular Recovery:
– Segregate startup scan & truncation logic.

##### Technologies / Frameworks
- Java SE (java.io.File, java.nio.*, java.util.zip.CRC32)
- Maven (pom.xml) for build and dependency management
- JUnit/TestNG (tests) for unit testing

[ ] Main components (API, Core, Page Buffer, File I/O, Recovery, Disk)
[ ] Key format details ([len|data|crc], fragmentation stages)
[ ] Configuration/policy hooks (fsync, page size)
[ ] Segment sequence representation
[ ] Recovery path on startup

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client API Layer
    subgraph "Client API Layer"
        API["LogFile API\n(append()/readNext()/iterator)"]:::api
    end

    %% Core / Manager Layer
    subgraph "Core (WAL Manager)"
        Core["WAL Manager\n(page buffering, fragmentation,\nsegment rotation, fsync coordination)"]:::core
        PS["Page Buffer & Serializer\n([len|data|crc], fragmentation)"]:::core
        Record["Record\n(payload + metadata,\nlength header + CRC32 footer)"]:::core
    end

    %% Recovery Module
    subgraph "Recovery Module"
        Rec["Recovery Module\n(startup scan, CRC validation,\ntruncation)"]:::core
    end

    %% File I/O & Segmentation
    subgraph "File I/O & Segmentation"
        IO["File I/O & Segmentation\n(wal-000001.log → wal-000002.log,\nreads/writes)"]:::io
    end

    %% Disk / External
    Disk[("Disk Storage\n(OS filesystem)")]:::external

    %% Policies & Configuration
    subgraph "Policies & Config"
        Pol["Fsync Policies\n(Always/Never/EveryN/TimeBased)"]:::policy
        Build["Maven Build Config\n(pom.xml)"]:::policy
    end

    %% Auxiliary Components
    subgraph "Auxiliary"
        RecTests["Unit Tests\n(TestLog.java)"]:::test
        Doc["Documentation & Usage Guide\n(README.md)"]:::doc
        Sample["Sample WAL Output File\n(mylog.bin)"]:::external
    end

    %% Data Flows
    API -->|"calls append()/readNext()"| Core
    Core --> PS
    PS --> Record
    PS -->|"serializes to"| IO
    IO --> Disk
    Core -->|"on startup"| Rec
    Rec --> IO
    Pol -->|"policy injected into"| Core
    Build --> Pol
    RecTests -.->|"validates"| API
    RecTests -.->|"validates"| Rec
    Doc -.-> API
    Sample -.-> IO

    %% Click Events
    click API "https://github.com/fmartinez09/java-wal/blob/main/src/main/java/LogFile.java"
    click Core "https://github.com/fmartinez09/java-wal/blob/main/src/main/java/LogFile.java"
    click Rec "https://github.com/fmartinez09/java-wal/blob/main/src/main/java/LogFile.java"
    click IO "https://github.com/fmartinez09/java-wal/blob/main/src/main/java/LogFile.java"
    click Record "https://github.com/fmartinez09/java-wal/blob/main/src/main/java/Record.java"
    click Pol "https://github.com/fmartinez09/java-wal/blob/main/pom.xml"
    click Build "https://github.com/fmartinez09/java-wal/blob/main/pom.xml"
    click RecTests "https://github.com/fmartinez09/java-wal/blob/main/src/test/java/TestLog.java"
    click Doc "https://github.com/fmartinez09/java-wal/blob/main/README.md"
    click Sample "https://github.com/fmartinez09/java-wal/blob/main/mylog.bin"

    %% Styles
    classDef api fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef core fill:#B0C4DE,stroke:#000,stroke-width:1px;
    classDef io fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px;
    classDef policy fill:#F0E68C,stroke:#000,stroke-width:1px;
    classDef test fill:#E6E6FA,stroke:#000,stroke-width:1px;
    classDef doc fill:#FFDAB9,stroke:#000,stroke-width:1px;
```
