How fgrosse/wal works
Project Type & Purpose
• This is a Go library (not a full-stack app) providing a Write-Ahead Log implementation.
• It exposes a WAL abstraction that clients instantiate, write entries to, and close.
Main Components
a. WAL Core (entry.go, wal.go, configuration.go, entry_registry.go)
b. Segment Writer (segment_writer.go)
c. Segment Reader (segment_reader.go)
d. Entry Registry (entry_registry.go + waltest for examples)
e. WAL Segment Files (on disk: *.wal under configurable path)
f. External Dependencies (os filesystem, zap.Logger, CRC library)
Relationships & Data Flow
• Client code calls wal.New(path, config, entryRegistry, logger).
→ WAL Core loads existing segment files (Segment Reader) and initializes state.
• Client calls WAL.Write(entry):
→ WAL Core serializes entry metadata (offset, type, CRC, payload) via Segment Writer
→ Writes binary frame to current segment file
→ Optionally calls fsync (SyncDelay / fsync config)
→ Returns offset
• When segment file reaches max size:
→ WAL Core rotates to new segment (closes old writer, opens new one)
• Client calls WAL.Close(): flushes and closes current segment writer.
• On restart, WAL Core enumerates segment files on disk and uses Segment Reader to scan/verify entries, delegating binary decoding to Entry Registry.
• Entry Registry maps byte Type codes ←→ Go Entry instances (factory pattern).
Architectural Patterns & Principles
• Single Responsibility: Reader, Writer, Registry, Core orchestration each in separate packages/files.
• Factory/Registry Pattern: entry_registry.go to map binary types to concrete Entry implementations.
• Append-only log segmentation: segment rotation simplifies clean-up and backup.
• Pluggable logging: zap.Logger injection for structured logs.
• Durability configuration via SyncDelay to batch fsyncs.