How proton02/mini-sql works
Project Type & Purpose
• Type: A standalone, C++-based relational database engine (MiniSQL), inspired by CMU’s BusTub.
• Purpose: Parse SQL, plan queries, execute against on-disk data with buffering, indexing, concurrency control, and recovery.
File Structure
• Top-level src/ modules map directly to core architectural layers:
– parser/
– planner/
– executor/
– buffer/ (Buffer Pool Manager)
– storage/ & page/ (Disk Manager & on-disk page formats)
– catalog/ (metadata)
– record/ (row, schema, fields)
– index/ (B+Tree index manager)
– concurrency/ (Lock Manager, Txn Manager)
– recovery/ (Log & Recovery Manager)
• Supporting dirs: common/ (config, singletons), utils/, test suites, third-party libs (glog, gtest).
Extract Additional Insights from README
• Core features: persistent buffer pool, index, record modules; on-disk bitmap and metadata pages; serialization for persistence; AST support in parser.
• Build & test instructions confirm separation of core engine and test harness.
Main Components & Their Interactions
A. Front-end / SQL Interface
• Component: SQL CLI (in main.cpp)
• Output: Raw SQL string → feeds into Parser.
B. Parser Layer
• Component: parser/
• Role: Lex & parse SQL → produce AST.
• Interaction: feeds AST into Planner.
C. Planner Layer
• Component: planner/
• Role: Convert AST → Logical/Physical Plan.
• Interaction: sends Plan tree to Executor Engine.
D. Execution Layer
• Component: executor/execute_engine.cpp + Executors (seq_scan, index_scan, insert, update, delete, values).
• Role: Walk Plan tree, perform operations.
• Interaction: invokes Catalog, Buffer Pool, Index, Record, Concurrency, Recovery.
E. Catalog & Metadata
• Component: catalog/
• Role: Store table/index metadata in persistent catalog; provide schema and index descriptors.
F. Record & Index Managers
• Record Manager: (record/) handles tuples, schemas, serialization.
• Index Manager: (index/, B+Tree) manages secondary data structures.
G. Storage & Buffering
• Disk Manager (storage/disk_manager.cpp): read/write raw pages.
• Buffer Pool Manager (buffer/buffer_pool_manager.cpp): in-memory cache, LRU/Clock replacer.
• Page Formats (page/): table pages, header page, bitmap page, B+Tree pages.
H. Concurrency Control
• Lock Manager (concurrency/lock_manager.cpp), Transaction Manager (txn_manager.cpp): manage locks, isolation.
I. Logging & Recovery
• Log Manager, Log Record, Recovery Manager (recovery/): write-ahead logging, crash recovery.
Relationships & Data Flows
– SQL CLI → Parser → Planner → Executor Engine
– Executor ↔ Catalog (fetch metadata)
– Executor ↔ Buffer Pool → Disk Manager
– Executor ↔ Record Manager & Index Manager
– Executor ↔ Concurrency Module & Recovery Module
Architectural Patterns & Design Principles
• Layered design: clear separation of concerns (Parsing → Planning → Execution → Storage).
• Modular managers: single-responsibility for buffer, disk, catalog, etc.
• Interfaces & abstract executors/plans: use of abstract classes for executors, plans, replacers.
• Singleton (common/singleton.h) for global components (Config, Instance).
Relevant Technologies & Frameworks
• C++17 (assumed), CMake build system.
• flex & bison for lexical and parsing (minisql.l, minisql.y).
• Google Logging (glog) and Google Test (gtest) for tests.
Final Checklist
– [ ] All major modules (Parser, Planner, Executor, Buffer Pool, Disk, Catalog, Record, Index, Concurrency, Recovery) represented.
– [ ] Interfaces & abstract base classes noted where appropriate.
– [ ] Persistence & serialization paths clearly shown.