SnowDB
An Embedded Relational Database Engine with LSM-Tree Storage

A self-contained, zero-dependency database implementation featuring a log-structured merge-tree (LSM-Tree) storage engine, SQL query interface, and full ACID transaction support. Built from systems-first principles without external storage libraries, query planners, or concurrency frameworks.

Core Architecture

Storage Engine
Log-structured merge-tree with tiered compaction, utilizing sorted string tables (SSTables) for immutable, versioned data tiers. Implements Bloom-filter-accelerated point lookups, memory-mapped I/O, and sparse file support for space-efficient multi-level storage.

Transaction & Durability
Multi-version concurrency control (MVCC) with snapshot isolation and optimistic conflict detection. Write-ahead logging (WAL) with CRC32 checksum validation ensures crash consistency; atomic state transitions via double-buffered metadata and fsync-guaranteed durability.

SQL Execution
Recursive descent parser supporting SELECT, INSERT, UPDATE, DELETE with expression evaluation, operator precedence, and tuple-based range predicates. Query execution leverages order-preserving key encoding for indexed scans and secondary index resolution.

Technical Specifications

• Storage Format: Custom binary serialization with little-endian encoding, null-terminated string escapes, and length-prefixed cell values
• Indexing: Composite primary keys and secondary indexes using concatenated key prefixes with infinity bounds for range queries
• Concurrency: Lock-free snapshot reads; serialized commit protocol with transaction conflict detection via snapshot timestamp tracking
• Compaction: Background SSTable merging with configurable growth factor and threshold-based triggering
• Recovery: Log replay with torn-write detection; atomic directory sync for metadata consistency

Implementation Highlights

• Zero Dependencies: Standard library only (encoding/binary, syscall, sync)
• Iterator-Based Scanning: Unified iterator interface spanning memory tables, SSTable levels, and merged sorted views
• Power-Safe Durability: Sector-aligned writes, parent-directory fsync, and atomic file replacement semantics
• Schema Management: Dynamic table creation with type-safe cell encoding (int64, variable-length byte strings)

Build

go test ./...        # Verify correctness
go build ./...       # Produce library binary

Use Cases

• Embedded applications requiring transactional persistence without SQLite complexity
• Systems research into storage engine internals and concurrency control
• OLTP workloads with write-heavy access patterns optimized for LSM-Tree architectures

---

Systems programming • Storage engines • Database internals