Storage Engine
The low-level storage layer: B+trees, paging, transactions, WAL, checkpointing, serialization, and diagnostics.
Opening a Database
The public low-level entry point is DefaultStorageEngineFactory, which opens a file-backed StorageEngineContext. In-memory and hybrid modes are exposed through Database.OpenInMemoryAsync() and Database.OpenHybridAsync() on the engine layer.
using CSharpDB.Storage.StorageEngine;
var storageOptions = new StorageEngineOptionsBuilder()
.UseBTreeIndexes()
.Build();
var factory = new DefaultStorageEngineFactory();
await using var ctx = await factory.OpenAsync("myapp.db", storageOptions);StorageEngineContext
| Property | Type | Purpose |
|---|---|---|
Pager | Pager | Page I/O, transaction control, snapshot readers |
Catalog | SchemaCatalog | Table/index/view/trigger management |
RecordSerializer | IRecordSerializer | Row encoding/decoding |
IndexProvider | IIndexProvider | Secondary index creation and lookup |
Transactions
CSharpDB uses a single-writer, multiple-reader concurrency model with snapshot isolation.
Writer Transactions
Only one write transaction can be active at a time. The writer acquires an exclusive lock and all page modifications are logged to the WAL before commit.
await ctx.Pager.BeginTransactionAsync();
try
{
var tree = ctx.Catalog.GetTableTree("Events");
await tree.InsertAsync(1, payload);
await ctx.Pager.CommitAsync();
}
catch
{
await ctx.Pager.RollbackAsync();
throw;
}Snapshot Readers
Readers see a consistent point-in-time snapshot. They never block the writer and the writer never blocks readers.
var snapshot = ctx.Pager.AcquireReaderSnapshot();
try
{
await using var reader = ctx.Pager.CreateSnapshotReader(snapshot);
var readCatalog = await SchemaCatalog.CreateAsync(
reader,
ctx.SchemaSerializer,
ctx.IndexProvider);
var tree = readCatalog.GetTableTree("Users");
byte[]? data = await tree.FindAsync(42);
}
finally
{
ctx.Pager.ReleaseReaderSnapshot();
}B+Tree & Cursors
Each table's data is stored in a B+tree keyed by long rowid. Interior nodes hold routing keys and child pointers; leaf nodes hold key-payload pairs linked for sequential scans.
using var cursor = tree.CreateCursor();
while (await cursor.MoveNextAsync())
{
var row = encoder.Decode(cursor.CurrentValue.Span);
Console.WriteLine($"Key {cursor.CurrentKey}: {row[1].AsText}");
}Schema Catalog
The catalog stores all schema metadata in a dedicated B+tree. It manages tables, indexes, views, and triggers.
// Create a table
await ctx.Catalog.CreateTableAsync(schema);
// Create an index
await ctx.Catalog.CreateIndexAsync(indexSchema);
// Query schema
var tableSchema = ctx.Catalog.GetTable("Users");
var tree = ctx.Catalog.GetTableTree("Users");
// Drop
await ctx.Catalog.DropTableAsync("Users");Serialization
The RecordEncoder uses varint-encoded type markers and lengths for space-efficient storage. Each cell in a slotted page contains one encoded record.
var encoder = ctx.RecordSerializer;
// Encode
byte[] bytes = encoder.Encode([
DbValue.FromInteger(42),
DbValue.FromText("Alice"),
]);
// Decode
DbValue[] row = encoder.Decode(bytes, schema);Write-Ahead Log
All page modifications are written to the WAL before being applied to the main database file. The WAL uses a 32-byte header and 24-byte frame headers with checksum validation.
Checkpoint Policies
| Policy | Triggers When |
|---|---|
FrameCountCheckpointPolicy | N frames committed (default: 1000) |
TimeIntervalCheckpointPolicy | N seconds elapsed since last checkpoint |
WalSizeCheckpointPolicy | WAL exceeds size threshold in bytes |
AnyCheckpointPolicy | Any sub-policy triggers (composite) |
ManualCheckpointPolicy | Only when explicitly requested |
Configuration
var options = new StorageEngineOptionsBuilder()
.WithCacheSize(4096) // Page cache slots
.WithLockTimeout(TimeSpan.FromSeconds(30)) // Write lock timeout
.WithCheckpointPolicy(
new AnyCheckpointPolicy(
new FrameCountCheckpointPolicy(1000),
new WalSizeCheckpointPolicy(10 * 1024 * 1024)
))
.Build();Diagnostics
Three inspector classes validate database integrity at different levels:
| Inspector | Validates |
|---|---|
DatabaseInspector | File header, page structure, B+tree consistency, page histograms |
WalInspector | WAL header, frame salts, checksums for each committed frame |
IndexInspector | Index root page validity, table/column references, B+tree reachability |