Architecture
CSharpDB is built in clean layers, each with a single responsibility. From client access down to file I/O.
Layered Architecture
Client & Access Layer
ADO.NET, Client SDK (HTTP/gRPC/Direct), REST API, CLI, MCP Server, VS Code Extension
▼
Application Layer
Admin UI (Blazor), Database class, Collection<T> API
▼
Query Execution
SQL Tokenizer, Parser, Query Planner, Operators, Expression Evaluator
▼
Pipeline & Procedures
PipelineOrchestrator, Sources, Transforms, Destinations, Stored Procedures
▼
Catalog & Indexing
SchemaCatalog, BTreeIndexStore, CachingIndexStore, collection path indexes
▼
Storage Engine
BTree, Pager, RecordEncoder, TransactionCoordinator
▼
Durability & Recovery
WriteAheadLog, WalIndex, CheckpointCoordinator, Snapshot Readers
▼
Device / File I/O
FileStream, LRU PageCache, DictionaryPageCache (in-memory mode)
Page Format (4096 bytes)
Every page uses a slotted format with a fixed header, cell pointer array growing forward, and cell data growing backward from the end of the page.
Page Header
Type, Cell Count, Free Offset
Type, Cell Count, Free Offset
Cell Pointer Array →
Free Space
← Cell Data
Header
Pointers (grow →)
Free space
Cells (grow ←)
WAL File Format
The Write-Ahead Log consists of a 32-byte header followed by a sequence of frames. Each frame contains a 24-byte header and a full 4096-byte page image.
WAL Header (32 bytes)
Magic "CWAL", Version, Page Size, Salts, Checksum Seed
Magic "CWAL", Version, Page Size, Salts, Checksum Seed
Frame Header (24 bytes)
Page Number, DB Page Count, Salt1, Salt2, Checksum
Page Number, DB Page Count, Salt1, Salt2, Checksum
Page Image (4096 bytes)
Frame Header (24 bytes)
Page Image (4096 bytes)
...
...
Transaction Flow
Writer Path
Acquire exclusive lock
Modify pages in memory
Write dirty pages to WAL
Mark commit frame
Update WAL index
Release lock
Checkpoint (if policy triggers)
Reader Path
Capture WAL snapshot
Create read-only pager
Read pages (WAL first, then DB file)
Return consistent results
Release snapshot on dispose
Database Modes
| Mode | Entry Point | Characteristics |
|---|---|---|
| Default | Database.OpenAsync(...) | Standard file-backed open with full durability. |
| In-Memory | Database.OpenInMemoryAsync(...) | Dictionary-based page cache. No file I/O. Fast but non-persistent. |
| Hybrid | Database.OpenHybridAsync(...) | Lazy-resident pages with durable backing file. On-demand loading. |
Project Structure
| Assembly | Purpose |
|---|---|
CSharpDB.Primitives | Shared types: DbValue, DbType, Schema, ErrorCodes |
CSharpDB.Storage | B+tree, Pager, WAL, page cache, serialization |
CSharpDB.Storage.Diagnostics | Database/WAL/Index inspection |
CSharpDB.Sql | SQL tokenizer, parser, AST |
CSharpDB.Execution | Query planner, operators, expression evaluator |
CSharpDB.Engine | Database class, Collection<T>, top-level API |
CSharpDB.Pipelines | ETL orchestrator, sources, transforms, destinations |
CSharpDB.Data | ADO.NET provider |
CSharpDB.Client | Unified client SDK with pluggable transports |
CSharpDB.Api | ASP.NET Core REST API |
CSharpDB.Daemon | gRPC service host |
CSharpDB.Cli | Interactive REPL shell |
CSharpDB.Admin | Blazor Server dashboard |
CSharpDB.Mcp | Model Context Protocol server for AI |
CSharpDB.Native | NativeAOT C library for FFI |