Move Data from Another Database into CSharpDB

Use the CSharpDB migration CLI to capture a source, review the proposed conversion, build a new staged CSharpDB database, resume safely after interruption, and validate the result before activation.

What This Tool Does

The migration workflow deliberately separates source access from target creation:

1. Inspect and capture the source into a retained, digest-pinned package.
2. Plan and preview the schema conversion and review every warning or exclusion.
3. Apply the reviewed plan to a new staged .csdb target. If interrupted, resume the same run.
4. Validate schema, row counts, and checksums. The target is activated only after validation passes.

SQL Server and MySQL credentials are needed only for capture. MySQL then uses the retained package without reconnecting to the source database. Access and SQL Server candidate packages can be reviewed offline, but those two lanes currently stop after capture and catalog review because their non-overrideable release-qualification diagnostics block planning.

Source Lanes and Status

Before You Start

Install the migration release

The migration release is framework-dependent and requires the Microsoft .NET 10 runtime. Extract the archive for your platform and keep its applicable adapter layout intact: adapters/sqlserver and adapters/mysql remain beside the base CLI on every platform, and the Windows archive also includes adapters/access. Verify the downloaded archive against MIGRATION-SHA256SUMS.txt from the same GitHub release before installation.

New-Item -ItemType Directory -Force `
  -Path "$env:LOCALAPPDATA\CSharpDB" | Out-Null

.\install\windows\install-csharpdb-migration-tool.ps1 `
  -InstallDirectory "$env:LOCALAPPDATA\CSharpDB\MigrationTool"

umask 077
mkdir -p "$HOME/.local/lib"

sh install/posix/install-csharpdb-migration-tool.sh \
  --install-dir "$HOME/.local/lib/csharpdb-migration-tool"

The installers use a user-selected directory, refuse to overwrite files by default, do not request administrator access, and do not change PATH. They print optional path setup instructions. You can also run csharpdb directly from the extracted archive. See Downloads for release and source-build options.

Prepare a private migration workspace

• Use a new target path such as staged.csdb. Apply does not overwrite an existing target.
• Keep retained packages in a private directory. They may contain plaintext-sensitive source rows.
• Create the workspace and spill directories before running apply or validation. The CLI requires those roots to exist and verifies their security boundary.
• Record the printed manifestDigest=sha256:... separately in trusted change control. Do not store the package and its only trusted digest together.
• Back up the source independently. The migration tool creates a new CSharpDB target; it is not a source backup system.

New-Item -ItemType Directory -Force `
  -Path .\migration-work, .\migration-spill | Out-Null

Review source readiness

Resolve blocking diagnostics before applying a plan. An accepted exclusion means you have reviewed a source object the tool will not reproduce automatically and have a separate disposition for it. Prefer accepting specific exclusion or diagnostic IDs; do not approve all without reviewing the catalog.

The Four-Step Workflow

1. Inspect and Capture

Choose the matching recipe below. A successful retained capture writes both catalog.json and a source package, then prints a manifest digest. Publication is create-new: an existing catalog or package is not replaced.

# Example: capture SQLite
csharpdb migrate inspect `
  --source sqlite `
  --input .\source.db `
  --package .\source.csdbsqlite `
  --out .\catalog.json

# Copy the exact value printed by the successful command.
$digest = 'sha256:<64-lowercase-hex>'

2. Plan and Preview

Build a deterministic plan from the captured catalog, then inspect both the exact target DDL and the isolated scratch result. Add only the specific reviewed exclusion or diagnostic IDs required by your catalog.

csharpdb migrate plan .\catalog.json --out .\plan.json

# Human-readable summary
csharpdb migrate preview .\plan.json --catalog .\catalog.json

# Exact target DDL and collection actions
csharpdb migrate preview .\plan.json --catalog .\catalog.json --ddl

# Execute the proposed schema in a new isolated scratch database
csharpdb migrate preview .\plan.json --catalog .\catalog.json --scratch

3. Apply or Resume

Apply the reviewed plan to a new staged target. The package extension must match the source recipe. The expected manifest digest must be the independently recorded value from capture.

$package = '.\source.csdbsqlite'

csharpdb migrate apply .\plan.json `
  --catalog .\catalog.json `
  --source-package $package `
  --expected-manifest-digest $digest `
  --workspace .\migration-work `
  --target .\staged.csdb `
  --out .\run.json

Apply commits rows and target receipts together. A completed apply stops at awaitingValidation; it does not silently replace an existing production database.

4. Validate and Activate

Checksum validation reopens the same retained package and compares normalized schema, 64-bit row counts, and partitioned canonical SHA-256 evidence. A passing validation activates the staged target. Failed or inconclusive validation withholds activation.

csharpdb migrate validate .\plan.json `
  --catalog .\catalog.json `
  --source-package $package `
  --expected-manifest-digest $digest `
  --workspace .\migration-work `
  --target .\staged.csdb `
  --out .\validation.json `
  --level checksum `
  --spill-dir .\migration-spill

Keep the plan, catalog, package digest, run report, validation report, and reviewed exclusions with the migration change record.

Source Recipes

For CSV, JSON/NDJSON, SQLite, LiteDB, and MySQL, use the common plan, preview, apply, and validation steps above after a successful capture. Access and SQL Server currently stop after candidate capture and catalog review; their qualification diagnostics cannot be accepted or overridden.

CSV

csharpdb migrate inspect `
  --source csv `
  --input .\source.csv `
  --package .\source.csdbcsv `
  --out .\catalog.json `
  --delimiter auto

CSV uses strict tabular inference. Add --no-header only when the source has no header row. You can explicitly select comma, semicolon, tab, pipe, or one delimiter character.

JSON or NDJSON

# Root JSON array
csharpdb migrate inspect `
  --source json `
  --input .\source.json `
  --framing root-array `
  --package .\source.csdbjson `
  --out .\catalog.json

# Newline-delimited JSON
csharpdb migrate inspect `
  --source json `
  --input .\source.ndjson `
  --framing ndjson `
  --package .\source.csdbjson `
  --out .\catalog.json

Use an explicitly selected, digest-pinned typed-intent sidecar when inference is not sufficient. The CLI does not automatically discover or author that contract.

SQLite

csharpdb migrate inspect `
  --source sqlite `
  --input .\source.db `
  --package .\source.csdbsqlite `
  --out .\catalog.json

The automatic row route is limited to supported ordinary UTF-8 rowid tables. Views, triggers, virtual tables, WITHOUT ROWID, generated columns, and other unsupported shapes remain visible as blocking diagnostics or reviewed exclusions.

LiteDB 5

# Close every LiteDB writer before capture.
csharpdb migrate inspect `
  --source litedb `
  --input .\source.db `
  --package .\source.csdblitedb `
  --out .\catalog.json

The v1 route requires an offline, quiesced, unencrypted LiteDB 5 database. Collections become CSharpDB document collections. Non-built-in LiteDB indexes are inventoried but excluded from automatic creation; review and recreate required indexes with CSharpDB semantics.

Microsoft Access

Access capture is a Windows-only optional adapter and requires a bitness-compatible Microsoft Access Database Engine. ACE 16 is the default; ACE 12 must be selected explicitly. Jet is never used. Close the source in Access and every other writer before capture.

csharpdb migrate inspect `
  --source access `
  --input .\legacy.accdb `
  --provider ace16 `
  --package .\source.csdbaccess `
  --out .\catalog.json

The retained route inventories supported local tables, columns, primary keys, and relationships, and reads rows in deterministic primary-key order. Linked tables, saved queries, forms, reports, macros, VBA, encrypted files, and unsupported scalar shapes are not silently translated. The catalog always contains MIG-ACCESS-LIVE-QUALIFICATION-PENDING-001, which blocks plan and apply until the disposable Windows VM matrix covers the supported .mdb/.accdb, ACE, and process-bitness combinations.

SQL Server

Put the connection string in a protected process environment variable or secret manager. It must select exactly one source database through Initial Catalog. For candidate capture, use a dedicated no-write account that can connect to that database, read the candidate tables, and view their definitions. Enable ALLOW_SNAPSHOT_ISOLATION for the database and confirm snapshot_isolation_state = ON before capture. These are capture-only prerequisites, not an apply-ready permission recipe. Pass only the environment variable name to the CLI:

csharpdb migrate inspect `
  --source sqlserver `
  --connection-env CSHARPDB_SQLSERVER_SOURCE `
  --package .\source.csdbsqlserver `
  --out .\catalog.json `
  --table-timeout-seconds 1800

The retained candidate lanes are on-premises SQL Server 2019, 2022, and 2025. Azure SQL Database, Azure SQL Managed Instance, Synapse, Fabric, and other compatible services require separate qualification. The source credential is not needed for offline package evaluation after capture.

MySQL

Use a connection string that selects exactly one source database through Database, and a dedicated read-only account with direct schema-level SELECT on that database's ordinary base tables. Retained v1 does not require TRIGGER, EXECUTE, or SHOW VIEW. TCP connections require TLS; place the reviewed TLS settings in the protected connection environment value.

csharpdb migrate inspect `
  --source mysql `
  --connection-env CSHARPDB_MYSQL_SOURCE `
  --package .\source.csdbmysql `
  --out .\catalog.json `
  --table-timeout-seconds 1800

The source credential is needed only for capture. Apply, resume, and validation reopen source.csdbmysql without reconnecting to MySQL.

Review Type Mappings and Query Compatibility

These commands are read-only report generators. They do not change a catalog, apply a rewrite, connect to the source, or create a target.

Data type mapping report

# Use the same preserve-first mapper used by migration planning.
csharpdb migrate type-map .\catalog.json `
  --profile preserve `
  --format text `
  --out .\type-map.txt

# Machine-readable report for review automation.
csharpdb migrate type-map .\catalog.json `
  --profile queryable `
  --format json `
  --out .\type-map.json

The report covers every typed catalog object and classifies the selected representation as exact, lossless re-encoding, lossy, or unsupported. A custom profile requires an explicit per-object mapping file. The file is one strict JSON object whose property names are exact catalog objectId values and whose string values are the case-insensitive persistent CSharpDB types integer, real, text, or blob:

{
  "<exact-column-object-id-from-catalog>": "text",
  "<another-exact-column-object-id>": "real"
}

Use the IDs printed in the catalog or type-map report. Unknown IDs, duplicate JSON properties, invalid type names, comments, and trailing commas are rejected. Generating a report does not approve a lossy conversion.

DDL compatibility check

# Native CSharpDB DDL
csharpdb migrate ddl-check .\schema.sql `
  --dialect csharpdb `
  --format json

# Bounded T-SQL lowering through the optional SQL Server worker
csharpdb migrate ddl-check .\schema.sql `
  --dialect tsql `
  --format text

The CSharpDB route checks one bounded UTF-8 script against the embedded, version-matched capability catalog and can prove supported persistent table and index actions. The T-SQL route runs behind the isolated SQL Server worker and reports bounded lowering candidates and unsupported constructs. It does not connect to SQL Server, execute against production, or automatically apply the generated CSharpDB DDL.

Static query check

# Native CSharpDB SQL
csharpdb migrate query-check .\query.sql `
  --dialect csharpdb `
  --out .\query-check.txt

# SQL Server 2019 grammar (compatibility level 150)
csharpdb migrate query-check .\query.sql `
  --dialect tsql `
  --compatibility-level 150 `
  --format json `
  --out .\query-check.json

The CSharpDB checker and optional SQL Server worker establish bounded parse-level evidence for one read-only statement. The only generated T-SQL candidate rewrite is a root integer TOP to CSharpDB LIMIT; it is never applied automatically. A Conditional result does not prove schema binding, execution, or equivalent results. MySQL, SQLite, and Access query dialects currently return Unknown rather than borrowing a different parser.

EF Core migration chain analysis

The EF Core analyzer is a separate .NET tool so design-time dependencies and trusted application loading do not enter the base CLI:

dotnet tool install --global CSharpDB.EntityFrameworkCore.Tools `
  --version 4.3.0

dotnet csharpdb-ef analyze `
  --project .\MyApp.csproj `
  --context MyApp.Data.AppDbContext `
  --scratch `
  --format json

The default generation tier compiles the restored net10.0 migration chain with the real CSharpDB provider, inspects Up/Down operations, generates provider SQL, and returns Conditional bound evidence. --scratch additionally executes every supported chain prefix, down/up step, and guarded replay only against tool-owned in-memory CSharpDB databases. A scratch pass proves the empty-database chain—not production data conversion, persistence, application deployment code, or configured-database safety. Treat the selected project as executable code. See the EF Core guide for normal provider setup.

Read-only Dual-run Validation

The CSharpDB.Migration.DualRun SDK compares a reviewed query pack against a read-only source snapshot and the staged CSharpDB target. It uses typed parameters, the same versioned canonical value encoding used by migration validation, ordered or duplicate-preserving unordered comparison, bounded resource limits, and digest-verified reports that omit SQL and data values.

dotnet add package CSharpDB.Migration.DualRun --version 4.3.0

Dual-run is additional cutover evidence, not an import path and not a replacement for schema, row-count, and checksum validation. Both endpoint snapshot identities and the canonicalization contract must be recorded. Any provider error, timeout, safety rejection, incoherent snapshot, or resource-limit exhaustion is Inconclusive, never a pass. The generic ADO.NET source route requires both an exact reviewed query allowlist and a provider-enforced read-only connection.

Version 1 retains bounded row digests in memory and does not spill dual-run comparison state to disk. Set row and canonical-byte limits for the available memory. A provider can materialize a field before the SDK checks its cell limit, so this API is not a process-isolation boundary for untrusted providers or unrestricted query text.

Export CSharpDB to CSV or JSON

Export is a two-command, offline workflow. First close every writer and capture the normal CSharpDB database into an immutable retained snapshot. The capture result contains the exact snapshotIdentity required by the export command, so no SDK code or separate hash calculation is needed.

New-Item -ItemType Directory -Force .\snapshot-work | Out-Null

$capture = csharpdb migrate snapshot .\app.csdb `
  --out .\app.export-snapshot.db `
  --offline `
  --workspace .\snapshot-work `
  --json | ConvertFrom-Json

# Lossless CSV plus its source-bound manifest
csharpdb migrate export .\app.export-snapshot.db `
  --format csv `
  --table users `
  --out .\users.csv `
  --manifest .\users.csv.manifest.json `
  --expected-snapshot-identity $capture.snapshotIdentity

# Or one compact JSON root array. Use --format ndjson for one object per line.
csharpdb migrate export .\app.export-snapshot.db `
  --format json `
  --table users `
  --out .\users.json `
  --manifest .\users.json.manifest.json `
  --expected-snapshot-identity $capture.snapshotIdentity

• --offline is an explicit confirmation that the source is quiesced. Capture never opens or recovers the source for writing; recovery happens only on a bounded private copy.
• The retained snapshot and export files use create-new/no-overwrite publication. Choose new paths, keep the snapshot identity with the export record, and repeat the same export command to resume an interrupted run.
• Export operates on one physical table per command. Repeat the command for each required table.
• Retained snapshot capture is portable. Resumable CSV/JSON publication currently requires a trusted local Windows directory. Other platforms fail with MIG-EXPORT-PLATFORM-001 because an equivalent handle-bound, atomic no-replace publication substrate is not yet qualified.
• CSV defaults to lossless-v1. The optional spreadsheet-safe-lossy-v1 CSV profile intentionally changes formula-like text and must be selected explicitly. JSON and NDJSON support only lossless-v1.

Recover an Interrupted Run

Repeat the exact apply binding—same plan, catalog, package, expected digest, and staged target—then add --resume:

csharpdb migrate apply .\plan.json `
  --catalog .\catalog.json `
  --source-package $package `
  --expected-manifest-digest $digest `
  --workspace .\migration-work `
  --target .\staged.csdb `
  --out .\resume.json `
  --resume

• The receipts stored in the staged target are the recovery authority.
• The workspace copy and run.json are not authoritative checkpoints. A fresh private workspace copy is created and requalified when resuming.
• Resume skips only batches whose target receipts exactly match the package and plan binding.
• A changed package, wrong digest, changed catalog, changed plan, or mismatched target fails closed instead of guessing.
• If the staged target was never created, run the original apply command again without --resume.

Protect Credentials and Retained Data

• Never put a SQL Server or MySQL connection string directly in command text. Use --connection-env with the name of a protected environment variable.
• Treat .csdbaccess, .csdbsqlserver, and .csdbmysql as plaintext-sensitive source data. Apply source-equivalent access, retention, backup, and deletion controls.
• Store the expected package digest in an independently trusted record. If an attacker can replace both a package and its pin, the pin provides no protection.
• Create package and catalog outputs beneath existing, normalized local directories owned by the current user. The capture command rejects links, remote filesystems, directories with untrusted write/delete authority, and directory identity changes during publication.
• Use a caller-controlled local workspace with private permissions. Do not put retained packages, workspaces, reject evidence, or spill files in a shared directory.
• Delete credentials from the capture process environment and remove retained packages according to the approved retention policy after migration evidence is complete.

Current Limits and Qualification Status

• Access: retained v1 supports bounded local-table capture through ACE on Windows, but its required VM matrix is deferred and the qualification diagnostic blocks planning.
• SQL Server: retained v1 implements a bounded relational schema and scalar row capture subset. The strict restricted-account metadata proof passes in the LocalDB qualification fixture, while the broader live release matrix remains a blocking diagnostic. Unsupported features remain blocking diagnostics or reviewed exclusions.
• MySQL: retained v1 is limited to supported ordinary, nonpartitioned InnoDB base tables with deterministic integer key ordering and supported scalar columns. Programmable objects are outside this retained migration scope.
• Live test matrix: disposable Windows VM and broader live-provider qualification remain deferred. Do not interpret a successful offline package test as proof for every server, authentication, or TLS configuration.
• Source products: MariaDB, Aurora MySQL, Percona, TiDB, Vitess, and other MySQL-compatible products require separate qualification.
• Package size: provider captures default to bounded ceilings. Lower the ceiling with --max-source-bytes when your change policy requires a smaller limit.

Troubleshooting