Chapter 17 · SQL Dialects, Tools, and Application Access

Migrations, Seed Data, and Database Changes in Git

A database schema is shared, stateful infrastructure. Changing model code is not enough: teams need an ordered history that transforms every supported database safely, records what ran, survives concurrent deployments, and can be reviewed like production code.

Intermediate170–210 minutesMigration engineering + delivery capstoneLast reviewed: August 2026

Learning outcomes

Learning outcomes

01

Treat migrations as an ordered, immutable transformation history rather than generated deployment side effects.

02

Design expand-and-contract changes compatible with rolling application deployments.

03

Separate reference, development, demonstration, and test seed data.

04

Track applied revisions, checksums, ownership, and execution evidence.

05

Review and test database changes in Git and CI, including fresh install, upgrade, rollback or forward-fix, and drift detection.

Desired model versus transformation history

MOD

Model state

ORM metadata or schema files describe what developers want now.

MIG

Migration history

Ordered revisions describe how existing databases reach that state.

RUN

Applied state

A version table records which revisions actually ran in an environment.

DRF

Drift

Manual or failed changes create differences not explained by versioned history.

Edit model + migration
Review SQL and compatibility
Test fresh + upgrade paths
Merge immutable revision
Deploy once under lock
Verify schema + application

Never assume that rebuilding an empty test database proves an upgrade is safe for real data.

Migration contract

PropertyRequired behavior
Unique identityEvery revision has a stable ID and parent/dependency relation
OrderingThe tool computes a deterministic path from current to target revision
ImmutabilityAn applied revision is not silently edited; a new corrective revision is added
AtomicityUse a transaction where the engine and operation support it
Idempotent deploymentThe orchestrator applies each revision once and handles concurrent deployers
ObservabilityRecord revision, checksum, actor/build, timestamps, duration, and outcome
CompatibilitySchema and application versions overlap safely during rollout
RecoveryDefine rollback when safe; otherwise define tested forward repair

A minimal migration ledger

sqlite · migration metadata
CREATE TABLE IF NOT EXISTS schema_migration (    revision_id TEXT PRIMARY KEY,    parent_revision TEXT,    checksum_sha256 TEXT NOT NULL,    description TEXT NOT NULL,    applied_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,    applied_by TEXT NOT NULL,    duration_ms INTEGER NOT NULL CHECK(duration_ms >= 0)) STRICT;SELECT revision_id, parent_revision, checksum_sha256, applied_atFROM schema_migrationORDER BY applied_at, revision_id;

Expand, migrate, contract

PhaseDatabase changeApplication behavior
ExpandAdd nullable column/table/index or compatible objectOld and new application versions continue to work
Dual compatibilityOptionally write/read both representationsTelemetry verifies the new path
BackfillMove existing data in bounded restartable batchesNormal traffic remains available
EnforceAdd validated constraints and switch readsNew version becomes authoritative
ContractRemove obsolete columns or compatibility codeOnly after old application versions are gone
sql · expand phase
ALTER TABLE customer ADD COLUMN normalized_email TEXT;CREATE INDEX idx_customer_normalized_email    ON customer(normalized_email);-- Backfill in bounded batches in a separate operational job.UPDATE customerSET normalized_email = lower(trim(email))WHERE normalized_email IS NULL;-- Contract only after every deployed version uses normalized_email.-- ALTER TABLE customer ALTER COLUMN normalized_email SET NOT NULL;-- ALTER TABLE customer DROP COLUMN legacy_email;

Alembic revision example

python · versioned upgrade and downgrade
from alembic import opimport sqlalchemy as sarevision = "20260805_01"down_revision = "20260730_03"branch_labels = Nonedepends_on = Nonedef upgrade() -> None:    op.add_column(        "customer",        sa.Column("normalized_email", sa.String(320), nullable=True),    )    op.create_index(        "idx_customer_normalized_email",        "customer",        ["normalized_email"],        unique=False,    )def downgrade() -> None:    op.drop_index("idx_customer_normalized_email", table_name="customer")    op.drop_column("customer", "normalized_email")

Autogeneration is a draft. Review names, data movement, locking, defaults, constraint validation, server-version behavior, and downgrade safety.

Seed data taxonomy

Seed categoryExamplesPolicy
Reference dataCountry codes, workflow states, permission definitionsVersioned, deterministic, stable keys, safe upsert
Bootstrap identityInitial service role or tenantSecret supplied externally; rotate immediately
Development fixturesSynthetic customers and ordersNever deployed to production
Demonstration dataCurated product tour datasetClearly labeled and removable
Test fixtures/factoriesMinimal edge-case recordsOwned by tests; isolated per run
Backfill dataDerived values for a schema transitionMigration or dedicated resumable job, not permanent seed
sql · deterministic reference seed
INSERT INTO order_status(status_code, display_name, terminal)VALUES  ('draft','Draft',0),  ('submitted','Submitted',0),  ('paid','Paid',1),  ('cancelled','Cancelled',1)ON CONFLICT(status_code) DO UPDATE SET  display_name = excluded.display_name,  terminal = excluded.terminal;

Database changes in Git

text · repository structure
db/  migrations/    20260805_01_add_normalized_email.py    20260807_01_backfill_normalized_email.py    20260812_01_enforce_normalized_email.py  seeds/    reference_order_status.sql  checks/    verify_normalized_email.sql  README.md  OWNERS
Pull-request questionEvidence
What is the data and lock impact?Plan, table size, estimated duration, batch design
Can old and new app versions coexist?Compatibility matrix and rollout order
How is failure detected?Metrics, logs, revision status, verification query
How is it recovered?Rollback or forward-fix procedure and backup dependency
Was generated SQL reviewed?Offline SQL artifact or migration logs
Does it work from every supported state?Fresh install, previous release upgrade, representative data tests
Who owns approval?Database/platform owner plus application owner

CI migration matrix

bash · representative pipeline
set -euo pipefail# 1. Build an empty database from the complete revision history.alembic upgrade headalembic current --check-headspytest tests/database# 2. Upgrade a copy representing the previous production release.restore_fixture previous_release.dbalembic upgrade headpython db/checks/verify_schema.pypytest tests/integration# 3. Render SQL for review where supported.alembic upgrade base:head --sql > migration-plan.sqlgit diff --exit-code -- migration-plan.sql || true

Migration review

  1. Why must applied migrations be immutable?
  2. Why is a nullable additive column often safer than an immediate required column?
  3. Why are large data backfills frequently separated from schema revisions?
  4. What does fresh-install testing fail to prove?
Review the answers

Editing history destroys reproducibility and checksum trust. Additive nullable changes allow old and new versions to coexist while data is populated. Backfills need batching, restartability, throttling, and separate monitoring. Fresh installs do not expose upgrade-time data conversion, locking, volume, or compatibility problems.

SQLite migration exercise

python · ordered transactional runner
import hashlibimport sqlite3from pathlib import PathMIGRATIONS = [    ("001", "create product", "CREATE TABLE product(id INTEGER PRIMARY KEY, sku TEXT UNIQUE)"),    ("002", "add active", "ALTER TABLE product ADD COLUMN active INTEGER NOT NULL DEFAULT 1"),]with sqlite3.connect("academy.db") as db:    db.execute("""CREATE TABLE IF NOT EXISTS schema_migration(        revision_id TEXT PRIMARY KEY,        checksum_sha256 TEXT NOT NULL,        description TEXT NOT NULL,        applied_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP    )""")    applied = {row[0]: row[1] for row in db.execute(        "SELECT revision_id, checksum_sha256 FROM schema_migration"    )}    for revision, description, sql in MIGRATIONS:        checksum = hashlib.sha256(sql.encode()).hexdigest()        if revision in applied:            if applied[revision] != checksum:                raise RuntimeError(f"edited applied migration: {revision}")            continue        db.execute(sql)        db.execute(            "INSERT INTO schema_migration(revision_id,checksum_sha256,description) VALUES(?,?,?)",            (revision, checksum, description),        )

Chapter summary

  • Version both the desired schema and the ordered path that changes existing databases.
  • Use expand-and-contract for rolling compatibility and isolate large data movement.
  • Keep reference seeds deterministic and production-safe; keep test/demo data separate.
  • Make migration evidence, ownership, review, and verification part of the Git delivery workflow.
  • Chapter 18 applies the complete course in a design-and-query capstone.

References

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only assets compatible with the Ethereum/ERC-20 network. Do not send TRC-20/TRON assets.