Chapter 17 · SQL Dialects, Tools, and Application Access

Migrations, Seed Data, and Database Changes in Git

A database schema is shared, stateful infrastructure. Changing model code is not enough: teams need an ordered history that transforms every supported database safely, records what ran, survives concurrent deployments, and can be reviewed like production code.

Intermediate170–210 minutesMigration engineering + delivery capstoneLast reviewed: August 2026

Learning outcomes

Learning outcomes

01

Treat migrations as an ordered, immutable transformation history rather than generated deployment side effects.

02

Design expand-and-contract changes compatible with rolling application deployments.

03

Separate reference, development, demonstration, and test seed data.

04

Track applied revisions, checksums, ownership, and execution evidence.

05

Review and test database changes in Git and CI, including fresh install, upgrade, rollback or forward-fix, and drift detection.

Desired model versus transformation history

MOD

Model state

ORM metadata or schema files describe what developers want now.

MIG

Migration history

Ordered revisions describe how existing databases reach that state.

RUN

Applied state

A version table records which revisions actually ran in an environment.

DRF

Drift

Manual or failed changes create differences not explained by versioned history.

Edit model + migration
Review SQL and compatibility
Test fresh + upgrade paths
Merge immutable revision
Deploy once under lock
Verify schema + application

Never assume that rebuilding an empty test database proves an upgrade is safe for real data.

Migration contract

Property Required behavior
Unique identity Every revision has a stable ID and parent/dependency relation
Ordering The tool computes a deterministic path from current to target revision
Immutability An applied revision is not silently edited; a new corrective revision is added
Atomicity Use a transaction where the engine and operation support it
Idempotent deployment The orchestrator applies each revision once and handles concurrent deployers
Observability Record revision, checksum, actor/build, timestamps, duration, and outcome
Compatibility Schema and application versions overlap safely during rollout
Recovery Define rollback when safe; otherwise define tested forward repair

A minimal migration ledger

sqlite · migration metadata
CREATE TABLE IF NOT EXISTS schema_migration (    revision_id TEXT PRIMARY KEY,    parent_revision TEXT,    checksum_sha256 TEXT NOT NULL,    description TEXT NOT NULL,    applied_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,    applied_by TEXT NOT NULL,    duration_ms INTEGER NOT NULL CHECK(duration_ms >= 0)) STRICT;SELECT revision_id, parent_revision, checksum_sha256, applied_atFROM schema_migrationORDER BY applied_at, revision_id;

Expand, migrate, contract

Phase Database change Application behavior
Expand Add nullable column/table/index or compatible object Old and new application versions continue to work
Dual compatibility Optionally write/read both representations Telemetry verifies the new path
Backfill Move existing data in bounded restartable batches Normal traffic remains available
Enforce Add validated constraints and switch reads New version becomes authoritative
Contract Remove obsolete columns or compatibility code Only after old application versions are gone
sql · expand phase
ALTER TABLE customer ADD COLUMN normalized_email TEXT;CREATE INDEX idx_customer_normalized_email    ON customer(normalized_email);-- Backfill in bounded batches in a separate operational job.UPDATE customerSET normalized_email = lower(trim(email))WHERE normalized_email IS NULL;-- Contract only after every deployed version uses normalized_email.-- ALTER TABLE customer ALTER COLUMN normalized_email SET NOT NULL;-- ALTER TABLE customer DROP COLUMN legacy_email;

Alembic revision example

python · versioned upgrade and downgrade
from alembic import opimport sqlalchemy as sarevision = "20260805_01"down_revision = "20260730_03"branch_labels = Nonedepends_on = Nonedef upgrade() -> None:    op.add_column(        "customer",        sa.Column("normalized_email", sa.String(320), nullable=True),    )    op.create_index(        "idx_customer_normalized_email",        "customer",        ["normalized_email"],        unique=False,    )def downgrade() -> None:    op.drop_index("idx_customer_normalized_email", table_name="customer")    op.drop_column("customer", "normalized_email")

Autogeneration is a draft. Review names, data movement, locking, defaults, constraint validation, server-version behavior, and downgrade safety.

Seed data taxonomy

Seed category Examples Policy
Reference data Country codes, workflow states, permission definitions Versioned, deterministic, stable keys, safe upsert
Bootstrap identity Initial service role or tenant Secret supplied externally; rotate immediately
Development fixtures Synthetic customers and orders Never deployed to production
Demonstration data Curated product tour dataset Clearly labeled and removable
Test fixtures/factories Minimal edge-case records Owned by tests; isolated per run
Backfill data Derived values for a schema transition Migration or dedicated resumable job, not permanent seed
sql · deterministic reference seed
INSERT INTO order_status(status_code, display_name, terminal)VALUES  ('draft','Draft',0),  ('submitted','Submitted',0),  ('paid','Paid',1),  ('cancelled','Cancelled',1)ON CONFLICT(status_code) DO UPDATE SET  display_name = excluded.display_name,  terminal = excluded.terminal;

Database changes in Git

text · repository structure
db/  migrations/    20260805_01_add_normalized_email.py    20260807_01_backfill_normalized_email.py    20260812_01_enforce_normalized_email.py  seeds/    reference_order_status.sql  checks/    verify_normalized_email.sql  README.md  OWNERS
Pull-request question Evidence
What is the data and lock impact? Plan, table size, estimated duration, batch design
Can old and new app versions coexist? Compatibility matrix and rollout order
How is failure detected? Metrics, logs, revision status, verification query
How is it recovered? Rollback or forward-fix procedure and backup dependency
Was generated SQL reviewed? Offline SQL artifact or migration logs
Does it work from every supported state? Fresh install, previous release upgrade, representative data tests
Who owns approval? Database/platform owner plus application owner

CI migration matrix

bash · representative pipeline
set -euo pipefail# 1. Build an empty database from the complete revision history.alembic upgrade headalembic current --check-headspytest tests/database# 2. Upgrade a copy representing the previous production release.restore_fixture previous_release.dbalembic upgrade headpython db/checks/verify_schema.pypytest tests/integration# 3. Render SQL for review where supported.alembic upgrade base:head --sql > migration-plan.sqlgit diff --exit-code -- migration-plan.sql || true

Migration review

  1. Why must applied migrations be immutable?
  2. Why is a nullable additive column often safer than an immediate required column?
  3. Why are large data backfills frequently separated from schema revisions?
  4. What does fresh-install testing fail to prove?
Review the answers

Editing history destroys reproducibility and checksum trust.

Additive nullable changes allow old and new versions to coexist while data is populated.

Backfills need batching, restartability, throttling, and separate monitoring.

Fresh installs do not expose upgrade-time data conversion, locking, volume, or compatibility problems.

SQLite migration exercise

python · ordered transactional runner
import hashlibimport sqlite3from pathlib import PathMIGRATIONS = [    ("001", "create product", "CREATE TABLE product(id INTEGER PRIMARY KEY, sku TEXT UNIQUE)"),    ("002", "add active", "ALTER TABLE product ADD COLUMN active INTEGER NOT NULL DEFAULT 1"),]with sqlite3.connect("academy.db") as db:    db.execute("""CREATE TABLE IF NOT EXISTS schema_migration(        revision_id TEXT PRIMARY KEY,        checksum_sha256 TEXT NOT NULL,        description TEXT NOT NULL,        applied_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP    )""")    applied = {row[0]: row[1] for row in db.execute(        "SELECT revision_id, checksum_sha256 FROM schema_migration"    )}    for revision, description, sql in MIGRATIONS:        checksum = hashlib.sha256(sql.encode()).hexdigest()        if revision in applied:            if applied[revision] != checksum:                raise RuntimeError(f"edited applied migration: {revision}")            continue        db.execute(sql)        db.execute(            "INSERT INTO schema_migration(revision_id,checksum_sha256,description) VALUES(?,?,?)",            (revision, checksum, description),        )

Chapter summary

  • Version both the desired schema and the ordered path that changes existing databases.
  • Use expand-and-contract for rolling compatibility and isolate large data movement.
  • Keep reference seeds deterministic and production-safe; keep test/demo data separate.
  • Make migration evidence, ownership, review, and verification part of the Git delivery workflow.
  • Chapter 18 applies the complete course in a design-and-query capstone.

References

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only assets compatible with the Ethereum/ERC-20 network. Do not send TRC-20/TRON assets.