Chapter 18 · Capstone: Design and Query a Complete Database
Review, Test, Document, and Present the Database
Professional database work is complete only when another engineer can understand, verify, operate, change, and defend it. The final lesson turns the implementation into a reviewable engineering deliverable with tests, documentation, recovery evidence, and a presentation narrative.
Learning outcomes
Learning outcomes
Run an automated acceptance suite that proves schema, data, query, transaction, index, and recovery behavior.
Review the capstone with explicit correctness, security, operability, and maintainability gates.
Produce a data dictionary, architecture decisions, query catalogue, and backup/release runbooks.
Present the database by connecting business requirements to design decisions and measured evidence.
Identify production gaps honestly and define the next evolution path without weakening the completed scope.
Definition of done
Requirements trace
Every major table, rule, query, role, and procedure links to an approved requirement or operational need.
Executable evidence
A clean build, negative constraints, query reconciliation, transaction rollback, plans, and restore checks all pass.
Transferable knowledge
Another engineer can rebuild, query, operate, review, and change the system from version-controlled documentation.
Honest review
Known limitations and deferred capabilities are explicit rather than hidden behind a polished demo.
The final artifact must survive reproduction by someone who did not build it.
Automated acceptance suite
import sqlite3from pathlib import PathROOT = Path(__file__).resolve().parents[1]SCHEMA = (ROOT / 'db/001_schema.sql').read_text()SEED = (ROOT / 'db/002_seed.sql').read_text()def build_memory_database(): db = sqlite3.connect(':memory:') db.execute('PRAGMA foreign_keys = ON') db.executescript(SCHEMA) db.executescript(SEED) return dbdef test_structure_and_integrity(): with build_memory_database() as db: assert db.execute('PRAGMA integrity_check').fetchone()[0] == 'ok' assert db.execute('PRAGMA foreign_key_check').fetchall() == [] tables = {row[0] for row in db.execute( "SELECT name FROM sqlite_schema WHERE type='table'" )} assert {'customer','product','inventory','sales_order','order_item', 'payment','shipment','audit_event'} <= tablesdef test_business_reconciliation(): with build_memory_database() as db: order_1001 = db.execute( 'SELECT SUM(line_total_cents) FROM order_item WHERE order_id=1001' ).fetchone()[0] captured_1001 = db.execute( "SELECT SUM(amount_cents) FROM payment WHERE order_id=1001 AND status='captured'" ).fetchone()[0] assert order_1001 == captured_1001 == 7800 non_cancelled = db.execute( '''SELECT SUM(oi.line_total_cents) FROM order_item oi JOIN sales_order so USING(order_id) WHERE so.status <> 'cancelled' ''' ).fetchone()[0] by_product = db.execute( '''SELECT SUM(gross_cents) FROM ( SELECT p.product_id, SUM(CASE WHEN so.status <> 'cancelled' THEN oi.line_total_cents ELSE 0 END) gross_cents FROM product p LEFT JOIN order_item oi ON oi.product_id=p.product_id LEFT JOIN sales_order so ON so.order_id=oi.order_id GROUP BY p.product_id )''' ).fetchone()[0] assert non_cancelled == by_productdef test_constraints_reject_invalid_state(): with build_memory_database() as db: try: db.execute( 'UPDATE inventory SET reserved=on_hand+1 WHERE warehouse_id=1 AND product_id=10' ) except sqlite3.IntegrityError: pass else: raise AssertionError('inventory invariant was not enforced')def test_expected_plan_after_indexes(): with build_memory_database() as db: db.executescript((ROOT / 'db/003_indexes.sql').read_text()) detail = ' '.join(row[3] for row in db.execute( '''EXPLAIN QUERY PLAN SELECT order_id, order_number, ordered_at FROM sales_order WHERE customer_id=1 ORDER BY ordered_at DESC, order_id DESC LIMIT 20''' )) assert 'idx_order_customer_history' in detailTest matrix and review gates
| Gate | Evidence required | Block release when |
|---|---|---|
| Clean build | Empty database built from versioned schema and seed scripts | Manual prerequisite or undocumented object is required |
| Integrity | integrity_check=ok; foreign_key_check empty | Corruption or orphan references appear |
| Constraints | Negative tests fail with expected integrity errors | Invalid state can be committed |
| Query correctness | Control totals, row grains, and fixtures match | Report totals disagree or fan-out is possible |
| Transactions | Success, retry, insufficient-stock, and exception tests | Partial order/reservation remains or retry duplicates work |
| Performance | Representative plans and timing notes | Important workload scans unexpectedly or regression lacks explanation |
| Security | Role matrix, parameterization tests, secret handling | Runtime has unnecessary privilege or input reaches SQL syntax |
| Recovery | Independent restore, integrity checks, smoke queries | Backup cannot be restored or procedure depends on memory |
| Documentation | Data dictionary, query catalogue, decisions, runbooks | Another engineer cannot reproduce or operate the database |
Data dictionary template
TABLE sales_orderPurpose: Durable header for one customer checkout request.Grain: One row per order.Primary key: order_id INTEGER.Alternate keys: order_number, request_key.Foreign keys: customer_id -> customer; shipping_address_id -> address.Lifecycle: draft | submitted | paid | packed | shipped | cancelled.Retention: Retain for the financial/operational policy; do not hard-delete casually.Sensitive fields: customer_id and shipping_address_id are indirect personal data.Main writers: checkout and order-management services.Main readers: customer service, fulfillment, finance, reporting views.Known invariants: request_key unique; status domain; currency length 3.Related tests: test_idempotent_request, test_address_ownership, test_order_reconciliation.Query catalogue template
| Field | Example |
|---|---|
| Name | Q-OPS-03 Fulfillment queue |
| Purpose | Prioritize order lines that can be allocated from a warehouse |
| Result grain | One candidate warehouse per order line |
| Inputs | Allowed order statuses; optional warehouse |
| Ordering | ordered_at, order_id, line_no, warehouse_id |
| Metric semantics | available = on_hand − reserved at query time |
| Expected cardinality | Small operational queue; alert if unexpectedly large |
| Indexes | idx_order_fulfillment plus inventory primary key |
| Security | Fulfillment role only; no customer email or payment details |
| Tests | ready/short fixture, stable ordering, plan check |
Architecture decision records
# ADR-002: Reserve inventory in the order transactionStatus: AcceptedContextNorthstar must prevent two checkouts from reserving the same available units.The database stores on_hand and reserved per warehouse-product balance.DecisionThe checkout service owns reservation. It begins a write transaction and runs aconditional UPDATE whose predicate requires on_hand - reserved >= requested quantity.Every order line and audit event commits in the same transaction.Consequences+ No partial order can outlive a failed reservation.+ The predicate rechecks availability at write time.+ request_key makes retries idempotent.- SQLite permits one writer at a time; production scale may require PostgreSQL row locks.- Reservation expiry/release needs a future workflow.EvidenceTransaction tests cover success, retry, insufficient stock, and rollback.Useful ADRs explain context, decision, alternatives, consequences, and evidence. They do not repeat obvious syntax.
Operational documentation set
README.md Purpose, prerequisites, quick start, supported engines, current limitations.docs/conceptual-model.md Scope, actors, entity grains, cardinalities, business rules.docs/data-dictionary.md Tables, columns, keys, domains, sensitivity, ownership, retention.db/queries/README.md Query names, grains, inputs, expected outputs, indexes, security.runbooks/backup_restore.md RPO/RTO, commands, storage, encryption, restore, verification, escalation.runbooks/release_checklist.md Migration order, compatibility, backup prerequisite, checks, rollback/forward fix.docs/decisions/*.md Price snapshot, inventory reservation, revenue definition, access model.CHANGELOG.md User-visible and operational database changes by version.Presentation narrative
| Slide/section | What to show | Evidence |
|---|---|---|
| 1. Problem | Actors, highest-value use cases, scope exclusions | Requirements catalogue |
| 2. Conceptual model | Entities, grains, cardinalities, history decisions | Model and rule register |
| 3. Physical design | Keys, constraints, generated total, normalized relationships | DDL excerpts and negative tests |
| 4. Operational workflow | Atomic idempotent order placement | Transaction timeline and tests |
| 5. Queries | One operational and one analytical query | Result grain, output, reconciliation |
| 6. Performance | Index chosen from workload | Before/after plan evidence |
| 7. Security and recovery | Role boundaries and restore procedure | Privilege matrix and restore log |
| 8. Limitations | Deferred tax, returns, split shipment detail, HA | Prioritized next-step roadmap |
| 9. Outcome | Requirements traced to verified evidence | Acceptance checklist |
Final rubric
| Dimension | Excellent evidence | Common failure |
|---|---|---|
| Requirements and model | Explicit grain, cardinality, history, scope, acceptance criteria | Tables appear without traceable business reasoning |
| Relational design | Normalized identities and constraints with justified snapshots | Duplicate facts or unprotected domains |
| SQL correctness | Readable queries, explicit grain, reconciliation, deterministic order | SELECT-star demos and unexplained duplicate totals |
| Transactions | Atomicity, idempotency, concurrency predicate, rollback tests | Multiple independent writes with partial failure |
| Performance | Measured representative plans and justified indexes | Index every column or claim speed without evidence |
| Security | Bound parameters, least privilege, data classification | Shared owner account or string-built SQL |
| Recovery | Restore test and business verification against RPO/RTO | A backup command with no restore evidence |
| Documentation | Another engineer can rebuild and operate it | Knowledge exists only in the author’s explanation |
Course completion checklist
[ ] Fresh clone builds northstar.db without manual edits.[ ] All 90 SQL and Database Fundamentals lessons are linked and published.[ ] Schema, seed, indexes, queries, and tests are version controlled.[ ] PRAGMA integrity_check returns ok.[ ] PRAGMA foreign_key_check returns no rows.[ ] Negative constraint tests pass.[ ] Operational and analytical reconciliation totals agree.[ ] Order placement is atomic and idempotent.[ ] Important query plans use intended access paths.[ ] Roles and parameterization boundaries are documented.[ ] Backup restores into a separate database and passes verification.[ ] Data dictionary, ADRs, runbooks, and limitations are complete.[ ] Presentation connects every major decision to evidence.Final review
- What is the strongest proof that the capstone is reproducible?
- Why should known limitations appear in the final presentation?
- What is the difference between a passing SQL result and a trusted business metric?
- When is the database ready for another engineer to own?
Review the answers
A clean build and automated acceptance suite from version-controlled artifacts are the strongest reproduction evidence. Limitations establish honest scope and guide safe evolution. A trusted metric has defined semantics, grain, exclusions, reconciliations, and ownership. Ownership can transfer when the design, tests, operations, recovery, and decisions are understandable without private knowledge.
Course summary
- You progressed from data and relational foundations through querying, schema changes, normalization, transactions, indexing, reusable database logic, security, tools, application access, and delivery workflows.
- The capstone connected those topics into one traceable engineering system.
- A database is complete when its requirements, constraints, queries, failure behavior, access model, recovery procedure, tests, and documentation agree.
- All 18 chapters and all 90 lessons of SQL and Database Fundamentals are now published.