Chapter 26 · Testing, Emulator Suite, CI/CD, Index/Rules Deployment, and Schema Migration

Create a Production Deployment Checklist Covering Rules, Indexes, Billing, Backups, Observability, and Rollback

Assemble an executable production release gate covering Rules, indexes, schema migration, billing, recovery, observability, canaries, and rollback.

Advanced · 180–240 minutesproduction checklist · canary · billing · backup/PITR · observability · rollbackNode 22+ · Firebase CLI course baseline 15.30.0 · JS SDK 12.19.0 · Admin SDK 14.4.0 · rules-unit-testing 5.0.2Mandatory lab demo project + Emulator Suite/no-cost · managed staging/production canaries explicitly optionalLast reviewed: 17 September 2026

1. AtlasMart go-live: a checklist must be executable evidence

A release checklist that says “security tested,” “indexes okay,” and “backups enabled” is not operational control. Each item needs an owner, command/evidence, environment, pass condition, rollback action, and timestamp. Chapter 26 ends by converting the previous 25 chapters into a release gate that distinguishes local proof from managed production proof.

Learning outcomes
  • Build a production checklist spanning Rules, indexes, schemas, billing, recovery, observability and rollback.
  • Attach every gate to evidence rather than subjective status.
  • Order deployment to preserve backward compatibility and minimize blast radius.
  • Define canary/rollback thresholds before release.
  • Keep Standard/Enterprise, client/server and emulator/production assumptions explicit.
Execution and safety note

Use the Emulator Suite, a Firebase demo project, or an isolated test project for destructive, security-sensitive, billing-sensitive, migration, backup/restore, or write-heavy exercises unless the lesson explicitly marks managed verification as required. Treat shown output as expected evidence unless it is explicitly identified as captured output, and re-check current Firebase/Google Cloud edition, mode, quota, pricing, and security documentation before production execution.

Chapter 26 reproducibility baseline · reviewed 17 September 2026

AtlasMart keeps project ID demo-atlasmart-firestore, Standard-edition Native mode, database (default), Node.js 22+, Firebase CLI course baseline 15.30.0, Firebase JavaScript SDK 12.19.0, Firebase Admin Node SDK 14.4.0, @firebase/rules-unit-testing 5.0.2, Firestore emulator 127.0.0.1:8080, Auth emulator 127.0.0.1:9099, and Emulator UI 127.0.0.1:4000. Mandatory work uses a demo- project and local emulators only. Cloud deployment, IAM, App Check enforcement, billing, production indexes, quotas, contention, Query Insights/Key Visualizer, backup/PITR, and Enterprise/MongoDB-compatibility canaries are explicitly optional managed checks, never silently inferred from emulator success.

2. Release evidence record

release-evidence.json
{  "release": "atlasmart-2026-09-17.1",  "commit": "<git-sha>",  "firebaseCli": "15.30.0",  "runtime": "node-22",  "database": { "edition":"standard", "mode":"native", "id":"(default)", "location":"<verify-managed>" },  "local": {    "project":"demo-atlasmart-firestore",    "rulesTests":"PASS",    "integrationTests":"PASS",    "migrationAudit":{"mismatches":0,"unreadable":0}  },  "managed": {    "stagingIndexCanary":"PENDING",    "iamCanary":"PENDING",    "billingBudgetAlerts":"VERIFY",    "backupPitrDrill":"VERIFY",    "observabilityDashboards":"VERIFY"  },  "rollback": { "featureFlag":"documented", "previousRulesSha":"<sha>", "owner":"release-oncall" }}

3. Gate 1 — repository and environment identity

  • Commit SHA/tag is immutable and matches deployed artifact.
  • Firebase CLI/SDK/runtime versions are recorded.
  • Target project/database ID/edition/mode/location are explicit; no reliance on implicit active CLI project.
  • Production does not contain emulator environment variables.
  • Secrets/service credentials are not stored in repository or logs.

4. Gate 2 — Rules and trust boundaries

  • Mobile/web paths have allow + deny regression tests, including tenant isolation, forged fields and query broadening.
  • Rules file hash is recorded and repository copy matches intended deployment.
  • Admin/server paths have application authorization tests because they bypass Rules.
  • IAM role set is least-privilege and tested with the actual runtime identity.
  • App Check enforcement is verified separately where used; it is not treated as user authorization.

5. Gate 3 — indexes and query contracts

  • Every new Standard composite/vector query has source-controlled index definition.
  • Staging/production-only canary proves required indexes exist and are ready; emulator success is not accepted as evidence.
  • Enterprise optional-index query plans are measured when Enterprise is the target.
  • Query Explain/Insights evidence exists for new high-frequency/high-cost paths where available.
  • Index removal has no active caller and has an application-first rollback path.

6. Gate 4 — schema/migration/backfill

  • Old and new readers/writers coexist for the planned compatibility window.
  • Backfill is idempotent, resumable, throttled and emits progress/error evidence.
  • Failure injection and resume have passed locally.
  • Dual-read mismatch and unreadable-document counts are within explicit threshold (normally zero for money-like fields).
  • Contract/destructive cleanup is a later release, not bundled with initial expansion.

7. Gate 5 — billing, quota, abuse and capacity

  • Chapter 25 workload model is refreshed with current traffic, document/index sizes, retry/listener rates and current dated prices.
  • Billing budgets/alerts are configured as notifications; they are not mistaken for hard spend caps.
  • Security/backup controls were not weakened to hit a cost target.
  • Abuse controls, Rules and App Check strategy are reviewed for worst-case listener/query amplification.
  • Bounded load/cost canary has stop conditions; emulator throughput is not used as capacity proof.

8. Gate 6 — backup, PITR, retention and recovery

  • Required backup schedule and PITR state are verified in managed configuration.
  • Last recovery drill measured RPO/RTO and validated data + index/rule/config dependencies.
  • TTL/retention policy excludes legal-hold/audit records as designed.
  • Rollback/runbook knows whether recent writes after cutover need reconciliation.
  • Operator IAM for backup/restore is least-privilege and auditable.

9. Gate 7 — observability and incident response

  • Dashboard/alerts cover request counts, p95/p99 latency, error codes, quota signals and cost/usage.
  • Correlation IDs are logged without PII/tokens.
  • New query/model has baseline before/after evidence.
  • Hotspot diagnostics / Query Insights / Key Visualizer are available where target supports them and thresholds justify use.
  • Runbook distinguishes permission, missing-index, hotspot, quota, latency and cost incidents.

10. Deployment order: expand before contract

Phase Action Rollback posture
0. Local gate Rules/integration/migration failure-injection tests. No production effect.
1. Managed prerequisites Create/build indexes, verify IAM/backup/observability. Do not release callers until ready.
2. Compatible config Deploy Rules that allow old + new schema. Restore previous reviewed Rules if deny/error threshold trips.
3. App canary Release dual reader/writer behind feature flag. Disable flag immediately.
4. Backfill Run bounded resumable job; monitor progress/mismatch/errors. Pause job; application still reads old/new.
5. Broaden rollout Increase cohort after SLO/cost/security evidence passes. Reduce cohort/flag off.
6. Contract later Remove legacy fields/rules/indexes only after rollback window closes. Separate approval; destructive work is not emergency rollback.

11. Canary thresholds must exist before traffic

canary-policy.json
{  "windowMinutes": 30,  "rollbackIf": {    "permissionDeniedRate": "> baseline + agreed delta",    "dualReadMismatchRate": "> 0 for price fields",    "unreadableDocumentCount": "> 0",    "p95Latency": "> release SLO budget",    "errorRate": "> release SLO budget",    "unexpectedCostMultiplier": "> 1.25x modeled canary envelope"  },  "note": "Thresholds are product/SLO decisions; these labels are not universal Firestore values."}

12. Controlled failure drill: rollback under a mixed-version dataset

Inject a canary failure after 40% of documents are migrated to v2. The correct rollback is:

  1. Stop rollout / disable v2-preferred feature flag.
  2. Pause backfill.
  3. Confirm v1/dual reader serves both migrated and unmigrated documents.
  4. Restore prior Rules only if compatible with the mixed dataset; otherwise keep the expanded Rules while application is rolled back.
  5. Preserve v2 fields; do not run destructive reverse migration during the incident.
  6. Record decision, timestamps, mismatch/error counts and recovery of SLO.
Important

“Rollback” is not always “restore every file to yesterday.” Database releases create persistent data/config state. The rollback unit is the business behavior, with reconciliation where needed.

13. Edition/mode matrix for the final gate

Target Local coverage Mandatory managed verification
Standard Native Core Firestore/Auth emulator, Rules, client/server behavior, migration logic. Composite/vector indexes, limits/contention, IAM, billing, backups/PITR, regional latency/observability.
Enterprise Native Core Enterprise edition emulator where supported for compatible API behavior. RU/WU/realtime billing, optional-index plans, Enterprise limits, IAM/network/recovery.
Enterprise Pipeline Deterministic pipeline/query-plan simulation for features the local stack cannot faithfully execute. Server SDK support, launch-stage feature behavior, Pipeline query plans/cost/security.
MongoDB compatibility Compatibility fixtures/local MongoDB source tests where useful. Real compatible endpoint/auth/driver/tool matrix, index/transaction/change semantics, Enterprise billing.

14. Mandatory local deployment rehearsal

  1. Run the full demo-project CI from clean seed.
  2. Hash Rules/index files and write release evidence JSON.
  3. Run schema-v2 failure injection, resume and audit.
  4. Simulate index readiness state and refuse to enable the feature until it reads READY.
  5. Enable a deterministic 25% canary cohort; inject a mismatch and assert rollback policy disables it.
  6. Produce final checklist with every cloud-only item marked VERIFY_MANAGED—never fake PASS.
Expected state

The release artifact is reproducible, rollback is executable, local evidence is complete, and every emulator gap has a named managed verification owner before production approval.

Production judgment and bridge to Chapter 27

The checklist is not bureaucracy; it is the boundary between a collection of Firestore features and an operable system. Chapter 27 uses this release discipline in the capstone: design the model, security, scaling, retrieval, recovery and operations together, then defend the decisions with measured evidence.

Knowledge check

  1. Why must a checklist item contain evidence and a rollback action?
  2. What should happen before releasing code that needs a new composite index?
  3. Why can Rules rollback be unsafe after a schema migration begins?
  4. What is the correct response to a canary failure in a mixed-version dataset?
  5. Why must cloud-only checklist items remain VERIFY_MANAGED locally?
Review the answers

1. A subjective “done” cannot be audited or executed during an incident; evidence proves the state and rollback bounds the blast radius.

2. Deploy/build the index and prove readiness in a managed staging/target environment.

3. The previous Rules may reject documents/clients in the new mixed schema; behavior rollback and data/config rollback are separate.

4. Disable the feature/canary, pause backfill, serve through compatible readers, then investigate/reconcile without destructive reverse migration.

5. The emulator cannot prove IAM, production indexes/limits, billing, recovery, real latency and other managed-service properties.

Summary and next step

This lesson established the working contract for Create a Production Deployment Checklist Covering Rules, Indexes, Billing, Backups, Observability, and Rollback. Keep its edition/mode assumptions, trust boundary, verification evidence, and operational constraints explicit when reusing the pattern.

Next, continue to Define User Journeys, Data Model, Queries, Realtime/Offline Needs, Atomicity, Retention, Security, SLOs, and Cost Budget.

Authoritative references

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only Ethereum or ERC-20 compatible assets to this address.