Chapter 23 · Encryption, CMEK, Network Access, Private Connectivity, and Data Governance

PII Classification, Field-Level Design, Logging Redaction, Backups, Exports, and Right-to-Delete Workflows

Classify AtlasMart data, minimize sensitive fields, redact logs, and build a deletion inventory that follows primary data into backups, exports, analytics copies, and retained evidence.

Advanced · 160–220 minutesPII · redaction · deletion · backup · exportNode 22+ · Firebase CLI 15.30.0 · Firestore emulator 127.0.0.1:8080Mandatory governance lab local/no-cost · CMEK/VPC/managed evidence optional cloud-onlyLast reviewed: 17 September 2026

1. AtlasMart problem: deleting the primary record is not the same as deleting the person’s footprint

A customer invokes AtlasMart’s account-deletion workflow. The application deletes profiles/u123, but the same email address appears in orders, support notes, debug logs, an export bucket, a recovery backup, and an analytics table. A “right to delete” workflow therefore needs an inventory of data copies, policy-based exceptions, evidence, and asynchronous follow-up—not a single Firestore delete().

Learning outcomes
  • Classify fields by sensitivity and purpose before choosing storage/logging patterns.
  • Minimize PII and separate identifiers from high-cardinality operational data where useful.
  • Redact logs/tokens before they leave application memory.
  • Track primary, backup, export, analytics, and support copies in deletion workflows.
  • Distinguish product deletion, legal hold, retention, and recoverability obligations.
Execution and safety note

Use the Emulator Suite, a Firebase demo project, or an isolated test project for destructive, security-sensitive, billing-sensitive, migration, backup/restore, or write-heavy exercises unless the lesson explicitly marks managed verification as required. Treat shown output as expected evidence unless it is explicitly identified as captured output, and re-check current Firebase/Google Cloud edition, mode, quota, pricing, and security documentation before production execution.

Chapter 23 reproducibility baseline · reviewed 17 September 2026

AtlasMart keeps project ID demo-atlasmart-firestore, Standard-edition Native mode, database (default), Node.js 22+, Firebase CLI 15.30.0, Firebase Admin Node SDK 14.4.0, Firestore emulator 127.0.0.1:8080, Auth emulator 127.0.0.1:9099, and Emulator UI 127.0.0.1:4000. Mandatory work is deterministic and local. Cloud KMS/CMEK, VPC Service Controls, Private Google Access, regional endpoints, Cloud Audit Logs, managed backup/export evidence, and real residency validation require a real Google Cloud project and are optional controlled verification steps. The emulator does not emulate KMS key availability, Google network routing, service perimeters, regional processing guarantees, or managed audit logs. The local lab uses synthetic identities only. Never paste real customer PII, ID tokens, service-account keys, or secrets into training logs.

Term Meaning in this chapter
Client SDK Mobile/web Firestore path normally authorized with Firebase Authentication and Security Rules; App Check can reduce abuse but does not replace authorization.
Server/Admin SDK Trusted workload path authenticated with Google credentials/IAM; it bypasses Firestore Security Rules and therefore requires application-layer authorization.
Standard / Enterprise Firestore editions with different query/index/pricing capabilities. Governance controls must be checked against the actual edition and interface.
Native Core / Pipeline Enterprise Native can expose familiar Core operations and advanced Pipeline operations. Network/encryption controls protect the database resource; query semantics still differ.
MongoDB compatibility Enterprise mode exposing a MongoDB-compatible protocol and firestore.goog endpoint. It is not MongoDB server software and has distinct private-connectivity instructions.
CMEK Customer-managed encryption key in Cloud KMS used by Firestore for server-side encryption at rest. It is not client-side encryption.
VPC Service Controls A Google Cloud data-exfiltration control that places supported managed services/resources behind a service perimeter. It complements rather than replaces IAM/Rules.

2. Classification drives controls

Class AtlasMart example Default handling
Public Published product title Normal application controls
Internal Inventory reconciliation marker Backend-only where appropriate; limited logs
Personal data Email, shipping recipient, IP-linked support record Purpose limitation, Rules/IAM, retention/deletion inventory
Secret/credential ID token, private key, API secret Do not store in Firestore/logs unless explicitly designed; use secret system
Regulated/high-risk Organization-specific sensitive category Security/privacy review; encryption/key/network/retention controls based on policy

Classification is organization-specific. The lesson uses labels to make the control mechanism visible; it does not claim that a field has a universal legal classification.

3. Field-level design: collect less, duplicate intentionally

Every duplicated personal field multiplies deletion and breach-response work. If orders need an immutable shipping snapshot for business/legal reasons, document that purpose and retention class explicitly. Do not duplicate current profile email into every telemetry event just because denormalization is convenient.

atlasmart-governance-schema.json
{  "profiles/{uid}": {    "email": {"class":"personal", "purpose":"account-contact", "retention":"account-life"},    "displayName": {"class":"personal", "purpose":"display", "retention":"account-life"}  },  "orders/{id}": {    "customerId": {"class":"pseudonymous-id", "purpose":"ownership", "retention":"order-policy"},    "shippingSnapshot": {"class":"personal", "purpose":"fulfillment", "retention":"order-policy"}  },  "events/{id}": {    "customerId": {"class":"pseudonymous-id", "purpose":"product-analytics", "retention":"30d"},    "email": {"forbidden":true}  }}

4. Log redaction is an application control, not a Firestore feature

Cloud Logging can provide powerful audit/operational evidence, but logs become another data store. Never log raw tokens, Authorization headers, service-account key material, or whole customer documents for convenience. Redact before emission so accidental sinks/exports inherit the safer payload.

redact-log.mjs
const SENSITIVE_KEYS = new Set([  "authorization", "token", "idToken", "refreshToken",  "email", "phone", "shippingAddress", "serviceAccountKey"]);function redact(value) {  if (Array.isArray(value)) return value.map(redact);  if (!value || typeof value !== "object") return value;  return Object.fromEntries(Object.entries(value).map(([k,v]) =>    [k, SENSITIVE_KEYS.has(k) ? "[REDACTED]" : redact(v)]  ));}const event = {action:"order.view", email:"synthetic@example.invalid", token:"secret", orderId:"o-1001"};console.log(JSON.stringify(redact(event)));

Add tests that assert known secret/PII patterns are absent from emitted log objects. Redaction that relies only on operators remembering not to log data is not a control.

5. Deletion is an inventory workflow with states

System/copy Deletion mechanism Possible exception Evidence
Primary profile Application delete/anonymize Legal hold Document absent/anonymized + audit event
Orders Field redaction/anonymization according to policy Required retention Sampled verification + policy ID
TTL events Product-visible expiry + async TTL cleanup Investigation hold Eligibility + eventual deletion metrics
Scheduled backups Retention expiry / backup deletion policy DR retention obligation Backup inventory and expiry timestamps
Managed exports Cloud Storage object/lifecycle deletion Approved archive Object inventory/lifecycle evidence
Analytics/logs Downstream deletion/redaction + retention expiry Security evidence retention Job results / sink retention policy

A backup is intentionally immutable historical evidence. If policy allows retention until scheduled expiry rather than immediate mutation, the deletion workflow should record that deferred scope and the time when the copy becomes inaccessible. Do not promise immediate erasure from systems whose mechanism does not provide it.

6. Legal hold must defeat automated deletion by design

Chapter 21 established that TTL is asynchronous and unsuitable as a legal-retention engine. Model a hold explicitly. A deletion request first resolves policy: if a valid hold exists, the system records the request and blocks destructive actions for the held scope; if no hold exists, the workflow proceeds. Avoid sprinkling one legalHold boolean across unrelated collections without an authoritative owner and release process.

deletion-state.json
{  "requestId": "del-u123-20260917",  "subjectId": "u123",  "state": "IN_PROGRESS",  "policyVersion": "privacy-retention-v4",  "scopes": {    "profile": "DONE",    "orders": "ANONYMIZED",    "events": "TTL_PENDING",    "exports": "OBJECT_DELETE_QUEUED",    "backups": "RETAIN_UNTIL_2026-11-01",    "analytics": "JOB_PENDING"  },  "hold": null}

7. Failure injection: primary-only delete falsely reports success

Seed a synthetic subject into six local fixture stores: Firestore-like primary JSON, order snapshot, event file, log file, export directory, and analytics CSV. Run an intentionally incomplete deleter that touches only the primary. The test must fail because five scopes remain. Repair it with a scope registry and per-scope state. This makes “forgotten copy” observable.

verify-deletion.mjs
const expectedScopes = ["primary","orders","events","logs","exports","analytics"];const evidence = JSON.parse(await fs.promises.readFile("governance/evidence/deletion-u123.json","utf8"));const incomplete = expectedScopes.filter(s => !["DONE","ANONYMIZED","DEFERRED_BY_POLICY"].includes(evidence.scopes[s]));if (incomplete.length) {  console.error("DELETION_INCOMPLETE", incomplete);  process.exit(1);}console.log("DELETION_SCOPE_COMPLETE");

8. Backups and exports are governed copies

Chapter 22 separated scheduled backups from managed export/import. Chapter 23 adds governance: who may create/restore/export, which encryption applies, where Cloud Storage objects reside, retention duration, and what a deletion request means for historical copies. CMEK on the primary does not prevent authorized restore/export into a different encryption posture unless IAM and organization policy constrain those actions.

Production judgment

Minimize collection, minimize duplication, isolate secrets, redact before logging, and automate a deletion inventory with resumable/idempotent states. Privacy/legal teams define policy; engineering must make the policy executable and observable. A “delete endpoint” is only complete when it accounts for every controlled copy and documents unavoidable/deferred retention.

Knowledge check

  1. Why is deleting profiles/u123 insufficient for subject deletion?
  2. Should raw ID tokens appear in debug logs?
  3. What should happen when a legal hold exists?
  4. Why must exports/backups appear in deletion evidence?
  5. Does CMEK automatically enforce right-to-delete?
Review the answers

1. Copies can exist in orders, logs, exports, analytics, and backups.

2. No; redact/avoid them before log emission.

3. Apply policy-defined hold behavior and record the blocked/deferred scope.

4. They are independent retained copies with their own deletion/expiry mechanisms.

5. No; it controls server-side encryption keys, not data-lifecycle policy.

Summary and next step

This lesson established the working contract for PII Classification, Field-Level Design, Logging Redaction, Backups, Exports, and Right-to-Delete Workflows. Keep its edition/mode assumptions, trust boundary, verification evidence, and operational constraints explicit when reusing the pattern.

Next, continue to Build a Security/Governance Control Matrix Mapping Threats and Compliance Requirements to Firestore/Google Cloud Controls.

Authoritative references

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only Ethereum or ERC-20 compatible assets to this address.