Chapter 23 · Encryption, CMEK, Network Access, Private Connectivity, and Data Governance
PII Classification, Field-Level Design, Logging Redaction, Backups, Exports, and Right-to-Delete Workflows
Classify AtlasMart data, minimize sensitive fields, redact logs, and build a deletion inventory that follows primary data into backups, exports, analytics copies, and retained evidence.
1. AtlasMart problem: deleting the primary record is not the same as deleting the person’s footprint
A customer invokes AtlasMart’s account-deletion workflow. The
application deletes profiles/u123, but the same
email address appears in orders, support notes, debug logs, an
export bucket, a recovery backup, and an analytics table. A
“right to delete” workflow therefore needs an inventory of data
copies, policy-based exceptions, evidence, and asynchronous
follow-up—not a single Firestore delete().
- Classify fields by sensitivity and purpose before choosing storage/logging patterns.
- Minimize PII and separate identifiers from high-cardinality operational data where useful.
- Redact logs/tokens before they leave application memory.
- Track primary, backup, export, analytics, and support copies in deletion workflows.
- Distinguish product deletion, legal hold, retention, and recoverability obligations.
Use the Emulator Suite, a Firebase demo project, or an isolated test project for destructive, security-sensitive, billing-sensitive, migration, backup/restore, or write-heavy exercises unless the lesson explicitly marks managed verification as required. Treat shown output as expected evidence unless it is explicitly identified as captured output, and re-check current Firebase/Google Cloud edition, mode, quota, pricing, and security documentation before production execution.
AtlasMart keeps project ID
demo-atlasmart-firestore, Standard-edition Native
mode, database (default), Node.js 22+, Firebase
CLI 15.30.0, Firebase Admin Node SDK
14.4.0, Firestore emulator
127.0.0.1:8080, Auth emulator
127.0.0.1:9099, and Emulator UI
127.0.0.1:4000. Mandatory work is deterministic
and local. Cloud KMS/CMEK, VPC Service Controls, Private
Google Access, regional endpoints, Cloud Audit Logs, managed
backup/export evidence, and real residency validation require
a real Google Cloud project and are optional controlled
verification steps. The emulator does not emulate KMS key
availability, Google network routing, service perimeters,
regional processing guarantees, or managed audit logs. The
local lab uses synthetic identities only. Never paste real
customer PII, ID tokens, service-account keys, or secrets into
training logs.
| Term | Meaning in this chapter |
|---|---|
| Client SDK | Mobile/web Firestore path normally authorized with Firebase Authentication and Security Rules; App Check can reduce abuse but does not replace authorization. |
| Server/Admin SDK | Trusted workload path authenticated with Google credentials/IAM; it bypasses Firestore Security Rules and therefore requires application-layer authorization. |
| Standard / Enterprise | Firestore editions with different query/index/pricing capabilities. Governance controls must be checked against the actual edition and interface. |
| Native Core / Pipeline | Enterprise Native can expose familiar Core operations and advanced Pipeline operations. Network/encryption controls protect the database resource; query semantics still differ. |
| MongoDB compatibility |
Enterprise mode exposing a MongoDB-compatible protocol
and firestore.goog endpoint. It is not
MongoDB server software and has distinct
private-connectivity instructions.
|
| CMEK | Customer-managed encryption key in Cloud KMS used by Firestore for server-side encryption at rest. It is not client-side encryption. |
| VPC Service Controls | A Google Cloud data-exfiltration control that places supported managed services/resources behind a service perimeter. It complements rather than replaces IAM/Rules. |
2. Classification drives controls
| Class | AtlasMart example | Default handling |
|---|---|---|
| Public | Published product title | Normal application controls |
| Internal | Inventory reconciliation marker | Backend-only where appropriate; limited logs |
| Personal data | Email, shipping recipient, IP-linked support record | Purpose limitation, Rules/IAM, retention/deletion inventory |
| Secret/credential | ID token, private key, API secret | Do not store in Firestore/logs unless explicitly designed; use secret system |
| Regulated/high-risk | Organization-specific sensitive category | Security/privacy review; encryption/key/network/retention controls based on policy |
Classification is organization-specific. The lesson uses labels to make the control mechanism visible; it does not claim that a field has a universal legal classification.
3. Field-level design: collect less, duplicate intentionally
Every duplicated personal field multiplies deletion and breach-response work. If orders need an immutable shipping snapshot for business/legal reasons, document that purpose and retention class explicitly. Do not duplicate current profile email into every telemetry event just because denormalization is convenient.
{ "profiles/{uid}": { "email": {"class":"personal", "purpose":"account-contact", "retention":"account-life"}, "displayName": {"class":"personal", "purpose":"display", "retention":"account-life"} }, "orders/{id}": { "customerId": {"class":"pseudonymous-id", "purpose":"ownership", "retention":"order-policy"}, "shippingSnapshot": {"class":"personal", "purpose":"fulfillment", "retention":"order-policy"} }, "events/{id}": { "customerId": {"class":"pseudonymous-id", "purpose":"product-analytics", "retention":"30d"}, "email": {"forbidden":true} }}
4. Log redaction is an application control, not a Firestore feature
Cloud Logging can provide powerful audit/operational evidence, but logs become another data store. Never log raw tokens, Authorization headers, service-account key material, or whole customer documents for convenience. Redact before emission so accidental sinks/exports inherit the safer payload.
const SENSITIVE_KEYS = new Set([ "authorization", "token", "idToken", "refreshToken", "email", "phone", "shippingAddress", "serviceAccountKey"]);function redact(value) { if (Array.isArray(value)) return value.map(redact); if (!value || typeof value !== "object") return value; return Object.fromEntries(Object.entries(value).map(([k,v]) => [k, SENSITIVE_KEYS.has(k) ? "[REDACTED]" : redact(v)] ));}const event = {action:"order.view", email:"synthetic@example.invalid", token:"secret", orderId:"o-1001"};console.log(JSON.stringify(redact(event)));
Add tests that assert known secret/PII patterns are absent from emitted log objects. Redaction that relies only on operators remembering not to log data is not a control.
5. Deletion is an inventory workflow with states
| System/copy | Deletion mechanism | Possible exception | Evidence |
|---|---|---|---|
| Primary profile | Application delete/anonymize | Legal hold | Document absent/anonymized + audit event |
| Orders | Field redaction/anonymization according to policy | Required retention | Sampled verification + policy ID |
| TTL events | Product-visible expiry + async TTL cleanup | Investigation hold | Eligibility + eventual deletion metrics |
| Scheduled backups | Retention expiry / backup deletion policy | DR retention obligation | Backup inventory and expiry timestamps |
| Managed exports | Cloud Storage object/lifecycle deletion | Approved archive | Object inventory/lifecycle evidence |
| Analytics/logs | Downstream deletion/redaction + retention expiry | Security evidence retention | Job results / sink retention policy |
A backup is intentionally immutable historical evidence. If policy allows retention until scheduled expiry rather than immediate mutation, the deletion workflow should record that deferred scope and the time when the copy becomes inaccessible. Do not promise immediate erasure from systems whose mechanism does not provide it.
6. Legal hold must defeat automated deletion by design
Chapter 21 established that TTL is asynchronous and unsuitable
as a legal-retention engine. Model a hold explicitly. A deletion
request first resolves policy: if a valid hold exists, the
system records the request and blocks destructive actions for
the held scope; if no hold exists, the workflow proceeds. Avoid
sprinkling one legalHold boolean across unrelated
collections without an authoritative owner and release process.
{ "requestId": "del-u123-20260917", "subjectId": "u123", "state": "IN_PROGRESS", "policyVersion": "privacy-retention-v4", "scopes": { "profile": "DONE", "orders": "ANONYMIZED", "events": "TTL_PENDING", "exports": "OBJECT_DELETE_QUEUED", "backups": "RETAIN_UNTIL_2026-11-01", "analytics": "JOB_PENDING" }, "hold": null}
7. Failure injection: primary-only delete falsely reports success
Seed a synthetic subject into six local fixture stores: Firestore-like primary JSON, order snapshot, event file, log file, export directory, and analytics CSV. Run an intentionally incomplete deleter that touches only the primary. The test must fail because five scopes remain. Repair it with a scope registry and per-scope state. This makes “forgotten copy” observable.
const expectedScopes = ["primary","orders","events","logs","exports","analytics"];const evidence = JSON.parse(await fs.promises.readFile("governance/evidence/deletion-u123.json","utf8"));const incomplete = expectedScopes.filter(s => !["DONE","ANONYMIZED","DEFERRED_BY_POLICY"].includes(evidence.scopes[s]));if (incomplete.length) { console.error("DELETION_INCOMPLETE", incomplete); process.exit(1);}console.log("DELETION_SCOPE_COMPLETE");
8. Backups and exports are governed copies
Chapter 22 separated scheduled backups from managed export/import. Chapter 23 adds governance: who may create/restore/export, which encryption applies, where Cloud Storage objects reside, retention duration, and what a deletion request means for historical copies. CMEK on the primary does not prevent authorized restore/export into a different encryption posture unless IAM and organization policy constrain those actions.
Production judgment
Minimize collection, minimize duplication, isolate secrets, redact before logging, and automate a deletion inventory with resumable/idempotent states. Privacy/legal teams define policy; engineering must make the policy executable and observable. A “delete endpoint” is only complete when it accounts for every controlled copy and documents unavoidable/deferred retention.
Knowledge check
-
Why is deleting
profiles/u123insufficient for subject deletion? - Should raw ID tokens appear in debug logs?
- What should happen when a legal hold exists?
- Why must exports/backups appear in deletion evidence?
- Does CMEK automatically enforce right-to-delete?
Review the answers
1. Copies can exist in orders, logs, exports, analytics, and backups.
2. No; redact/avoid them before log emission.
3. Apply policy-defined hold behavior and record the blocked/deferred scope.
4. They are independent retained copies with their own deletion/expiry mechanisms.
5. No; it controls server-side encryption keys, not data-lifecycle policy.
Summary and next step
This lesson established the working contract for PII Classification, Field-Level Design, Logging Redaction, Backups, Exports, and Right-to-Delete Workflows. Keep its edition/mode assumptions, trust boundary, verification evidence, and operational constraints explicit when reusing the pattern.
Next, continue to Build a Security/Governance Control Matrix Mapping Threats and Compliance Requirements to Firestore/Google Cloud Controls.
Authoritative references
- Firebase: Server-side encryption — Google-managed encryption, CMEK, client-side encryption distinction, and TLS in transit.
- Google Cloud: Firestore Native CMEK — availability behavior, rotation, backup/restore interaction, and limitations.
- Google Cloud: Firestore Native VPC Service Controls — Firestore service perimeter behavior.
- Google Cloud: Firestore locations — regional/multi-region topology and immutable database location.
- Google Cloud: Firestore regional endpoints — server-library data-locality endpoint behavior.
-
Google Cloud: Private Google Access for Firestore with
MongoDB compatibility
—
firestore.googandrestricted.firestore.googprivate routing.