Index Exemptions for TTL Fields, Sequential Timestamp Costs, and Write Throughput Considerations

1. AtlasMart problem: the retention field becomes the write bottleneck

AtlasMart writes large numbers of short-lived telemetry events. Every event gets a timestamp-like expireAt that increases with creation time. In Standard edition, automatic single-field indexing means this monotonic field can concentrate index writes in a narrow range. The documented limit for a collection with an indexed sequential field is 500 writes per second. TTL itself needs the field value; your application usually does not need that field indexed for queries. Keeping the automatic index can therefore impose a throughput constraint that the retention policy did not require.

2. TTL does not need a query index to delete documents

In Standard Native mode, the TTL field may be indexed or unindexed. Firestore's TTL subsystem can use the policy even when you exempt the field from single-field query indexing. That means two independent decisions live on one field: “is this a TTL field?” and “should user queries get automatic indexes on it?” A single field-level configuration can hold both TTL and index-exemption settings.

3. Standard configuration: TTL + no automatic query index

{  "indexes": [],  "fieldOverrides": [    {      "collectionGroup": "events",      "fieldPath": "expireAt",      "ttl": true,      "indexes": []    }  ]}

This expresses two things together: events.expireAt is the TTL policy field, and automatic single-field indexing for that field is disabled. Before deployment, verify that no query contract depends on where("expireAt", ...) or orderBy("expireAt"). If such a query exists, redesign it or retain the needed index deliberately.

4. Why “disable all indexes” is the wrong repair

Chapter 15 already established that index fan-out affects writes. The correct response is not to delete indexes indiscriminately. In Standard edition every supported query must be backed by an index; removing an index that a production access pattern uses converts a write optimization into a read outage. Keep a versioned query inventory and tie every exemption to evidence.

5. Mandatory local lab: compare schema/index intent, not fake throughput

The Firestore emulator is useful for validating writes, queries, and configuration parsing, but it is not evidence for production key-range/index hotspot physics. The lab therefore measures application-level operation counts and checks query contracts; it labels all latency numbers as local-only.

const queryContracts = [  { name:"tenant-feed", fields:["tenantId","createdAt"], required:true },  { name:"retention-sweeper", fields:["expireAt"], required:false, reason:"managed TTL" }];const exemptions = new Set(["events.expireAt","events.payload"]);for (const q of queryContracts) {  if (q.required && q.fields.some(f => exemptions.has(`events.${f}`))) {    throw new Error(`Unsafe exemption breaks ${q.name}`);  }}console.log("index-contract: PASS");

Expected result: the index contract passes because no user query relies on expireAt. Then seed the same number of local events with and without your application-side payload fields and capture operation counts. Do not present emulator p95/p99 as Firestore production throughput.

6. Standard vs Enterprise interpretation

Standard automatically creates single-field indexes and enforces index-backed Core queries, so exemptions are an explicit performance tool. Enterprise Native uses optional indexing; an unindexed query can run as a scan, which changes the failure mode from “missing index” to “possibly large scan/read-unit cost.” MongoDB compatibility TTL indexes also do not double as query-planning indexes. The design question is therefore always: which index exists for retention, which for query execution, and what billing/performance model applies to this edition and operation family?

7. Controlled failure: benchmark only averages

A write path that averages 12 ms can still have unacceptable p95/p99 latency or contention errors during bursts. Production validation must report distribution and error classes, not only mean latency. Ramp safely in an isolated environment, record document IDs/key distribution/index configuration, and stop at budget/operation caps. Chapter 15's 500/50/5 guidance remains a ramp heuristic for new Standard collections—not a universal TTL limit.

Production judgment

Exempt a TTL timestamp when it is sequential, high-write, and not queried. Keep an index if product/query/security requirements genuinely need it, then test the consequences. In Enterprise, do not interpret “query runs without index” as proof that the scan is acceptable; use Query Explain and unit/latency evidence.

Verification checklist and cleanup

• Every index exemption maps to a documented query inventory.
• TTL configuration remains enabled after removing the query index.
• No local latency is mislabeled as production throughput.
• Standard/Enterprise index semantics are explicitly separated.
• Rollback is the exact previous firestore.indexes.json plus redeployment and query verification.

Bridge to Lesson 3

Indexing answers how to store expirable data efficiently. Lesson 3 asks a more fundamental question: should the data be soft-deleted, asynchronously TTL-deleted, batch-cleaned, or archived at all?

Knowledge check

1. Does TTL require the field to remain query-indexed in Standard?
2. Why is an indexed expireAt risky at high write rates?
3. Why not disable all indexes?
4. What changes in Enterprise optional-index mode?
5. Can emulator p99 prove production hotspot behavior?

Authoritative references

• Firebase · Manage data retention with TTL policies
• Firebase · Firestore index overview and index exemptions
• Firebase · Cloud Firestore pricing and TTL billing
• Firebase · Firestore quotas and limits
• Firebase · Cloud Firestore triggers
• Firebase · Enterprise TTL indexes
• Google Cloud · MongoDB compatibility TTL indexes
• Google Cloud · MongoDB compatibility release notes
• Google Cloud · Firestore Monitoring metrics