Load-Test and Cost-Test a Feature Before Launch with Explicit Volume, Growth, and Worst-Case Assumptions

1. AtlasMart launch gate: performance test without cost test is incomplete

A load test reports acceptable p99 latency for 15 minutes, but it does not record documents returned, index entries scanned, listener updates, retries, network bytes, storage growth or recovery overhead. The team cannot tell whether the feature is economically safe at 10× traffic. Chapter 25 ends by combining load evidence and billable-work evidence into one launch gate.

2. Two test environments answer different questions

3. Define the test matrix before generating load

Replace these illustrative multipliers with AtlasMart telemetry when available. They are scenario inputs, not universal defaults.

4. Local harness: one trace feeds latency and cost models

import { performance } from "node:perf_hooks";const samples=[];export async function measured(name, accounting, fn) {  const t0=performance.now();  try {    const out=await fn();    samples.push({name, ok:true, ms:performance.now()-t0, ...accounting(out)});    return out;  } catch (e) {    samples.push({name, ok:false, code:e.code ?? "UNKNOWN", ms:performance.now()-t0});    throw e;  }}export function percentile(values,p){  const a=[...values].sort((x,y)=>x-y);  return a[Math.max(0, Math.ceil(a.length*p/100)-1)] ?? 0;}export function report(){  const ok=samples.filter(x=>x.ok);  return {    requests:samples.length,    errors:samples.length-ok.length,    p50_ms:percentile(ok.map(x=>x.ms),50),    p95_ms:percentile(ok.map(x=>x.ms),95),    p99_ms:percentile(ok.map(x=>x.ms),99),    documentReads:ok.reduce((s,x)=>s+(x.documentReads||0),0),    writes:ok.reduce((s,x)=>s+(x.writes||0),0),    vectorEntries:ok.reduce((s,x)=>s+(x.vectorEntries||0),0),    bytesReturned:ok.reduce((s,x)=>s+(x.bytesReturned||0),0)  };}

For emulator runs, the accounting function uses fixture knowledge (documents returned, local candidate count, serialized bytes). In managed cloud, replace approximations with Query Explain/Monitoring/Billing-export evidence where available.

5. Mandatory no-cost rehearsal

1. Start Firestore/Auth emulators and seed deterministic AtlasMart catalog, carts, orders and listener fixtures.
2. Run 500 browse requests, 200 cart mutations, 50 order workflows, 100 listener lifecycles and 50 vector-simulation queries.
3. Inject bounded failures: 2% retriable backend simulation, 10 reconnect cycles, one listener leak that the test must catch, and one broad-query variant.
4. Write trace.jsonl plus summary.json containing p50/p95/p99, errors/retries, reads/writes/vector entries/bytes and cleanup state.
5. Feed the summary into the Standard/Enterprise cost models for best/base/growth/worst cases.
6. Assert emulator listeners are cleaned up and all test data is reset.

6. Optional billed managed test: bounded by design

If launch requires production evidence, create a disposable project/database in the intended edition/location, define a maximum request count and test duration, enable required observability, seed synthetic non-sensitive data, and run only the agreed matrix. Capture Query Explain for representative queries, Monitoring latency/errors, billing export/usage evidence and network path. Tear down test data/resources afterward.

7. Launch gate: combine SLO and cost envelopes

8. Controlled failure: benchmark average latency and average user only

Run two forecasts with identical mean requests: one evenly distributed, one with a 2% heavy cohort and synchronized listener reconnect storm. The second can have much worse p99 and cost amplification despite the same average. Repair by keeping percentile/cohort and worst-case scenarios in the launch gate.

Production judgment: the forecast becomes a living operational contract

After launch, replace assumptions with observed distributions: per-feature DAU, documents/query, index entries scanned, bytes, listener churn, retry rate, storage/index growth, egress and recovery footprint. Compare actuals to the forecast monthly and after every data-model/query/index/retention change. A cost model that is never reconciled is documentation, not control.

Chapter 26 will take this same discipline into CI/CD: Rules/index deployment, emulator tests, schema migrations and release gates.

Knowledge check

1. Why can’t emulator throughput certify production capacity?
2. Why should latency and accounting use the same request trace?
3. What makes a managed load test safe?
4. Why does the worst-case scenario include abuse/reconnect tails?
5. What should happen to the forecast after launch?

Authoritative references

• Google Cloud: Firestore Standard edition pricing — location-specific document operations, storage, PITR, backups/restores and network pricing.
• Google Cloud: Firestore Enterprise edition pricing — Read Units, Write Units, realtime updates, storage, networking, Query Explain and recovery pricing.
• Firebase: Understand Cloud Firestore billing — index-entry reads, aggregations, listeners, offsets, Rules-dependent reads and free quota.
• Firebase: Firestore usage and limits — Standard free quota and current hard/configuration limits.
• Firebase: Enterprise Native mode quotas and limits — Enterprise free-tier units and limits.
• Firebase: Compare Standard and Enterprise Native mode — billing/index/realtime differences.
• Google Cloud Billing budgets — alert budgets notify; ordinary alert budgets do not automatically cap spend.