Choose Between Query-Time Aggregation, Materialized Document, Distributed Counter, and External Warehouse

1. Capstone decision: four mechanisms can produce “a total,” but they solve different systems problems

The final Chapter 16 task is not to memorize APIs. It is to decide which aggregate mechanism should own an AtlasMart requirement. We will classify four examples: product average rating, seller order count, high-frequency view counter and quarterly revenue analytics. The answer depends on freshness, fan-out/contention, read frequency, write amplification, accuracy, repairability, authorization and billing.

Learning outcomes

2. The four-option decision matrix

3. Decision 1: product average rating

If a product detail page is viewed occasionally and the review set is modest, query-time count/sum/average gives authoritative server arithmetic with no derived-state repair burden. If every product card across the catalog shows the rating and changes should appear through a listener, use a materialized summary document—provided write contention is acceptable and reconciliation exists. If rating writes become extremely hot, distribute count/sum shards and expose a roll-up with a declared lag bound.

4. Decision 2: seller order count and revenue

An occasional seller dashboard refresh can use an authorized collection-group aggregate with seller/owner constraints. A constantly visible operational dashboard may justify materialized daily/seller summaries. A finance report that groups years of orders across sellers, regions and campaigns belongs in an analytical system. The fact that both return a SUM(totalCents) does not make them the same workload.

5. Decision 3: high-frequency view/favorite counter

A single document increment can become a hotspot. Distributed counters deliberately spread write pressure across shards. Reading every shard on every page view can then become expensive, so a slower roll-up may be appropriate. The summary becomes an approximation within a declared freshness window; never label it exact-at-this-millisecond unless the read path actually computes exact shard truth.

6. Decision 4: quarterly revenue and cohort analytics

This is a warehouse problem: wide historical scans, many dimensions, repeated groupings and integration with non-Firestore data. Replicate/export source facts, model warehouse tables, and reconcile against bounded Firestore partitions. Keep Firestore operational summaries only where the application itself needs them.

7. Same fixture: prove every implementation against one truth

process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import { initializeApp } from "firebase-admin/app";import { getFirestore, Timestamp } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();const t = (s) => Timestamp.fromDate(new Date(s));await db.doc("products/p-1001").set({  name: "Trail Camera", public: true, sellerId: "seller-a", schemaVersion: 4});const reviews = {  "r-001": { userId: "u-alice", published: true,  rating: 5,   body: "Excellent", createdAt: t("2026-09-01T10:00:00Z") },  "r-002": { userId: "u-bob",   published: true,  rating: 4,   body: "Good",      createdAt: t("2026-09-02T10:00:00Z") },  "r-003": { userId: "u-cara",  published: false, rating: 2,   body: "Draft",     createdAt: t("2026-09-03T10:00:00Z") },  "r-004": { userId: "u-dan",   published: true,  rating: 3,   body: "Okay",      createdAt: t("2026-09-04T10:00:00Z") },  "r-005": { userId: "u-erin",  published: true,  rating: "5", body: "Legacy",    createdAt: t("2026-09-05T10:00:00Z") },  "r-006": { userId: "u-faye",  published: true,               body: "No score",  createdAt: t("2026-09-06T10:00:00Z") }};for (const [id, data] of Object.entries(reviews)) {  await db.doc(`products/p-1001/reviews/${id}`).set(data);}await db.doc("users/u-alice/orders/o-1001").set({  ownerUid: "u-alice", tenantId: "seller-a", status: "paid",  totalCents: 19800, createdAt: t("2026-09-10T10:00:00Z")});await db.doc("users/u-alice/orders/o-1002").set({  ownerUid: "u-alice", tenantId: "seller-b", status: "shipped",  totalCents: 4900, createdAt: t("2026-09-11T10:00:00Z")});await db.doc("users/u-bob/orders/o-1003").set({  ownerUid: "u-bob", tenantId: "seller-a", status: "paid",  totalCents: 9900, createdAt: t("2026-09-12T10:00:00Z")});console.log(JSON.stringify({  expected: {    publishedReviewDocuments: 5,    publishedNumericRatings: 3,    publishedRatingSum: 12,    publishedRatingAverage: 4,    sellerAOrders: 2,    sellerAOrderTotalCents: 29700,    sellerAOrderAverageCents: 14850  }}, null, 2));

process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import assert from "node:assert/strict";import { initializeApp } from "firebase-admin/app";import { getFirestore, AggregateField } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();const reviews = db.collection("products/p-1001/reviews").where("published", "==", true);const result = (await reviews.aggregate({  documents: AggregateField.count(),  ratingSum: AggregateField.sum("rating"),  ratingAverage: AggregateField.average("rating")}).get()).data();assert.equal(result.documents, 5);assert.equal(result.ratingSum, 12);assert.equal(result.ratingAverage, 4);const orders = db.collectionGroup("orders").where("tenantId", "==", "seller-a");const orderAgg = (await orders.aggregate({  orders: AggregateField.count(),  totalCents: AggregateField.sum("totalCents"),  averageCents: AggregateField.average("totalCents")}).get()).data();assert.deepEqual(orderAgg, { orders: 2, totalCents: 29700, averageCents: 14850 });console.log(JSON.stringify({ reviews: result, sellerAOrders: orderAgg }, null, 2));

import assert from "node:assert/strict";// Pretend this is a snapshot delivered to an analytical system.const exportedOrders = [  { id: "o-1001", tenantId: "seller-a", totalCents: 19800 },  { id: "o-1002", tenantId: "seller-b", totalCents: 4900 },  { id: "o-1003", tenantId: "seller-a", totalCents: 9900 }];const sellerA = exportedOrders.filter(x => x.tenantId === "seller-a");const warehouse = {  count: sellerA.length,  sum: sellerA.reduce((a, x) => a + x.totalCents, 0)};assert.deepEqual(warehouse, { count: 2, sum: 29700 });console.log({ warehouse, freshness: "snapshot fixture; not realtime" });

The important engineering habit is to compare mechanisms against the same source fixture. If the query-time aggregate says seller-a revenue is 29,700 cents but a materialized summary or warehouse snapshot says 19,800, the problem is not “eventual consistency” in the abstract; it is a concrete missing/duplicate/unprocessed change that must be traced and repaired.

8. Decision record template

{  "requirement": "AtlasMart product rating summary",  "sourceOfTruth": "products/{productId}/reviews where published == true",  "chosenPattern": "query-time | materialized | distributed-counter | warehouse",  "freshness": {    "target": "state exact user-visible requirement",    "pendingLocalWritesIncluded": false,    "realtimeListenerRequired": false  },  "scale": {    "expectedSourceDocuments": "measure/forecast",    "sourceWritesPerSecond": "measure/forecast",    "displayReadsPerSecond": "measure/forecast"  },  "costModel": {    "edition": "Standard Native",    "queryIndexEntriesRead": "managed evidence when relevant",    "derivedWritesPerSourceWrite": "documented",    "warehouseExportOrStreaming": "documented if used"  },  "correctness": {    "idempotencyKey": "required for async materialization",    "reconciliation": "script/query and cadence",    "authoritativeRepairSource": "source query"  },  "security": {    "clientPath": "Firebase Auth + Rules",    "backendPath": "IAM + application authorization",    "warehousePath": "separate dataset IAM/governance"  },  "rollback": "how to fall back without losing evidence"}

9. Failure-injection matrix

10. Cost reasoning without invented numbers

Do not fabricate p95 latency or a monthly bill. Record workload dimensions: number of matching documents/index entries, aggregate refresh rate, source write rate, derived writes per mutation, shard count, summary listener fan-out, replication volume, warehouse scan bytes and storage. Use current pricing for the actual edition/location and Query Explain/monitoring where available. An emulator timing is useful only as a functional benchmark, not a production cost/capacity number.

11. Security review

• Client aggregation requires the underlying query to be Rules-compatible; Rules are not filters.
• Admin/server aggregate code bypasses Rules and must authorize the user/tenant in application code under least-privilege IAM.
• Materialized summaries should usually be client-readable but backend-writable.
• Counter shard write permissions must prevent arbitrary overwrites/negative manipulation unless explicitly supported.
• Warehouse tables need independent IAM, retention and deletion controls.

12. Verification and cleanup

1. Run the deterministic seed and verify source truth.
2. Run query-time aggregations and capture exact results.
3. Intentionally corrupt a materialized summary, run reconciliation and verify repair.
4. Replay the same materializer event and prove no double application.
5. Read all counter shards and compare with the roll-up if one exists.
6. Run the local warehouse simulation and reconcile seller-a count/sum.
7. Document which evidence is local-only and which requires managed production verification.

process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import { initializeApp } from "firebase-admin/app";import { getFirestore } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();for (const path of [  "products/p-1001/metrics",  "products/p-1001/ratingCounterShards",  "aggregateEvents"]) {  await db.recursiveDelete(db.collection(path));}console.log("Chapter 16 derived aggregate state removed; shared source fixtures retained.");

Production judgment and bridge to Chapter 17

Aggregation is a read-model choice. Query-time arithmetic minimizes write complexity; materialization buys cheap/realtime reads at the price of derived-state operations; distributed counters buy write distribution at the price of read/roll-up complexity; warehouses buy analytical power at the price of replication lag and a second security/cost domain. Chapter 17 moves from scalar summaries to vector retrieval, where index design, model versioning and ranking quality become the dominant concerns.

Knowledge check

1. Which pattern has no persisted derived-state drift?
2. When does a materialized summary become attractive?
3. Why is a distributed counter not automatically cheap to read?
4. What requirement strongly points to a warehouse?
5. What must every async derived aggregate have?

Summary

Chapter 16 ends with a decision discipline rather than a favorite API: classify freshness and workload, choose the smallest mechanism that fits, measure its real cost, and make every derived answer repairable.

Authoritative references

• Summarize data with aggregation queries
• Understand Cloud Firestore billing
• Write-time aggregations
• Distributed counters
• Securely query data
• Connect to the Firestore Emulator and emulator differences
• Firestore Standard Core operations overview
• Firestore Enterprise Native Core/Pipeline overview
• Enterprise Pipeline aggregate stage
• Enterprise pricing examples
• MongoDB compatibility behavior differences
• Managed Firestore export/import
• Firebase Extensions migration guidance, including Stream Firestore to BigQuery
• Firebase JavaScript SDK release notes
• Firebase Admin Node.js SDK release notes