Chapter 16 · Aggregation Queries, Server-Side Count/Sum/Avg, Materialized Aggregates, and Analytics Boundaries

When to Maintain Write-Time Aggregates for Realtime Display vs Run Read-Time Aggregations

Compare read-time aggregation with transaction-maintained and event-maintained summary documents, including lag, idempotency, contention, repair, and security tradeoffs.

Advanced · 175–210 minutesmaterialized aggregates · realtime · repair · idempotencyFirebase JS 12.19.0 · Admin 14.4.0 · @google-cloud/firestore 9.1.0CLI 15.30.0 lab pin · Standard Native canonical lab · Enterprise differences explicitLast reviewed: September 2026

1. AtlasMart problem: rating average appears on every product card

Execution and safety note

Use the Emulator Suite, a Firebase demo project, or an isolated test project for destructive, security-sensitive, billing-sensitive, migration, backup/restore, or write-heavy exercises unless the lesson explicitly marks managed verification as required. Treat shown output as expected evidence unless it is explicitly identified as captured output, and re-check current Firebase/Google Cloud edition, mode, quota, pricing, and security documentation before production execution.

A query-time average is elegant for occasional detail views. It is less attractive when the same value is rendered on hundreds of cards, refreshed often, expected to update through a realtime listener, or computed across a growing review set. A materialized aggregate stores a derived summary as a normal Firestore document. Reads become cheap and listenable, but every source mutation now has to maintain derived state correctly.

Chapter 16 reproducibility baseline · reviewed 17 September 2026

AtlasMart continues the same mandatory environment used in Chapters 01–15: project ID demo-atlasmart-firestore, Standard edition / Native mode / (default) database, Firestore emulator 127.0.0.1:8080, Authentication emulator 127.0.0.1:9099, Emulator UI 127.0.0.1:4000, Firebase JavaScript SDK 12.19.0, Firebase Admin Node.js SDK 14.4.0 carrying @google-cloud/firestore 9.1.0, @firebase/rules-unit-testing 5.0.2, and Node.js 22+. For continuity with Chapters 01–15 the lab remains pinned to Firebase CLI 15.30.0; CLI 15.30.1 is now available, but its patch notes do not change the aggregation semantics taught here. Mandatory work remains local/no-cost. The emulator is useful for deterministic behavior and rules tests, but it is not billing evidence, production latency evidence, Enterprise scan-cost evidence, or a warehouse.

Current documentation check

Core aggregation queries support count(), sum() and average()/avg() naming depending on SDK. They execute on the backend, skip local cache and pending local writes, do not support realtime listeners or offline queries, and can return DEADLINE_EXCEEDED when an aggregation cannot complete within 60 seconds. In Standard Native, aggregation pricing is based on index entries read, billed as one read for each batch of up to 1,000 index entries with a minimum of one document read. Enterprise billing uses read units based on data processed rather than this Standard formula. Enterprise Pipeline operations have a distinct aggregate(...) stage with grouping and accumulator capabilities. Firestore with MongoDB compatibility has its own MongoDB aggregation surface and must not be treated as the Native SDK API.

Learning outcomes

01

Choose synchronous transaction materialization when the summary and source write fit one safe atomic boundary.

02

Design asynchronous materialization as an idempotent workflow that tolerates duplicate delivery and measurable lag.

03

Recognize when one summary document becomes a write hotspot and when sharded counters are appropriate.

04

Build reconciliation as a first-class repair path rather than trusting derived counters forever.

05

Compare read-time and write-time cost/freshness without claiming asynchronous materialization is realtime.

2. Same source truth, four read models

Pattern Freshness/UX Write amplification Read work Correctness/repair Best fit
Query-time aggregation Fresh server answer when requested; not realtime/offline None beyond source writes Scales with index/data scanned; Standard bills index entries No derived state to drift; underlying query/rules must be valid Occasional counts/sums/averages on bounded query shapes
Materialized summary document Realtime/listenable if clients listen to summary Extra write(s) per source mutation One/few document reads Can drift if async; needs idempotency/reconciliation Frequently displayed dashboard value with bounded writer contention
Distributed counter/shards Near-realtime after shard writes; exact read requires all shards or roll-up One shard write per event plus optional roll-up Reads grow with shard count unless roll-up used Shard total is deterministic; roll-up may lag and needs repair High-frequency additive counters
External warehouse Usually delayed relative to operational DB Replication/export pipeline Analytical scans/SQL in warehouse Must track lag, duplicates, backfills, reconciliation Historical, multidimensional, large-scale analytics

No row wins universally. The correct choice depends on how often the value changes, how often it is read, freshness requirements, contention, cost model, and operational appetite for repair.

3. Pattern A: synchronous transaction-maintained summary

write-review-and-summary.mjs · synchronous materialization for bounded write rates
import { FieldValue } from "firebase-admin/firestore";export async function createPublishedReviewWithSummary(db, productId, reviewId, review) {  const reviewRef = db.doc(`products/${productId}/reviews/${reviewId}`);  const summaryRef = db.doc(`products/${productId}/metrics/ratings`);  await db.runTransaction(async (tx) => {    const current = await tx.get(summaryRef);    const s = current.exists ? current.data() : { numericCount: 0, ratingSum: 0 };    const isNumeric = Number.isFinite(review.rating);    const numericCount = s.numericCount + (isNumeric ? 1 : 0);    const ratingSum = s.ratingSum + (isNumeric ? review.rating : 0);    tx.create(reviewRef, review);    tx.set(summaryRef, {      public: true,      publishedCount: FieldValue.increment(1),      numericCount,      ratingSum,      ratingAverage: numericCount ? ratingSum / numericCount : null,      updatedAt: FieldValue.serverTimestamp(),      source: "transaction"    }, { merge: true });  });}// The review and summary are atomic here, but the summary document can become a hotspot.// External side effects still do not belong inside the retryable transaction callback.

This pattern makes creation of a published review and update of its rating summary atomic. It is compelling when the source write path is centralized and write rate is comfortably below a hot-document contention threshold. The transaction callback may retry, so external side effects still stay outside it. If many independent writers all update products/p-1001/metrics/ratings, that summary document becomes a shared contention point.

4. Pattern B: asynchronous idempotent materializer

materialize-review-event.mjs · at-least-once-safe asynchronous summary
import { FieldValue } from "firebase-admin/firestore";export async function applyReviewEvent(db, event) {  const eventRef = db.doc(`aggregateEvents/${event.id}`);  const summaryRef = db.doc(`products/${event.productId}/metrics/ratings`);  await db.runTransaction(async (tx) => {    if ((await tx.get(eventRef)).exists) return; // deduplicate replay    const summary = await tx.get(summaryRef);    const s = summary.exists ? summary.data() : { numericCount: 0, ratingSum: 0 };    const deltaCount = Number.isFinite(event.rating) ? event.direction : 0;    const deltaSum = Number.isFinite(event.rating) ? event.direction * event.rating : 0;    const numericCount = s.numericCount + deltaCount;    const ratingSum = s.ratingSum + deltaSum;    tx.set(eventRef, {      appliedAt: FieldValue.serverTimestamp(),      productId: event.productId,      direction: event.direction    });    tx.set(summaryRef, {      public: true,      numericCount,      ratingSum,      ratingAverage: numericCount ? ratingSum / numericCount : null,      source: "event-materializer",      updatedAt: FieldValue.serverTimestamp()    }, { merge: true });  });}

Asynchronous processing breaks the atomic boundary: the review can commit before the summary changes. That creates an explicit consistency window. The durable aggregateEvents/{eventId} record makes duplicate delivery harmless at the database layer. This mirrors the at-least-once event reality established in Chapter 10. The UI may listen to the summary, but the summary itself can still lag the source event.

“Realtime listener” does not make async materialization realtime.

A listener can deliver the summary update quickly after it is written; it cannot make an event handler execute exactly once or instantly. Measure source-event-to-summary lag separately.

5. Pattern C: distributed count/sum shards

rating-shards.mjs · distribute count/sum writes
import { FieldValue } from "firebase-admin/firestore";import { randomInt } from "node:crypto";const SHARDS = 8;export async function addRatingToShard(db, productId, numericRating) {  const shard = String(randomInt(0, SHARDS)).padStart(2, "0");  await db.doc(`products/${productId}/ratingCounterShards/${shard}`).set({    numericCount: FieldValue.increment(1),    ratingSum: FieldValue.increment(numericRating)  }, { merge: true });}export async function readRatingShards(db, productId) {  const snap = await db.collection(`products/${productId}/ratingCounterShards`).get();  let numericCount = 0, ratingSum = 0;  for (const doc of snap.docs) {    numericCount += doc.get("numericCount") ?? 0;    ratingSum += doc.get("ratingSum") ?? 0;  }  return {    numericCount,    ratingSum,    ratingAverage: numericCount ? ratingSum / numericCount : null,    shardReads: snap.size  };}

For additive metrics, shards distribute write pressure. Average is derived as sum / count. The tradeoff moves to reads: exact shard aggregation requires reading each shard, and more shards mean more read work. A slower roll-up document can reduce client reads at the cost of bounded staleness. This is a good example of intentionally exchanging write distribution for read complexity.

6. Repair is part of the design

reconcile-summary.mjs · repair materialized state from source truth
import { AggregateField, FieldValue } from "firebase-admin/firestore";export async function reconcileRatings(db, productId) {  const q = db.collection(`products/${productId}/reviews`).where("published", "==", true);  const truth = (await q.aggregate({    publishedCount: AggregateField.count(),    ratingSum: AggregateField.sum("rating"),    ratingAverage: AggregateField.average("rating")  }).get()).data();  await db.doc(`products/${productId}/metrics/ratings`).set({    public: true,    ...truth,    source: "reconciliation",    reconciledAt: FieldValue.serverTimestamp()  }, { merge: true });  return truth;}

Derived state should be treated as rebuildable. Reconciliation compares the materialized value against source truth and records when the repair happened. In the local fixture, the expected published review truth is count=5, numeric sum=12, and average=4. A repair job that produces a different answer is either exposing data drift or using a different query population—both deserve investigation.

7. Deliberately wrong approach: non-idempotent trigger increments forever

A function receives a review-created event and executes FieldValue.increment(1) without deduplicating the event. If the event is delivered twice, the counter is wrong. Retrying the trigger makes the damage worse, not better. Repair it by giving every logical change a durable event identity, applying it transactionally once, recording application evidence, and retaining a reconciliation path that can rebuild the summary.

8. Cost and correctness surface

Question Read-time aggregate Materialized summary Distributed counter
Source write cost No extra derived write At least one extra summary/event write Shard write; optional roll-up writes
Display read Aggregation scan/index work per refresh One summary document read/listener All shards for exact total, or one roll-up read
Realtime/offline No listener/offline aggregation Normal document can be cached/listened to Shard/roll-up documents can be cached/listened to
Drift risk No persisted derived state Yes if maintenance fails/duplicates Shard truth additive; roll-up can lag/drift
Repair Rerun query Recompute from source and overwrite Sum shards and/or rebuild roll-up

9. Security boundary for derived documents

Clients that may read a product summary do not automatically need permission to write it. A safe pattern is public/readable metrics with writes denied to untrusted clients; a trusted backend or tightly validated transaction updates the summary. If client-side transactions maintain the aggregate, Rules must validate both the source document and aggregate invariants across the write set, which adds complexity and access-call pressure. Prefer the simplest trust path that meets the product requirement.

Production judgment and bridge to Lesson 4

Materialization is an operational feature, not a cache checkbox. It introduces lag targets, deduplication, contention, repair and monitoring. When the question expands from “current rating average” to “monthly revenue by seller, region, category, cohort and campaign over years,” even a well-designed materialized Firestore document is the wrong analytical substrate. Lesson 4 draws that boundary.

Knowledge check

  1. What new failure mode appears when an aggregate is materialized?
  2. Why can a single materialized summary become a hotspot?
  3. What does a distributed counter trade?
  4. Can a listener make an asynchronous summary exactly realtime?
  5. What is the safest mindset for derived aggregate data?
Review the answers

1. Derived state can lag or drift from source truth, so idempotency and reconciliation become required.

2. All writers may contend on the same document even if source documents are distributed.

3. It improves write distribution but increases exact-read work with the number of shards, unless a lagging roll-up is used.

4. No. It can deliver the summary after it changes; upstream event processing can still be delayed or duplicated.

5. Treat it as rebuildable state with a recorded source query/contract and a deterministic reconciliation path.

Summary

AtlasMart can now choose between no-derived-state query-time arithmetic, a low-read materialized summary, or distributed additive shards, with lag, contention and repair made explicit.

Authoritative references

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only Ethereum or ERC-20 compatible assets to this address.