Chapter 16 · Aggregation Queries, Server-Side Count/Sum/Avg, Materialized Aggregates, and Analytics Boundaries
count(), sum(), avg() Semantics, Supported SDKs / Queries, and Index-Entry-Based Cost
Use AtlasMart fixtures to prove Firestore count/sum/average semantics, backend-only execution, numeric-field edge cases, Standard index-entry billing, and edition/API boundaries.
1. AtlasMart problem: “just show the average” can hide three different data contracts
Use the Emulator Suite, a Firebase demo project, or an isolated test project for destructive, security-sensitive, billing-sensitive, migration, backup/restore, or write-heavy exercises unless the lesson explicitly marks managed verification as required. Treat shown output as expected evidence unless it is explicitly identified as captured output, and re-check current Firebase/Google Cloud edition, mode, quota, pricing, and security documentation before production execution.
The product page for products/p-1001 needs a
published review count and average rating. A naive
implementation downloads every review, filters in application
code and calculates the result. That wastes bandwidth, creates
more document reads than necessary, and makes the answer depend
on client-side parsing. Firestore Core aggregation queries
instead execute the aggregation on the backend and return a
compact summary result. The mechanism still has a cost: the
database must examine the query/index surface, and that work
grows with the scanned index entries or data processed.
AtlasMart continues the same mandatory environment used in
Chapters 01–15: project ID
demo-atlasmart-firestore, Standard edition /
Native mode / (default) database, Firestore
emulator 127.0.0.1:8080, Authentication emulator
127.0.0.1:9099, Emulator UI
127.0.0.1:4000, Firebase JavaScript SDK
12.19.0, Firebase Admin Node.js SDK
14.4.0 carrying
@google-cloud/firestore 9.1.0,
@firebase/rules-unit-testing 5.0.2,
and Node.js 22+. For continuity with Chapters 01–15 the lab
remains pinned to Firebase CLI 15.30.0; CLI
15.30.1 is now available, but its patch notes do
not change the aggregation semantics taught here. Mandatory
work remains local/no-cost. The emulator is useful for
deterministic behavior and rules tests, but it is not billing
evidence, production latency evidence, Enterprise scan-cost
evidence, or a warehouse.
Core aggregation queries support count(),
sum() and average()/avg()
naming depending on SDK. They execute on the backend, skip
local cache and pending local writes, do not support realtime
listeners or offline queries, and can return
DEADLINE_EXCEEDED when an aggregation cannot
complete within 60 seconds. In Standard Native, aggregation
pricing is based on index entries read, billed as one read for
each batch of up to 1,000 index entries with a minimum of one
document read. Enterprise billing uses read units based on
data processed rather than this Standard formula. Enterprise
Pipeline operations have a distinct
aggregate(...) stage with grouping and
accumulator capabilities. Firestore with MongoDB compatibility
has its own MongoDB aggregation surface and must not be
treated as the Native SDK API.
Learning outcomes
Predict count, sum and average results for missing, numeric and non-numeric fields before running the query.
Use the modular Web SDK and server SDK aggregation APIs against the same AtlasMart query contract.
Explain why aggregation queries are backend-only and do not include local cache or pending writes.
Model Standard aggregation billing from index entries read without applying that formula to Enterprise.
Distinguish Native Core aggregation from Enterprise Pipeline aggregation and MongoDB-compatibility aggregation.
2. Reproducible fixture and expected truth
{ "name": "atlasmart-firestore-ch16", "private": true, "type": "module", "engines": { "node": ">=22" }, "dependencies": { "firebase": "12.19.0", "firebase-admin": "14.4.0" }, "devDependencies": { "firebase-tools": "15.30.0" }, "scripts": { "emulators": "firebase emulators:start --only firestore,auth --project demo-atlasmart-firestore", "seed": "node seed-ch16.mjs", "verify": "node verify-ch16.mjs", "reset": "node reset-ch16.mjs" }}
{ "firestore": { "rules": "firestore.rules", "indexes": "firestore.indexes.json", "edition": "standard" }, "emulators": { "firestore": { "host": "127.0.0.1", "port": 8080, "edition": "standard" }, "auth": { "host": "127.0.0.1", "port": 9099 }, "ui": { "enabled": true, "port": 4000 } }}
rules_version = '2';service cloud.firestore { match /databases/{database}/documents { function signedIn() { return request.auth != null; } match /products/{productId} { allow read: if resource.data.public == true; allow write: if false; match /reviews/{reviewId} { allow get: if resource.data.published == true || (signedIn() && resource.data.userId == request.auth.uid); allow list: if resource.data.published == true; allow write: if signedIn() && request.resource.data.userId == request.auth.uid; } match /metrics/{metricId} { allow read: if resource.data.public == true; allow write: if false; // trusted backend/materializer only } } match /{path=**}/orders/{orderId} { allow read: if signedIn() && resource.data.ownerUid == request.auth.uid; allow write: if false; } }}
{ "indexes": [ { "collectionGroup": "orders", "queryScope": "COLLECTION_GROUP", "fields": [ { "fieldPath": "tenantId", "order": "ASCENDING" }, { "fieldPath": "createdAt", "order": "DESCENDING" } ] } ], "fieldOverrides": [ { "collectionGroup": "reviews", "fieldPath": "body", "indexes": [] } ]}
process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import { initializeApp } from "firebase-admin/app";import { getFirestore, Timestamp } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();const t = (s) => Timestamp.fromDate(new Date(s));await db.doc("products/p-1001").set({ name: "Trail Camera", public: true, sellerId: "seller-a", schemaVersion: 4});const reviews = { "r-001": { userId: "u-alice", published: true, rating: 5, body: "Excellent", createdAt: t("2026-09-01T10:00:00Z") }, "r-002": { userId: "u-bob", published: true, rating: 4, body: "Good", createdAt: t("2026-09-02T10:00:00Z") }, "r-003": { userId: "u-cara", published: false, rating: 2, body: "Draft", createdAt: t("2026-09-03T10:00:00Z") }, "r-004": { userId: "u-dan", published: true, rating: 3, body: "Okay", createdAt: t("2026-09-04T10:00:00Z") }, "r-005": { userId: "u-erin", published: true, rating: "5", body: "Legacy", createdAt: t("2026-09-05T10:00:00Z") }, "r-006": { userId: "u-faye", published: true, body: "No score", createdAt: t("2026-09-06T10:00:00Z") }};for (const [id, data] of Object.entries(reviews)) { await db.doc(`products/p-1001/reviews/${id}`).set(data);}await db.doc("users/u-alice/orders/o-1001").set({ ownerUid: "u-alice", tenantId: "seller-a", status: "paid", totalCents: 19800, createdAt: t("2026-09-10T10:00:00Z")});await db.doc("users/u-alice/orders/o-1002").set({ ownerUid: "u-alice", tenantId: "seller-b", status: "shipped", totalCents: 4900, createdAt: t("2026-09-11T10:00:00Z")});await db.doc("users/u-bob/orders/o-1003").set({ ownerUid: "u-bob", tenantId: "seller-a", status: "paid", totalCents: 9900, createdAt: t("2026-09-12T10:00:00Z")});console.log(JSON.stringify({ expected: { publishedReviewDocuments: 5, publishedNumericRatings: 3, publishedRatingSum: 12, publishedRatingAverage: 4, sellerAOrders: 2, sellerAOrderTotalCents: 29700, sellerAOrderAverageCents: 14850 }}, null, 2));
The fixture deliberately contains five published review
documents but only three published numeric ratings.
Therefore count() over
published == true returns 5;
sum("rating") returns 12; and
average("rating") returns 4. The
string value "5" is not coerced into a number, and
the missing rating is not invented. This is why schema
validation remains important even when an aggregate appears
mathematically valid.
3. Core aggregation mechanism: query first, aggregate second
import { collection, collectionGroup, query, where, getAggregateFromServer, count, sum, average} from "firebase/firestore";export async function publishedRatingTruth(db) { const q = query( collection(db, "products", "p-1001", "reviews"), where("published", "==", true) ); const snap = await getAggregateFromServer(q, { documents: count(), ratingSum: sum("rating"), ratingAverage: average("rating") }); return snap.data();}export async function sellerAOrderTruth(db) { const q = query(collectionGroup(db, "orders"), where("tenantId", "==", "seller-a")); const snap = await getAggregateFromServer(q, { orders: count(), totalCents: sum("totalCents"), averageCents: average("totalCents") }); return snap.data();}
getAggregateFromServer() tells you two essential
facts. First, the aggregation applies to a normal Firestore
query, so filters, collection-group scope, indexes and Security
Rules still matter. Second, the answer comes from the backend
rather than the local cache. Pending offline writes are not
folded into the aggregate. If the UI needs optimistic local
feedback, model that state separately instead of labeling the
server aggregation as local truth.
process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import assert from "node:assert/strict";import { initializeApp } from "firebase-admin/app";import { getFirestore, AggregateField } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();const reviews = db.collection("products/p-1001/reviews").where("published", "==", true);const result = (await reviews.aggregate({ documents: AggregateField.count(), ratingSum: AggregateField.sum("rating"), ratingAverage: AggregateField.average("rating")}).get()).data();assert.equal(result.documents, 5);assert.equal(result.ratingSum, 12);assert.equal(result.ratingAverage, 4);const orders = db.collectionGroup("orders").where("tenantId", "==", "seller-a");const orderAgg = (await orders.aggregate({ orders: AggregateField.count(), totalCents: AggregateField.sum("totalCents"), averageCents: AggregateField.average("totalCents")}).get()).data();assert.deepEqual(orderAgg, { orders: 2, totalCents: 29700, averageCents: 14850 });console.log(JSON.stringify({ reviews: result, sellerAOrders: orderAgg }, null, 2));
Web/mobile aggregation requests are authorized through Firebase Authentication and Security Rules. Admin/server libraries bypass Security Rules and use IAM/ADC, exactly as Chapter 14 established. The arithmetic semantics may be similar, but the authorization path is not.
4. Field behavior that changes the answer
| Case | count() | sum(field) | average(field) | Design implication |
|---|---|---|---|---|
| Document matches query and numeric field exists | Included | Included | Included | Normal case |
| Document matches query but field is non-numeric | Included | Ignored | Ignored | Schema drift can make count and numeric aggregates describe different populations |
| Document matches query but field is missing | Included in plain count | No numeric contribution | No numeric contribution | Do not assume count is the denominator for average |
| Query adds orderBy on a field | Only documents where the ordered field exists are eligible | Same query population | Same query population | Ordering can silently narrow the population |
| Multiple aggregations depend on different fields | Population follows the fields required by the combined aggregation | May exclude documents lacking required fields | May exclude documents lacking required fields | Test the exact multi-aggregation shape, not each aggregate in isolation |
5. Standard cost: index entries are work, not magic metadata
Current Standard pricing charges aggregation queries by index entries read. Each batch of up to 1,000 index entries is billed as one document read, with a minimum charge of one read even when zero index entries are read. The local emulator cannot tell you a production bill, so use a deterministic calculator only to learn the formula and use managed Query Explain/actual billing evidence for production decisions.
export function standardAggregationReadCharges(indexEntriesRead) { if (!Number.isInteger(indexEntriesRead) || indexEntriesRead < 0) { throw new TypeError("indexEntriesRead must be a non-negative integer"); } return Math.max(1, Math.ceil(indexEntriesRead / 1000));}for (const n of [0, 1, 999, 1000, 1001, 1500, 10000]) { console.log({ indexEntriesRead: n, billedDocumentReads: standardAggregationReadCharges(n) });}// This formula is for Standard Core aggregation pricing described by current docs.// Do not reuse it for Enterprise Core/Pipeline billing, which uses read units/data processed.
A query that scans 1,500 index entries is therefore two billed reads under the documented Standard aggregation rule. This does not mean every aggregation over 1,500 documents scans exactly 1,500 entries; index shape, filters and query planning determine the actual entries examined.
6. Boundary: backend-only does not mean realtime
Aggregation queries cannot be attached to realtime listeners and cannot run as offline queries. A product screen may listen to individual review changes while separately refreshing a server aggregate, but those are two different freshness mechanisms. If a live badge must update with every accepted review, a materialized summary document may be a better read model. Lesson 3 develops that tradeoff.
If a read-time aggregation cannot complete within the
documented 60-second deadline, Firestore returns
DEADLINE_EXCEEDED. Treat that as a design signal:
improve the query/index shape, reduce the working set,
maintain a write-time aggregate, or move the workload to an
analytical system. Repeatedly retrying an intrinsically
oversized aggregation is not a scaling strategy.
7. Edition and mode matrix
| Surface | Aggregation model | Index/cost reasoning | Do not assume |
|---|---|---|---|
| Standard Native Core | count/sum/average over Core query | Required indexes; Standard aggregation index-entry read billing | Enterprise read-unit formula or Pipeline grouping |
| Enterprise Native Core | Core query aggregation exists in Enterprise context | Optional indexes; Enterprise read-unit/data-processed billing | Standard 1-per-1000 billing formula |
| Enterprise Native Pipeline | aggregate stage with accumulators and optional grouping | Pipeline query plan/data scan; Enterprise billing | That Core SDK methods describe all Pipeline capabilities |
| Enterprise MongoDB compatibility | MongoDB aggregate/count surfaces | MongoDB-compatible query/aggregation behavior and Enterprise billing | That Native Core/Pipeline syntax or semantics are drop-in equivalents |
| Emulator | Deterministic local behavior/rules learning | No production billing or scan-performance proof | That emulator timings or index enforcement certify production |
8. Deliberately wrong approach: download all reviews to “save aggregate cost”
This usually does the opposite. Downloading every matching document moves full document payloads over the network and incurs document reads, then spends client CPU to reproduce an answer the backend can calculate. The repaired design uses a backend aggregation when the working set is bounded and freshness-on-demand is acceptable. If the aggregation itself becomes frequent or very large, change the read model rather than returning to a client-side warehouse scan.
Verification checklist and cleanup
- Seed output states the expected truth before queries run.
-
Web/server aggregate results match
5 / 12 / 4for the published review fixture. - The learner can explain why the numeric average denominator is three, not five.
- No local cache or pending-write state is described as part of the aggregate.
- Standard cost examples are labeled as billing-model exercises, not emulator charges.
- Enterprise and MongoDB-compatible aggregation surfaces remain explicitly separate.
process.env.FIRESTORE_EMULATOR_HOST = "127.0.0.1:8080";process.env.GCLOUD_PROJECT = "demo-atlasmart-firestore";import { initializeApp } from "firebase-admin/app";import { getFirestore } from "firebase-admin/firestore";initializeApp({ projectId: "demo-atlasmart-firestore" });const db = getFirestore();for (const path of [ "products/p-1001/metrics", "products/p-1001/ratingCounterShards", "aggregateEvents"]) { await db.recursiveDelete(db.collection(path));}console.log("Chapter 16 derived aggregate state removed; shared source fixtures retained.");
Production judgment and bridge to Lesson 2
Use query-time aggregation when the answer can be computed on demand from a query whose scan size, latency and cost are acceptable. The next lesson adds filters, collection-group scope and Security Rules, where the same arithmetic can be correct yet the request can still fail because the query is not provably authorized or the required index/query shape is wrong.
Knowledge check
- Does count() use the local cache?
- Why can count() be 5 while average(rating) uses only 3 documents?
- How is a Standard aggregation that reads 1,500 index entries billed under the documented rule?
- Can that same formula be used for Enterprise?
- What does a 60-second deadline failure suggest?
Review the answers
1. No. Core aggregation queries are served directly by the backend and ignore local cache/pending local writes.
2. Plain count includes all five query matches; sum/average ignore the string rating and the missing rating, leaving three numeric values.
3. Two document reads, because charges are one read per batch of up to 1,000 index entries, with a minimum of one read.
4. No. Enterprise uses a read-unit/data-processed billing model.
5. The aggregation/query is too expensive for this read-time shape and should be optimized, materialized, narrowed, or moved to analytics rather than blindly retried.
Summary
AtlasMart now has a precise aggregation contract: backend-only Core arithmetic over an authorized query, field-sensitive numeric semantics, edition-aware cost reasoning, and a deterministic fixture that catches silent schema drift.
Authoritative references
- Summarize data with aggregation queries
- Understand Cloud Firestore billing
- Write-time aggregations
- Distributed counters
- Securely query data
- Connect to the Firestore Emulator and emulator differences
- Firestore Standard Core operations overview
- Firestore Enterprise Native Core/Pipeline overview
- Enterprise Pipeline aggregate stage
- Enterprise pricing examples
- MongoDB compatibility behavior differences
- Managed Firestore export/import
- Firebase Extensions migration guidance, including Stream Firestore to BigQuery
- Firebase JavaScript SDK release notes
- Firebase Admin Node.js SDK release notes