Composite Indexes: Ordering, Equality / Range Fields, Collection vs Collection-Group Scope, and Error-Driven Creation

Learning outcomes

AtlasMart now has three concrete query families: catalog category + price, tag membership + rating order, and seller order history across user subcollections. The mistake is to translate each UI screen into an arbitrary pile of index definitions. A manual index should be a direct consequence of filter/order semantics and query scope.

1. A composite index is an ordered access path

A manual/composite index is not “a set of fields.” It is an ordered list of fields, each with an index mode, bound to a collection ID and query scope. Firestore uses equality predicates to narrow a prefix and then uses the query's range/order fields to determine how it scans the ordered entries. Changing field order can change how much of the index must be scanned even when two definitions contain the same names.

For equality-only fields in a manual index, Firestore still requires an ascending or descending mode for scalar fields, but that choice does not change equality semantics. For fields used in ordering, the direction is part of the query contract. Standard also appends document-name ordering to many indexes as the final tie-break dimension; when you need a different document-name direction, define it explicitly.

2. Capture only the query-owned indexes

{  "indexes": [    {      "collectionGroup": "catalogItems",      "queryScope": "COLLECTION",      "fields": [        { "fieldPath": "category", "order": "ASCENDING" },        { "fieldPath": "price", "order": "ASCENDING" }      ]    },    {      "collectionGroup": "orders",      "queryScope": "COLLECTION_GROUP",      "fields": [        { "fieldPath": "tenantId", "order": "ASCENDING" },        { "fieldPath": "createdAt", "order": "DESCENDING" }      ]    },    {      "collectionGroup": "catalogItems",      "queryScope": "COLLECTION",      "fields": [        { "fieldPath": "tags", "arrayConfig": "CONTAINS" },        { "fieldPath": "rating", "order": "DESCENDING" }      ]    }  ],  "fieldOverrides": [    { "collectionGroup": "catalogItems", "fieldPath": "description", "indexes": [] },    { "collectionGroup": "catalogItems", "fieldPath": "attributes", "indexes": [] },    { "collectionGroup": "catalogEvents", "fieldPath": "expiresAt", "ttl": true, "indexes": [] }  ]}

The third index is intentionally marked experimental for this chapter. The current Chapter 05 contract does not require “tag + rating descending” in its core acceptance suite, so retaining it without a screen/query owner would be unjustified write/storage overhead. Lesson 5 will remove it after proving no contract depends on it.

3. Error-driven creation is a hint, not a governance process

In managed Standard Firestore, a valid compound query that lacks an index typically fails with a FAILED_PRECONDITION-style response and a console link populated with the suggested index. Vector-index failures instead can provide a Google Cloud CLI command. The error proves one thing: this query cannot run with the current managed index state. It does not prove the suggested index is desirable, non-duplicate, affordable, correctly scoped for your architecture, or safe to keep forever.

A disciplined workflow is: reproduce the owning query in a test, inspect the suggested fields/scope, compare against the repository manifest, approve or remodel, commit the definition, deploy to a disposable/controlled environment, wait for build readiness, rerun the exact query, then retain evidence linking query → index → test.

4. Test scope explicitly: collection is not collection group

import assert from "node:assert/strict";import { initializeApp } from "firebase/app";import { collection, collectionGroup, connectFirestoreEmulator, getDocs, getFirestore, orderBy, query, where } from "firebase/firestore";const app=initializeApp({projectId:"demo-atlasmart-firestore",apiKey:"demo",appId:"demo"});const db=getFirestore(app); connectFirestoreEmulator(db,"127.0.0.1",8080);const ids=async q => (await getDocs(q)).docs.map(d=>d.id);const cameras=query(collection(db,"catalogItems"),where("category","==","camera"),orderBy("price","asc"));assert.deepEqual(await ids(cameras),["p-1001","p-1002","p-1005"]);const sellerOrders=query(collectionGroup(db,"orders"),where("tenantId","==","seller-a"),orderBy("createdAt","desc"));assert.deepEqual((await getDocs(sellerOrders)).docs.map(d=>d.ref.path),[  "users/u-bob/orders/o-1003",  "users/u-alice/orders/o-1001"]);console.log("query owners passed");

The test owns semantics, not performance. In production Standard, the first query is tied to a collection-scope manual index and the second to a collection-group manual index. If you accidentally deploy the second definition with COLLECTION scope, the collection-group query remains unsupported even though the field names look correct.

5. Field ordering follows selectivity and sort semantics

For a query with equality filters plus ranges/order, equalities normally belong first, followed by the first range/order fields in the order that best reduces the scan while still satisfying query ordering rules. When multiple range/inequality fields exist, current guidance is to order the index after equality fields by the most selective range constraints where compatible with the required sort. Query Explain can reveal indexesUsed, index entries scanned, documents scanned, and execution/billing details on the managed service.

Do not hard-code folklore such as “always put the most selective field first” without respecting the query's order requirements. The query and index must agree; an apparently selective index that cannot serve the required order is not a substitute.

// Run only against an isolated managed project/database with IAM credentials.const q = db.collection("catalogItems")  .where("category", "==", "camera")  .orderBy("price", "asc");const explain = await q.explain({ analyze: "true" });console.dir(explain.metrics.planSummary, { depth: null });console.dir(explain.metrics.executionStats, { depth: null });// Record actual values; never paste tutorial numbers as your benchmark.

6. Failure injection: wrong scope, duplicate structure, premature deletion

Three controlled mistakes teach more than a happy path:

1. Change the orders index scope to COLLECTION and document why it cannot own the collection-group query.
2. Add a second category + price definition with no distinct query owner; your manifest audit should flag it as duplicate/redundant even before deployment.
3. Simulate removing category + price from the repository and run the query-owner tests. The local query still succeeds because the emulator does not enforce the managed index, which is itself the lesson: CI needs a configuration test as well as an emulator query test.

import assert from "node:assert/strict";import fs from "node:fs";const cfg=JSON.parse(fs.readFileSync("firestore.indexes.json","utf8"));const canon=x=>JSON.stringify(x);const required={collectionGroup:"catalogItems",queryScope:"COLLECTION",fields:[  {fieldPath:"category",order:"ASCENDING"},{fieldPath:"price",order:"ASCENDING"}]};assert.ok(cfg.indexes.some(x=>canon(x)===canon(required)),"category+price index contract missing");const cg=cfg.indexes.find(x=>x.collectionGroup==="orders"&&x.queryScope==="COLLECTION_GROUP");assert.ok(cg,"orders collection-group index missing");console.log("index configuration contracts passed");

Hands-on lab and rollback plan

1. Seed the Chapter 05/06 fixtures and run the collection and collection-group query tests.
2. Save the three-index experimental manifest and run the configuration contract test.
3. Document each manual index with owner, query signature, scope, reason, expected delete impact, and rollback definition.
4. Optional managed project: deploy with firebase deploy --only firestore:indexes, observe build state, rerun owning queries only after the relevant indexes are ready, and capture Query Explain evidence where useful.
5. Never delete an index as the first experiment. First remove/disable the query dependency or prove no owner remains; keep the exact JSON definition so rollback is re-add → wait ready → re-enable traffic.

Production judgment

An index should be treated like an API dependency with ownership and tests. Index count alone is not the objective; the objective is the smallest set of structures that support authorized, stable, cost-justified query contracts. Build time/backfill can be operationally significant, so migrations must account for readiness instead of assuming deploy is instantaneous.

Knowledge check

1. Why can the same two field names define two materially different indexes?
2. Why does a collection-group query need its own scope?
3. What should you do with a console missing-index link?
4. Why do you need a configuration contract test if emulator query tests pass?
5. What is the safe index-removal rollback sequence?

Summary and next step

Composite indexes are ordered, scoped query dependencies, not an error-message scavenger hunt. Lesson 3 turns to the opposite decision: which fields should deliberately not be indexed because their size, cardinality, sequential nature, or lifecycle makes the default expensive.

Authoritative references

• Index types in Cloud Firestore — Standard automatic/manual index modes, scopes, entry limits, exemptions, and index fan-out guidance.
• Manage indexes in Cloud Firestore — Missing-index workflow, roles, build state, CLI/console management, and vector indexes.
• Cloud Firestore Index Definition Reference — Current firestore.indexes.json schema, vector configuration, field overrides, and TTL configuration.
• Best practices for Cloud Firestore — Index fan-out, sequential-field, TTL, large string/array/map exemption guidance.
• Understand query performance using Query Explain — Planner versus analyze evidence and billing/scan statistics for managed Firestore.
• Enterprise edition index overview — Optional indexing, sparse/dense behavior, and query-performance reasoning in Enterprise Native mode.
• Firestore Native mode Core/Pipeline overview — Standard/Enterprise indexing requirements and interface differences.
• Search with vector embeddings — Vector index management, flat index type, supported dimensions, and vector-search limitations.
• Firestore pricing — Current document/index-entry billing semantics; re-check region and edition before budgeting.
• Firebase release notes — Current CLI/SDK versions used by the pinned lab.
• Firestore release notes — Enterprise Native/Pipeline launch-stage changes and emulator support.