Create / Manage Firestore Vector Indexes and Understand Supported Dimensions / Query Limits

1. AtlasMart problem: a correct vector can still be unqueryable without the right managed index

The local oracle can rank six vectors without Firestore, but managed KNN depends on a vector index whose field, dimension and prefilter fields match the query shape. AtlasMart needs an index lifecycle that is reviewable and deployable, not an “error message → click create → forget” ritual.

Learning outcomes

2. Flat is the current Standard Native vector-index type

Current Firestore vector-search documentation requires flat for the vector configuration. The index dimension must match the stored/query vector dimension. AtlasMart’s teaching fixture uses dimension 8; a production model can use another supported dimension up to 2,048. Choosing a smaller fixture does not imply a tuning recommendation.

# Optional managed Standard Native exercise; billing/real project may be required.gcloud components updategcloud firestore indexes composite create \  --collection-group=vectorProducts \  --query-scope=COLLECTION \  --field-config=order=ASCENDING,field-path="tenantId" \  --field-config=order=ASCENDING,field-path="category" \  --field-config=field-path="embeddingV1",vector-config='{"dimension":"8","flat":"{}"}' \  --database='(default)'# Inspect current indexes in the selected project/database before querying.gcloud firestore indexes composite list --database='(default)' 

3. Index shape follows the retrieval contract

A single-field vector index is enough for an unfiltered KNN search on one collection. If AtlasMart must first constrain by tenantId and category, the index needs those scalar fields plus the vector field. The filter reduces the candidate population before nearest-neighbor evaluation, which improves isolation and can reduce scanned work, but only if application authorization guarantees the filter cannot be broadened by an untrusted caller.

4. Index readiness is operational state

Index creation is asynchronous. Treat “command accepted” and “index ready” as separate states. A release that depends on a new vector index should verify readiness before shifting traffic. If the index is missing, Firestore can suggest a creation command; that suggestion is useful, but the application still needs reviewable infrastructure ownership and rollback instructions.

export function validateSearchContract({ vector, expectedDimension, modelVersion }) {  if (!Array.isArray(vector) || vector.length !== expectedDimension) {    throw new Error(`dimension mismatch: expected ${expectedDimension}, got ${vector?.length}`);  }  if (modelVersion !== 1) throw new Error(`unsupported embeddingVersion ${modelVersion}`);  if (vector.some(v => !Number.isFinite(v))) throw new Error("non-finite vector value");  return true;}validateSearchContract({ vector:[0,0,0,0,0,0,0,0], expectedDimension:8, modelVersion:1 });

5. Standard Core limits that change design

Current Standard Native documentation caps an embedding at 2,048 dimensions and a nearest-neighbor query at 1,000 returned documents. It also documents no realtime snapshot listeners for vector queries and server-library support in Python, Node.js, Go and Java. Those are implementation boundaries, not reasons to request 1,000 neighbors. K should be chosen from product needs and quality evaluation.

6. Emulator trap: “the query worked locally” is not an index test

The Firestore emulator does not enforce production composite-index requirements. Therefore an emulator query succeeding without firestore.indexes.json or a managed vector index does not prove the production query can run. Use the emulator for data/rules/application tests, and an isolated managed project for index-readiness tests when release risk justifies it.

7. Optional managed Node.js query

import { Firestore, FieldValue } from "@google-cloud/firestore";const db = new Firestore({ projectId: process.env.GCLOUD_PROJECT });const q = [0.90,0.06,0.02,0.02,0.24,0.05,0.01,0.13];const snap = await db.collection("vectorProducts")  .where("tenantId", "==", "tenant-a")  .where("category", "==", "cameras")  .findNearest("embeddingV1", FieldValue.vector(q), {    limit: 5,    distanceMeasure: "EUCLIDEAN",    distanceResultField: "vectorDistance"  })  .get();for (const doc of snap.docs) {  console.log(doc.id, doc.get("vectorDistance"));}

8. Wrong approach: silently change the field dimension in place

Suppose embeddingV1 moves from 8 dimensions to 768 but the index and older documents remain 8-D. That is not a gradual schema evolution; it is a broken vector contract. The repair is a new versioned field/index, a backfill with explicit progress state, dual-read evaluation, then a controlled cutover.

9. Index ownership and cleanup

Record an owner, purpose, model version, dimension, scalar prefilters, creation command, rollout date and removal gate for every vector index. Deleting an index too early can break live search; keeping obsolete indexes forever increases storage/write overhead and complicates migrations.

{  "owner": "atlasmart-search",  "collectionGroup": "vectorProducts",  "vectorField": "embeddingV1",  "dimension": 8,  "indexType": "flat",  "prefilters": ["tenantId ASC", "category ASC"],  "embeddingModel": "atlasmart-local-v1",  "embeddingVersion": 1,  "retireAfter": "v2 cutover + rollback window + zero v1 query traffic"}

10. Edition/mode/client matrix

Verification checklist

• The optional gcloud command declares dimension 8 and flat.
• The query vector is validated before a managed call.
• The scalar prefilter fields are part of the composite vector index.
• Index creation and index readiness are treated as different states.
• Standard’s 1,000-result and supported-library statements are not copied to Enterprise Pipeline without checking its docs.

Production judgment and bridge to Lesson 3

An index makes KNN executable; it does not make the ranking useful. Lesson 3 turns to distance measures, thresholds, metadata prefilters and ranking interpretation, using the same six-product fixture so index mechanics and retrieval quality remain separate variables.

Knowledge check

1. What vector index type does current Standard Native documentation require?
2. Why does AtlasMart include tenantId/category in a composite vector index?
3. Does emulator success prove a production composite/vector index exists?
4. What is the current Standard Core maximum returned-document count for a nearest-neighbor query?
5. What is safer than overwriting an 8-D field with a 768-D model?

Authoritative references

• Firebase · Search with vector embeddings
• Google Cloud · Firestore Standard pricing
• Firebase · Connect to the Firestore emulator and understand differences from production
• Google Cloud · Enterprise Native Pipeline findNearest stage
• Firebase · Enterprise Native supported data types
• Google Cloud · Vertex AI embedding sample