Traverse AtlasMart category relationships recursively with explicit depth, tenant scope, deterministic presentation, and memory/explain evidence.
$graphLookup for Recursive Relationships and Traversal Boundaries
Batch heterogeneous writes safely, interpret partial success, compare ordered and unordered execution, and use modern cross-namespace bulk APIs without assuming all-or-nothing behavior.
Learning objectives
Explain recursive graph traversal in terms of startWith, connectFromField, connectToField, depth, and visited documents.
Bound recursive work with maxDepth and restrictSearchWithMatch rather than trusting a small development graph.
Keep tenant/security scope inside the recursive search so traversal cannot cross logical ownership boundaries.
Recognize that graphLookup result order is not guaranteed and must be explicitly sorted for presentation.
Use explain and spill evidence to assess recursive memory and operational cost on representative data.
This lesson pins MongoDB Community Server
8.3.8 with
mongodb/mongodb-community-server:8.3.8-ubuntu2204-slim
and mongosh 2.10.0. The server is a disposable
standalone published only on loopback
127.0.0.1:27058. Authentication and TLS are
disabled only for this isolated lab. Feature Compatibility
Version (FCV) and allowDiskUseByDefault are
observed but never changed. Default read/write concern and
primary read preference apply. Atlas, Search, KMS, Enterprise
Advanced, and paid services are not required. Runtime output
shown as “expected” is documentation-derived because this
generation environment has no Docker/mongod/mongosh runtime.
The exact optimizer tree, execution counters, spill fields, and stage-specific explain shape can vary with patch version, FCV, indexes, data distribution, and topology. The lesson therefore names the invariant to verify—matched documents, traversal set/depth, facet counts, window values, indexes used, disk-use evidence, and target collection state—instead of requiring byte-for-byte explain output.
1. AtlasMart problem: descendant categories, not an unbounded graph crawl
AtlasMart’s category tree is stored as parent references. A
product-navigation request starting at “electronics” needs
descendants, but only for the current tenant, only active
categories, and only to a deliberate depth.
$graphLookup performs recursive matching: it starts
from startWith, matches that value against
connectToField, then follows each match’s
connectFromField to the next recursion step.
| Term | Operational meaning |
|---|---|
| startWith | Seed value(s) from the input document. |
| connectToField | Foreign field matched against the current frontier value. |
| connectFromField | Value(s) extracted from each match to form the next frontier. |
| maxDepth | Non-negative maximum recursion depth; depth 0 is one non-recursive lookup step. |
| depthField | NumberLong depth added to each traversed document; first match is depth 0. |
| restrictSearchWithMatch | Query-filter document applied during traversal; it is not an arbitrary aggregation-expression context. |
| fan-out | Number of neighbors reached per node; fan-out compounded by depth can make traversal grow rapidly. |
2. Seed a graph that can expose cross-tenant traversal
docker rm -f atlasmart-mongo-ch09-l2 2>/dev/null || truedocker volume rm atlasmart-mongo-ch09-l2-data 2>/dev/null || truedocker run -d --name atlasmart-mongo-ch09-l2 \ -p 127.0.0.1:27058:27017 \ -v atlasmart-mongo-ch09-l2-data:/data/db \ mongodb/mongodb-community-server:8.3.8-ubuntu2204-slimmongosh "mongodb://127.0.0.1:27058/atlasmart?directConnection=true" --quiet --eval \'printjson({server:db.version(),hello:db.hello().isWritablePrimary}); printjson(db.getSiblingDB("admin").runCommand({getParameter:1,featureCompatibilityVersion:1,allowDiskUseByDefault:1}))' const c=db.categories_ch09_l2;c.drop();c.insertMany([ {_id:"electronics",tenantId:"tenant-a",parentId:null,active:true}, {_id:"phones",tenantId:"tenant-a",parentId:"electronics",active:true}, {_id:"laptops",tenantId:"tenant-a",parentId:"electronics",active:true}, {_id:"android",tenantId:"tenant-a",parentId:"phones",active:true}, {_id:"ios",tenantId:"tenant-a",parentId:"phones",active:true}, {_id:"refurb",tenantId:"tenant-a",parentId:"phones",active:false}, {_id:"gaming",tenantId:"tenant-a",parentId:"laptops",active:true}, {_id:"tb-phones",tenantId:"tenant-b",parentId:"electronics",active:true}, {_id:"tb-budget",tenantId:"tenant-b",parentId:"tb-phones",active:true}]);c.createIndex({tenantId:1,parentId:1});printjson({count:c.countDocuments({}),indexes:c.getIndexes()});
Tenant-b also has a node whose parentId is
electronics. This makes an unsafe traversal
visibly cross the tenant boundary instead of letting a weak
fixture hide the bug.
3. Safe traversal: tenant filter + active filter + depth bound
const r=c.aggregate([ {$match:{_id:"electronics",tenantId:"tenant-a"}}, {$graphLookup:{ from:"categories_ch09_l2", startWith:"$_id", connectFromField:"_id", connectToField:"parentId", as:"descendants", maxDepth:2, depthField:"depth", restrictSearchWithMatch:{tenantId:"tenant-a",active:true} }}, {$project:{_id:1,descendants:{$sortArray:{input:"$descendants",sortBy:{depth:1,_id:1}}}}}]).toArray();printjson(r);depth 0: laptops, phonesdepth 1: android, gaming, iosrefurb is excluded because active=falsetenant-b nodes are excluded by restrictSearchWithMatch
The as array is not guaranteed to be ordered. The
projection uses $sortArray only for deterministic
display. Business logic must not infer path order from the raw
array position.
4. Controlled failure: omit tenant restriction
The unsafe version still starts from a tenant-a document, but
recursive matching is based on relationship fields, not an
implicit ownership boundary. Because tenant-b has a child whose
parentId matches the same root, it appears in the
traversal.
const unsafe=c.aggregate([ {$match:{_id:"electronics",tenantId:"tenant-a"}}, {$graphLookup:{from:"categories_ch09_l2",startWith:"$_id",connectFromField:"_id",connectToField:"parentId",as:"descendants",maxDepth:2}}, {$project:{_id:1,"descendants._id":1,"descendants.tenantId":1}}]).toArray();printjson(unsafe);
Put tenant and status/lifecycle constraints in
restrictSearchWithMatch when those constraints
must hold at every recursion step. Filtering the resulting
array afterward can already have traversed and exposed data
that should never have entered the result set.
5. maxDepth is a resource and semantics control
for(const d of [0,1,2]){ const x=c.aggregate([ {$match:{_id:"electronics",tenantId:"tenant-a"}}, {$graphLookup:{from:"categories_ch09_l2",startWith:"$_id",connectFromField:"_id",connectToField:"parentId",as:"d",maxDepth:d,depthField:"depth",restrictSearchWithMatch:{tenantId:"tenant-a",active:true}}}, {$project:{count:{$size:"$d"},ids:"$d._id"}} ]).toArray()[0]; print(`maxDepth=${d}`); printjson(x);}maxDepth=0 -> 2 descendantsmaxDepth=1 -> 5 descendantsmaxDepth=2 -> 5 descendants (no deeper active nodes in this fixture)Depth is not just a performance knob. It changes the business result. If a page promises “two levels of subcategories,” encode that contract explicitly rather than relying on the current graph’s accidental height.
6. Memory, disk use, and explain evidence
Under the current MongoDB 8.3 documentation, a
$graphLookup stage that exceeds 100 MB of memory
can write temporary files when disk use is allowed; with
allowDiskUse:false, exceeding the threshold returns
an error. The tiny fixture will not force a spill. Use
representative fan-out/depth distributions and inspect
executionStats, server metrics, temporary-file
evidence, and latency percentiles.
const e=c.explain("executionStats").aggregate([ {$match:{_id:"electronics",tenantId:"tenant-a"}}, {$graphLookup:{from:"categories_ch09_l2",startWith:"$_id",connectFromField:"_id",connectToField:"parentId",as:"d",maxDepth:2,depthField:"depth",restrictSearchWithMatch:{tenantId:"tenant-a",active:true}}}],{allowDiskUse:true});printjson(e);The chapter pins 8.3.8 because memory/spill behavior is version-sensitive. Do not copy an older-version graphLookup memory rule into an 8.3 production runbook without re-checking current documentation.
7. Verification, cleanup, and production judgment
Verification checklist
- Nine category documents exist, including inactive and other-tenant nodes.
- The safe traversal returns only active tenant-a descendants.
- The unsafe traversal visibly includes tenant-b data.
-
maxDepth=0andmaxDepth=1produce different descendant counts. - Raw graphLookup output order is not treated as a path-order guarantee.
- No claim of an actual disk spill is made from the tiny fixture.
Production judgment.
$graphLookup is appropriate for bounded recursive
relationships that remain selective enough to execute
predictably. It does not turn a high-fan-out graph into a cheap
tree query. On sharded deployments, MongoDB supports a sharded
from collection starting in 5.1, but a graph lookup
targeting a sharded collection cannot be used inside a
transaction. Track traversal depth, visited cardinality,
temporary-disk use, latency tails, and tenant-scope failures.
For hot or repeated traversals, consider precomputed paths,
materialized hierarchy fields, or a different data model rather
than merely increasing resource limits.
Lesson 3 shifts from one recursive dimension to multiple
analytic dimensions in a single $facet, where
memory has a very different rule.
docker rm -f atlasmart-mongo-ch09-l2docker volume rm atlasmart-mongo-ch09-l2-dataCheck your understanding
- What does maxDepth=0 mean?
- Why can starting from a tenant-a root still leak tenant-b nodes?
- Is the graphLookup output array guaranteed to be traversal order?
- Why should maxDepth be treated as part of business semantics?
- What current 8.3 behavior applies when graphLookup exceeds 100 MB and disk use is allowed?
Review the answers
One non-recursive lookup step: matching documents are depth 0 and no further recursion is followed.
Because recursive matching follows relationship-field values; ownership is not implicit. A foreign node with the same parent value matches unless the traversal filter excludes it.
No. Sort the array explicitly if presentation order matters.
It changes which descendants are considered part of the result, not only resource consumption.
The current 8.3 documentation says it can write temporary files; with allowDiskUse:false, exceeding the threshold errors.
Authoritative references
- MongoDB 8.3 release notes — Current 8.3 behavior and version-sensitive aggregation changes; re-check before reproducing.
- MongoDB aggregation pipeline — Ordered-stage execution model used throughout the chapter.
- Aggregation pipeline limits — Memory, disk-spill, stage-count, and 16 MiB output-document constraints.
- mongosh release notes — mongosh version used for the chapter commands.
- $graphLookup stage — Recursive semantics, maxDepth, depthField, sharded restrictions, result ordering, and current memory behavior.
- $sortArray expression — Explicit array sorting for deterministic presentation.