N+1 Graph Queries, Chatty Traversals, Projection Design, and Returning Only Needed Data

Learning outcomes

AtlasMart’s order page first asks for 50 orders, then loops over them and issues one query per order for products, then another query per product for supplier data. The database may execute every query quickly, yet the page is slow because hundreds of Bolt round trips, pool borrows and serializations accumulate. This is the graph version of the N+1 problem: a client traversal encoded as chatty network calls.

Reproducible AtlasMart setup

$env:NEO4J_URI='bolt://127.0.0.1:7687'$env:NEO4J_USER='neo4j'$env:NEO4J_PASSWORD='atlasmart-course-2026'$env:NEO4J_DATABASE='neo4j'py -3 -m venv .venv.\.venv\Scripts\Activate.ps1python -m pip install --upgrade pippython -m pip install neo4j==6.3.0

CYPHER 25CREATE CONSTRAINT customer_id IF NOT EXISTSFOR (c:Customer) REQUIRE c.customerId IS UNIQUE;CREATE CONSTRAINT product_id IF NOT EXISTSFOR (p:Product) REQUIRE p.productId IS UNIQUE;CREATE CONSTRAINT order_id IF NOT EXISTSFOR (o:Order) REQUIRE o.orderId IS UNIQUE;MERGE (c:Customer {customerId:'C-1001'})SET c.name='Mina Rahimi', c.tier='GOLD'MERGE (p1:Product {productId:'P-1001'})SET p1.name='Trail Camera', p1.category='Cameras', p1.price=129.90MERGE (p2:Product {productId:'P-2001'})SET p2.name='Smart Shelf Sensor', p2.category='Store IoT', p2.price=79.50MERGE (o:Order {orderId:'O-5001'})SET o.status='PAID', o.orderedAt=datetime('2026-09-08T16:30:00Z')MERGE (c)-[:PLACED]->(o)MERGE (o)-[r1:CONTAINS]->(p1)SET r1.quantity=1, r1.unitPrice=129.90MERGE (o)-[r2:CONTAINS]->(p2)SET r2.quantity=2, r2.unitPrice=79.50;

The fixture is intentionally tiny. Performance conclusions come from the synthetic workload generated by the harness, not from pretending two products are representative of production. The stable keys and uniqueness constraints make repeated batch-write experiments safe to reconcile.

1. N+1 is a request-shape problem

async def wrong_order_feed(driver, customer_id):    records, _, _ = await driver.execute_query(        "CYPHER 25 MATCH (:Customer {customerId:$id})-[:PLACED]->(o:Order) RETURN o.orderId AS id",        id=customer_id, database_="neo4j")    feed = []    for row in records:        lines, _, _ = await driver.execute_query(            "CYPHER 25 MATCH (:Order {orderId:$id})-[r:CONTAINS]->(p:Product) RETURN p, r.quantity AS q",            id=row["id"], database_="neo4j")        feed.append({"orderId": row["id"], "lines": [x.data() for x in lines]})    return feed

For N orders this is at least N+1 database requests. Async parallelism can hide some wall-clock waiting but increases concurrent load; it does not erase network handshakes, pool pressure or duplicate planning/execution work.

2. One bounded traversal + projection

CYPHER 25MATCH (:Customer {customerId:$customerId})-[:PLACED]->(o:Order)OPTIONAL MATCH (o)-[r:CONTAINS]->(p:Product)WITH o,     collect(CASE WHEN p IS NULL THEN null ELSE {       productId:p.productId,       name:p.name,       quantity:r.quantity,       unitPrice:r.unitPrice     } END) AS maybeLinesRETURN {  orderId:o.orderId,  status:o.status,  orderedAt:o.orderedAt,  lines:[x IN maybeLines WHERE x IS NOT NULL]} AS orderORDER BY order.orderedAt DESC, order.orderIdLIMIT $limit;

async def order_feed(driver, customer_id, limit=50):    records, summary, _ = await driver.execute_query(        QUERY,        customerId=customer_id,        limit=limit,        database_="neo4j",        routing_="r",    )    return [r["order"] for r in records], summary

The query has an explicit API bound ($limit) and returns a stable DTO-like map. Returning entire nodes/relationships makes payload size and accidental field disclosure track graph evolution rather than API ownership.

3. Compare causal evidence, not syntax

Do not assume the one-query version is automatically fastest. Profile it with realistic graph density. A single query that accidentally traverses millions of relationships can be worse than a deliberately segmented API. The design objective is minimal necessary round trips with bounded server cardinality.

4. Wrong repair: collect the whole subgraph

// WRONG for an API feed: no customer/page bound and raw entities.CYPHER 25MATCH (o:Order)-[r:CONTAINS]->(p:Product)RETURN collect({order:o, rel:r, product:p}) AS everything;

This hides chatty access by creating one giant response. It can consume query/client memory, serialize properties the API never uses and create security coupling to future graph properties. The repair is to bound the anchor, project needed scalar/map fields and paginate with a stable order.

5. Result-shaping checklist

6. Mini benchmark harness

import json, timeasync def measure(label, fn):    t0 = time.perf_counter()    value = await fn()    elapsed = (time.perf_counter() - t0) * 1000    payload = json.dumps(value, default=str, separators=(",", ":")).encode()    return {"variant":label, "elapsed_ms":elapsed, "result_bytes":len(payload)}# Run each variant repeatedly after warmup with the same customer/page size.# Also count actual driver query calls in your repository instrumentation.

Production judgment

Summary and next step

High-throughput graph services reduce avoidable round trips and return bounded application-owned projections. Lesson 5 puts all client/server stages on one measurement timeline so “database time” is not blamed for everything.

Authoritative references

• Current Neo4j versions — Current server and 5.26 LTS release snapshot.
• Neo4j Python Driver Manual — Official Python driver guide used by the mandatory async lab.
• Python Driver 6.3 API — Current synchronous and asynchronous API contracts and configuration.
• Python async API — Async driver/session/result lifecycle, cancellation and concurrency rules.
• Python concurrency guide — AsyncGraphDatabase and concurrent workflow guidance.
• Python performance recommendations — Driver performance, routing and batching guidance.
• Cypher UNWIND — List-to-row semantics and ordering boundary.
• Transaction management — Server-side transaction limits and timeout concepts.
• Python simple queries — Parameterized execute_query and summary timing.
• Cypher ORDER BY — Deterministic ordering boundary used by paged projections.