Page Cache vs Heap, Transaction Memory, Query Memory, File Descriptors, CPU, Storage I/O, and Network

AtlasMart sees a memory alert and a disk-latency spike at the same time. The fastest wrong response is to increase heap. That could actually reduce room for page cache and the operating system, increasing disk pressure. The correct question is: which memory/resource plane is constrained, and what work is consuming it?

Learning outcomes

Reproducible lab baseline

1. Four memory planes that must not be collapsed

2. Read the configured budget

CYPHER 25
SHOW SETTINGS
YIELD name, value, isDynamic, isExplicitlySet
WHERE name IN [
  'server.memory.heap.initial_size',
  'server.memory.heap.max_size',
  'server.memory.pagecache.size',
  'dbms.memory.transaction.total.max',
  'db.memory.transaction.total.max',
  'db.memory.transaction.max',
  'db.tx_log.rotation.size',
  'db.tx_log.rotation.retention_policy'
]
RETURN name, value, isDynamic, isExplicitlySet
ORDER BY name;

3. Observe a bounded memory-heavy query

The following query materializes a list of maps. It is intentionally inefficient for teaching: start at 50,000 rows in a disposable lab and increase only if your workstation has headroom. While it runs, inspect the same user’s transaction from another session.

CYPHER 25
// Disposable lab only. Start smaller; increase gradually while observing transaction memory.
UNWIND range(1,50000) AS n
WITH collect({n:n, bucket:n % 100}) AS rows
RETURN size(rows) AS materializedRows,
       reduce(s=0, x IN rows | s + x.n) AS checksum;

CYPHER 25
SHOW TRANSACTIONS
YIELD transactionId, username, database, currentQuery, currentQueryStatus,
      elapsedTime, waitTime, currentQueryElapsedTime, currentQueryWaitTime,
      activeLockCount, resourceInformation,
      currentQueryAllocatedBytes, allocatedDirectBytes, estimatedUsedHeapMemory,
      pageHits, pageFaults, currentQueryPageHits, currentQueryPageFaults,
      indexes, runtime, statusDetails
RETURN *
ORDER BY currentQueryElapsedTime DESC;

4. Repair the query before the server

If the business requirement is a checksum/count, do not collect 50,000 maps just to reduce them afterward. Stream rows through aggregation instead.

CYPHER 25
UNWIND range(1,50000) AS n
RETURN count(*) AS rowCount, sum(n) AS checksum;

5. Page cache and disk are coupled

The page cache avoids disk access for frequently used store pages. After a restart it begins cold; page faults and block reads can be high until the active working set warms. If the store+native-index working set is much larger than the page-cache budget, sustained random traversal can continue faulting. But a high page-cache hit ratio cannot excuse high p99 caused by locks, CPU saturation, network, or huge result serialization.

6. File descriptors / handles

Neo4j and Lucene open many files. Enterprise metrics expose current and maximum file-descriptor gauges on supported JVM/OS combinations. The free lab can inspect host/container process handles using platform tools. The question is not “is 5,000 high?” but “is usage approaching the configured OS limit, growing unexpectedly, or correlated with failures?”

# Portable container-level snapshot (Docker Desktop / Linux / macOS):
docker stats --no-stream atlasmart-neo4j

# Neo4j store/index sizing guidance from inside the official image:
docker exec atlasmart-neo4j neo4j-admin server memory-recommendation

# Linux host examples when Neo4j runs directly (not Docker Desktop VM):
# ps -o pid,%cpu,%mem,rss,vsz,etime -p <neo4j_pid>
# ls /proc/<neo4j_pid>/fd | wc -l
# iostat -xz 1
# ss -s

# Windows host examples for a directly running Neo4j process:
# Get-Process neo4j | Select-Object Id,CPU,WorkingSet64,PrivateMemorySize64,HandleCount
# Get-Counter '\\PhysicalDisk(*)\\Avg. Disk sec/Read','\\Network Interface(*)\\Bytes Total/sec'

7. CPU, storage I/O and network need denominator/context

8. Memory budget worksheet

Production judgment

Summary and next step

Resource planes are now separated. Lesson 3 joins that resource evidence to Cypher plans, running transactions, lock waits and application timing for a concrete slow-request diagnosis.

Authoritative references

• Current Neo4j versions — Current Neo4j 2026.07.1 and 5.26.30 LTS release snapshot.
• Monitoring overview — Current monitoring surfaces: logs, metrics, query/transaction management, connections, jobs, and reports.
• Metrics — Enterprise metrics architecture and supported export surfaces.
• Essential metrics — Server, Neo4j, cluster, and workload signals that require correlation rather than single-metric tuning.
• Metrics reference — Current page-cache, JVM, CPU, file-descriptor, transaction and store metric names.
• Logging — Current neo4j/debug/http/gc/query/security log behavior and edition boundaries.
• Show and terminate transactions — Current SHOW TRANSACTIONS fields for locks, wait time, memory, page hits/faults, runtime, indexes, and status.
• Manage queries — Current query visibility through transaction management rather than legacy dbms.listQueries().
• Memory configuration — Heap, page cache, native memory, transaction limits, and memory-recommendation guidance.
• Disks, RAM and other tips — Page-cache warmup, disk behavior, and operating-system considerations.
• Performance — Performance topics spanning memory, disks, statistics, execution plans, Bolt threads, and filesystem tuning.
• Show configuration settings — Current SHOW SETTINGS syntax and self-managed server scope.
• Transaction logs — Transaction-log location, rotation, retention, pruning, and capacity implications.
• Query tuning and PROFILE — Execution-plan evidence for rows, database hits, memory, access paths, and operator shape.
• Python driver performance — Driver-side timing, database selection, result consumption, and application/database boundary considerations.