Evaluate a Graph ML Workflow Against Non-Graph Baselines and Document Where the Graph Actually Adds Signal

Learning outcomes

1. AtlasMart final question: does graph information improve the decision enough to pay for itself?

The chapter succeeds even if the graph model does not beat the baseline. The correct conclusion is empirical: compare the same supervised target, feature cutoff, split protocol, trainer family and metric; change only the graph-derived feature set. Then judge the observed lift against added complexity, compute, refresh cadence, serving dependencies, privacy/security scope, and operational risk.

2. Acceptance criteria before reading any scores

3. Read the two models from the catalog side by side

CALL gds.model.list()
YIELD modelName, modelType, modelInfo, trainConfig, graphSchema
WHERE modelName IN ['atlas-ch25-baseline-model','atlas-ch25-graph-model']
RETURN modelName,
       modelType,
       modelInfo.metrics.F1_WEIGHTED.test AS testF1Weighted,
       modelInfo.bestParameters AS bestParameters,
       trainConfig,
       graphSchema
ORDER BY modelName;

4. Ablation matrix: what changed?

5. Sensitivity without test-set fishing

After the primary comparison is frozen, run a planned sensitivity study. For example, retrain one experiment at a time with seeds 41, 42, and 43, dropping each temporary model before the next so Community model capacity is not exceeded. Compare validation variability and only use a newly reserved holdout period if the experiment design changes after inspecting the original test.

// Example process, not a loop to paste blindly:
// 1. Keep baseline + active graph model in catalog.
// 2. Train at most one temporary sensitivity model (third Community slot).
// 3. Record validation/test evidence under the predeclared protocol.
// 4. Drop the temporary model before the next seed/config.
CALL gds.model.list()
YIELD modelName, modelType
RETURN modelName, modelType ORDER BY modelName;

6. Decision template: graph-added-value statement

7. Failure injection: prove the evaluation catches a leaked graph

In a disposable copy only, deliberately add one target-window relationship type such as FUTURE_PURCHASE_CH25 that directly follows willRepeat, then project it into a throwaway graph and observe how easily model metrics can inflate. Do not use the contaminated graph for the real comparison. The educational point is that a “better” score can be a data-integrity alarm rather than progress.

// DISPOSABLE DEMONSTRATION ONLY. Do not add this relationship to atlas-ch25-ml.
MATCH (c:CH25Customer), (p:CH25Product {productId:'CH25-P-01'})
WHERE c.willRepeat = 1
CREATE (c)-[:FUTURE_PURCHASE_CH25 {chapter25:true, syntheticLeak:true}]->(p);

// Verify the contamination exists, explain why it is forbidden, then delete it.
MATCH (:CH25Customer)-[r:FUTURE_PURCHASE_CH25]->(:CH25Product)
RETURN count(r) AS leakedFutureEdges;

MATCH ()-[r:FUTURE_PURCHASE_CH25]->() DELETE r;

8. Production-readiness matrix

9. Chapter 25 final verification checklist

1. Record gds.version(), Neo4j version/edition, Cypher version, Java/runtime and deployment.
2. Confirm only feature-window relationships enter atlas-ch25-ml.
3. Record class counts and split configuration before training.
4. Train the non-graph baseline and graph-augmented model under the same protocol.
5. Copy actual test metrics, winning parameters, train time and estimate evidence into the experiment log.
6. Run at least one predeclared sensitivity/ablation check without test-set fishing.
7. Write an explicit graph-added-value conclusion including cost and non-guarantees.
8. Drop Chapter 25 models, pipelines and projection and delete only Chapter 25 fixtures.

// Check names before dropping. These calls fail if a named object does not exist,
// so run the corresponding list() command first when rerunning a partial lab.
CALL gds.model.list() YIELD modelName
RETURN modelName ORDER BY modelName;
CALL gds.pipeline.list() YIELD pipelineName
RETURN pipelineName ORDER BY pipelineName;
CALL gds.graph.list() YIELD graphName
RETURN graphName ORDER BY graphName;

// Drop the two Chapter 25 models when present.
CALL gds.model.drop('atlas-ch25-baseline-model') YIELD modelName;
CALL gds.model.drop('atlas-ch25-graph-model') YIELD modelName;

// Drop the two training pipelines when present.
CALL gds.pipeline.drop('atlas-ch25-baseline-pipe') YIELD pipelineName;
CALL gds.pipeline.drop('atlas-ch25-graph-pipe') YIELD pipelineName;

// Drop the in-memory graph when present.
CALL gds.graph.drop('atlas-ch25-ml') YIELD graphName;

// Delete only persisted synthetic Chapter 25 data.
MATCH (n) WHERE n.chapter25 = true DETACH DELETE n;

10. Bridge to Chapter 26: once the model is correct, make the workload sustainable

Chapter 26 moves from analytical correctness to end-to-end performance engineering: workload mix, fan-out, traversal depth, hot nodes, result sizes, query selectivity, heap/page cache/transaction memory, storage, containers and saturation. A graph ML workflow that adds predictive signal but starves production queries is not production-ready.

Summary and next step

Evaluate a Graph ML Workflow Against Non-Graph Baselines and Document Where the Graph Actually Adds Signal is useful only when its assumptions and observed evidence stay attached to the decision. The examples above establish a reproducible mechanism and boundary; they do not turn one lab result into a universal production rule.

Next, continue to Profile Workload Mix: Read/Write Ratios, Traversal Depth, Fan-Out, Hot Nodes, Result Sizes, and Concurrency. Carry forward the verified assumptions, fixture state, version/edition boundaries, and measurements from this lesson instead of treating the next topic as an isolated recipe.

Authoritative references

• GDS Manual v2026.07 — Current GDS manual and release baseline.
• GDS supported Neo4j versions — Compatibility matrix for Neo4j and GDS releases.
• GDS editions — Community/Enterprise limits including four-core concurrency and three-model catalog capacity.
• Node embeddings overview — Current embedding families, quality tiers, inductive/transductive guidance.
• Fast Random Projection — FastRP dimensions, propertyRatio, featureProperties, randomSeed, iterations and modes.
• Node2Vec — Current Node2Vec algorithm and transductive embedding behavior.
• Machine learning pipelines — Current node classification, link prediction and node regression pipeline tiers.
• Node classification pipelines — End-to-end node classification semantics and prediction model behavior.
• Link prediction pipelines — Feature/train/test split semantics and negative examples.
• Link prediction configuration — Split configuration, FastRP property steps, link features and candidate models.
• Link prediction training — Cross-validation, model selection and model-catalog registration semantics.
• Training methods — Supported classification/regression trainers and auto-tuning.
• Pipeline catalog — Pipeline catalog list/exists/drop lifecycle.
• Model catalog listing — Model metadata, training config, schema and loaded/stored/published state.
• Store models on disk — Enterprise-only persistent model storage semantics.
• Getting started ML pipeline — Current link-prediction pipeline example and split workflow.
• GDS server installation — Bundled plugin installation and procedure configuration.
• Neo4j GDS release notes — Current GDS 2026.07.0 release line.
• Neo4j current versions — Current Neo4j database release and LTS line.