Chapter 08 · Importing Data: LOAD CSV, Data Importer, Bulk Import, Transformation, and Validation
LOAD CSV with Parameters, Transactions, MERGE Discipline, and Incremental Import Patterns
Make LOAD CSV restartable with constrained identity and bounded transactions.
Learning outcomes
The cleaned AtlasMart files now need an online import that can stop, restart and rerun without duplicating business entities.
Use constrained identity with narrow MERGE patterns.
Batch writes with CALL subqueries IN TRANSACTIONS.
Explain partial-commit behavior and why restartability is mandatory.
Load endpoint nodes before dependent relationships.
Prove idempotency by rerunning the accepted source.
The mandatory lab continues Neo4j Community
2026.07.1, database neo4j, explicit
CYPHER 25 for version-sensitive examples,
authentication enabled, no mandatory APOC/GDS plugin, and
stable AtlasMart business identifiers from Chapters 01–07.
Neo4j 5.26.30 remains the LTS comparison line.
Local file:/// sources are a self-managed
server-filesystem feature and are not available on Aura.
This generation environment does not run Neo4j or Docker.
Commands were checked against current official documentation
but were not executed here. Expected outputs are fixture
invariants, not fabricated captures. Every write is scoped to
labTag='ch08', dedicated CSV files, or a
disposable database. Never use broad cleanup, store overwrite,
or offline import commands against unrelated or production
data.
Deterministic Chapter 08 source fixture
The source intentionally contains a duplicate business key, an empty optional field, a malformed numeric value, and orphan foreign keys. Those defects are evidence for preflight and reconciliation, not details to ignore.
customerId,name,email,tierC-8001,Ada Lovelace,ada@example.test,goldC-8002,Grace Hopper,grace@example.test,silverC-8002,Grace Hopper Duplicate,grace.dup@example.test,silverC-8003,Linus Torvalds,,bronzeproductId,name,categoryId,priceP-8001,Trail Camera,CAT-8001,129.90P-8002,Weather Case,CAT-8001,39.50P-8003,Sensor Hub,CAT-8002,not-a-numberP-8004,Field Battery,CAT-MISSING,49.00orderId,customerId,orderedAt,statusO-8001,C-8001,2026-09-01T10:00:00Z,PAIDO-8002,C-8002,2026-09-02T11:30:00Z,SHIPPEDO-8003,C-MISSING,2026-09-03T12:00:00Z,PAIDorderId,productId,quantity,unitPriceO-8001,P-8001,1,129.90O-8001,P-8002,2,39.50O-8002,P-8004,1,49.00O-9999,P-8001,1,129.90The corrected files remove/quarantine duplicate and orphan rows, repair the malformed price, and apply a documented empty-email policy. Keep the rejection ledger so accepted + rejected rows reconcile to the source.
1. Constrain identity before MERGE
CYPHER 25CREATE CONSTRAINT customer_id IF NOT EXISTS FOR (c:Customer) REQUIRE c.customerId IS UNIQUE;CREATE CONSTRAINT product_id IF NOT EXISTS FOR (p:Product) REQUIRE p.productId IS UNIQUE;CREATE CONSTRAINT order_id IF NOT EXISTS FOR (o:Order) REQUIRE o.orderId IS UNIQUE;CREATE CONSTRAINT category_id IF NOT EXISTS FOR (c:Category) REQUIRE c.categoryId IS UNIQUE;CYPHER 25LOAD CSV WITH HEADERS FROM 'file:///ch08/customers_clean.csv' AS rowCALL (row) { MERGE (c:Customer {customerId:row.customerId}) ON CREATE SET c.createdAt=datetime(),c.labTag='ch08' SET c.name=row.name,c.tier=row.tier, c.email=CASE WHEN trim(row.email)='' THEN null ELSE row.email END} IN TRANSACTIONS OF 500 ROWS;
MERGE is still pattern match-or-create, not a
generic SQL upsert. Keep the identity pattern narrow and
constrained; update descriptive fields separately.
2. Batching changes failure semantics
Current CALL { … } IN TRANSACTIONS groups 1000
input rows per inner transaction by default, or the explicit
OF n ROWS size. Earlier successful batches stay
committed if a later batch fails. Therefore the source and write
pattern must be safe to rerun.
The construct is allowed only in an implicit outer
transaction; Browser requires :auto. Do not wrap
it in a manually opened transaction.
3. Load nodes before relationships
CYPHER 25LOAD CSV WITH HEADERS FROM 'file:///ch08/products_clean.csv' AS rowCALL (row) { MERGE (p:Product {productId:row.productId}) ON CREATE SET p.labTag='ch08' SET p.name=row.name,p.price=toFloat(row.price) WITH p,row MATCH (c:Category {categoryId:row.categoryId}) MERGE (p)-[:IN_CATEGORY]->(c)} IN TRANSACTIONS OF 500 ROWS;CYPHER 25LOAD CSV WITH HEADERS FROM 'file:///ch08/order_items_clean.csv' AS rowCALL (row) { MATCH (o:Order {orderId:row.orderId}) MATCH (p:Product {productId:row.productId}) MERGE (o)-[r:CONTAINS]->(p) SET r.quantity=toInteger(row.quantity),r.unitPrice=toFloat(row.unitPrice)} IN TRANSACTIONS OF 500 ROWS;4. Deliberately wrong: retryable CREATE
LOAD CSV WITH HEADERS FROM 'file:///ch08/orders_clean.csv' AS rowCREATE (o:Order {orderId:row.orderId})CREATE (c:Customer {customerId:row.customerId})CREATE (c)-[:PLACED]->(o);A retry repeats creation. The repaired workflow establishes constrained endpoints once, then MERGEs the intended relationship.
5. Prove restartability
CYPHER 25MATCH (c:Customer) WHERE c.labTag='ch08' WITH count(c) AS customersMATCH (p:Product) WHERE p.labTag='ch08' WITH customers,count(p) AS productsMATCH (o:Order) WHERE o.labTag='ch08'RETURN customers,products,count(o) AS orders;Run the accepted import twice. Business-key counts and required edge cardinalities must remain unchanged, while descriptive properties may legitimately reflect changed source values.
Check your understanding
- Why constrain identity before MERGE?
- What if a later transaction batch fails?
- Why separate node and relationship phases?
- Why is CREATE unsafe in a retryable import?
- What proves idempotency?
Review the answers
1. To make duplicate business keys an enforced failure rather than a silent race.
2. Previously committed batches remain, so rerun behavior must converge.
3. It makes missing endpoint/reference failures observable and easier to reconcile.
4. A retry creates another node/edge instead of converging on the existing identity.
5. Repeated execution of the same accepted source leaves business identity and relationship invariants stable.
Summary and next step
The same mapping can also be expressed visually. Next, Data Importer is evaluated as a client—not as proof of a correct graph.
Authoritative references
- Current Neo4j versions — Release/LTS snapshot used for the chapter baseline.
- Cypher LOAD CSV — Current LOAD CSV parsing, typing and local/remote source semantics.
- CALL subqueries in transactions — Bounded inner transactions, default batch size and partial-commit behavior.
- Neo4j Admin import — Current full/incremental import, dry run, headers, memory and report behavior.
- Default file locations — Self-managed import-directory and store/log boundaries.
- Transactional subqueries — Default 1000-row batches, implicit transaction requirement and partial commits.
- MERGE — Match-or-create semantics and constraint-backed identity.