Chapter 08 · Importing Data: LOAD CSV, Data Importer, Bulk Import, Transformation, and Validation
Prepare Source Data: Stable Keys, Referential Integrity, Encoding, Nulls, Duplicates, and Relationship Files
Build a source-data trust boundary before any graph mutation.
Learning outcomes
AtlasMart is moving customer, catalog and order extracts from multiple systems. CSV syntax can be valid while the business data is unusable. This lesson builds the trust boundary before any graph write.
Define stable source keys and reject row-number/file-position identity.
Check duplicate keys and required references before relationship creation.
Distinguish empty string, missing value, valid business null and malformed data.
Prepare relationship endpoint keys that survive retries and file reordering.
Produce an auditable accepted/rejected preflight ledger.
The mandatory lab continues Neo4j Community
2026.07.1, database neo4j, explicit
CYPHER 25 for version-sensitive examples,
authentication enabled, no mandatory APOC/GDS plugin, and
stable AtlasMart business identifiers from Chapters 01–07.
Neo4j 5.26.30 remains the LTS comparison line.
Local file:/// sources are a self-managed
server-filesystem feature and are not available on Aura.
This generation environment does not run Neo4j or Docker.
Commands were checked against current official documentation
but were not executed here. Expected outputs are fixture
invariants, not fabricated captures. Every write is scoped to
labTag='ch08', dedicated CSV files, or a
disposable database. Never use broad cleanup, store overwrite,
or offline import commands against unrelated or production
data.
Deterministic Chapter 08 source fixture
The source intentionally contains a duplicate business key, an empty optional field, a malformed numeric value, and orphan foreign keys. Those defects are evidence for preflight and reconciliation, not details to ignore.
customerId,name,email,tierC-8001,Ada Lovelace,ada@example.test,goldC-8002,Grace Hopper,grace@example.test,silverC-8002,Grace Hopper Duplicate,grace.dup@example.test,silverC-8003,Linus Torvalds,,bronzeproductId,name,categoryId,priceP-8001,Trail Camera,CAT-8001,129.90P-8002,Weather Case,CAT-8001,39.50P-8003,Sensor Hub,CAT-8002,not-a-numberP-8004,Field Battery,CAT-MISSING,49.00orderId,customerId,orderedAt,statusO-8001,C-8001,2026-09-01T10:00:00Z,PAIDO-8002,C-8002,2026-09-02T11:30:00Z,SHIPPEDO-8003,C-MISSING,2026-09-03T12:00:00Z,PAIDorderId,productId,quantity,unitPriceO-8001,P-8001,1,129.90O-8001,P-8002,2,39.50O-8002,P-8004,1,49.00O-9999,P-8001,1,129.90The corrected files remove/quarantine duplicate and orphan rows, repair the malformed price, and apply a documented empty-email policy. Keep the rejection ledger so accepted + rejected rows reconcile to the source.
1. Stable keys are the import contract
A source key must survive file reordering, splitting, retries
and fresh extracts. AtlasMart therefore uses
customerId, productId,
orderId and categoryId. A line number
is evidence about one file version, not business identity.
| Defect | Risk | Preflight response |
|---|---|---|
| Duplicate C-8002 | Duplicate/ambiguous identity | Quarantine or resolve before load |
| P-8004 → CAT-MISSING | Unresolvable relationship endpoint | Add valid reference or quarantine row |
| P-8003 price text | Numeric conversion failure | Validate and correct before cast |
| C-8003 empty email | Ambiguous null/empty meaning | Apply explicit domain policy |
2. Relationship files are referential contracts
Online Cypher can match relationship endpoints by constrained
domain IDs. The admin importer uses explicit :ID,
:START_ID and :END_ID fields
(optionally grouped into ID spaces). Either way, every required
endpoint must resolve exactly once.
CYPHER 25LOAD CSV WITH HEADERS FROM 'file:///ch08/order_items.csv' AS rowWITH rowWHERE NOT EXISTS { MATCH (:Order {orderId: row.orderId}) }RETURN linenumber() AS sourceLine,row.orderId,row.productId;3. Encoding and type policy
Current LOAD CSV expects UTF-8. Header rows expose
each CSV record as a map, but imported fields are strings until
converted. Validate first, then use toInteger(),
toFloat(), date() or
datetime(). Do not use exceptions as the primary
data-quality detector.
CYPHER 25LOAD CSV WITH HEADERS FROM 'file:///ch08/products.csv' AS rowRETURN linenumber() AS line,row.productId,row.price, CASE WHEN row.price =~ '^[0-9]+(\.[0-9]+)?$' THEN toFloat(row.price) ELSE null END AS parsedPrice;4. Deliberately wrong: import first, clean later
LOAD CSV WITH HEADERS FROM 'file:///ch08/customers.csv' AS rowCREATE (:Customer {sourceRow:linenumber(),name:row.name});Reordering the same file changes identity and makes relationship retries unsafe. The repair is to establish stable keys, report defects and only then create graph state.
Hands-on preflight lab
CYPHER 25CREATE CONSTRAINT customer_id IF NOT EXISTS FOR (c:Customer) REQUIRE c.customerId IS UNIQUE;CREATE CONSTRAINT product_id IF NOT EXISTS FOR (p:Product) REQUIRE p.productId IS UNIQUE;CREATE CONSTRAINT order_id IF NOT EXISTS FOR (o:Order) REQUIRE o.orderId IS UNIQUE;CREATE CONSTRAINT category_id IF NOT EXISTS FOR (c:Category) REQUIRE c.categoryId IS UNIQUE;Place only the dedicated Chapter 08 files under the disposable Neo4j import directory. Run duplicate, orphan and malformed-value checks. Record source rows, accepted rows, rejected rows and rejection reasons before any mutation.
Check your understanding
- Why is line-number identity unsafe?
- What is an orphan row?
- Does successful CSV parsing prove property types?
- Why detect duplicates before relationships?
- What should happen to rejected rows?
Review the answers
1. File reordering or regeneration changes the number even though the business entity is the same.
2. A row whose required reference cannot resolve to the intended endpoint entity.
3. No. LOAD CSV fields are strings until the query validates/converts them.
4. Duplicate endpoints can multiply or ambiguously match relationship creation.
5. Retain them with reason and lineage in an auditable quarantine ledger.
Summary and next step
Source trust precedes graph writes. Next, the corrected files are loaded with bounded, restartable transactions.
Authoritative references
- Current Neo4j versions — Release/LTS snapshot used for the chapter baseline.
- Cypher LOAD CSV — Current LOAD CSV parsing, typing and local/remote source semantics.
- CALL subqueries in transactions — Bounded inner transactions, default batch size and partial-commit behavior.
- Neo4j Admin import — Current full/incremental import, dry run, headers, memory and report behavior.
- Default file locations — Self-managed import-directory and store/log boundaries.
- LOAD CSV format — UTF-8, header maps, type conversion and local/remote behavior.
- Admin import headers — Bulk importer ID spaces and relationship endpoint semantics.