Journaling and Checkpoints: Durability Timeline, Crash Recovery, and Write Concern Interaction

Learning objectives

1. Four different moments can be hidden behind the word “written”

For AtlasMart, an application acknowledgement, a journal flush, a checkpoint, and replica-set majority acknowledgement answer different questions. WiredTiger uses a write-ahead log (the MongoDB journal) together with checkpoints. A checkpoint makes a consistent snapshot of data files. The journal records modifications that occurred after the last checkpoint so an unclean restart can replay durable records. A client’s write concern controls when the server replies; it does not redefine what a checkpoint is.

2. Start the replica-set lab and inspect the actual defaults instead of memorizing them

docker rm -f atlasmart-ch24-l2 2>/dev/null || truedocker volume rm atlasmart-ch24-l2-db 2>/dev/null || truedocker run -d --name atlasmart-ch24-l2 \  -p 127.0.0.1:27187:27017 \  -v atlasmart-ch24-l2-db:/data/db \  mongodb/mongodb-community-server:8.3.8-ubuntu2204-slim \  --bind_ip_all --port 27017 --replSet rs24until docker exec atlasmart-ch24-l2 mongosh --quiet --eval 'db.runCommand({ping:1}).ok' 2>/dev/null | grep -q 1; do sleep 1; donedocker exec atlasmart-ch24-l2 mongosh --quiet --eval '  try { rs.initiate({_id:"rs24",members:[{_id:0,host:"localhost:27017"}]}) } catch(e) { print(e.codeName) }'until docker exec atlasmart-ch24-l2 mongosh --quiet --eval 'db.hello().isWritablePrimary' 2>/dev/null | grep -q true; do sleep 1; donemongosh "mongodb://127.0.0.1:27187/admin?directConnection=true" --quiet --eval '  printjson(db.adminCommand({getParameter:1,featureCompatibilityVersion:1}));  printjson(rs.status().members.map(m=>({name:m.name,stateStr:m.stateStr})));'

const admin=db.getSiblingDB("admin");printjson(admin.runCommand({getParameter:1,syncdelay:1,journalCommitInterval:1}));const cfg=rs.conf();printjson({writeConcernMajorityJournalDefault:cfg.writeConcernMajorityJournalDefault});const s=admin.serverStatus();printjson({  log:{    bytes:s.wiredTiger.log["log bytes written"],    syncOps:s.wiredTiger.log["log sync operations"],    syncMicros:s.wiredTiger.log["log sync time duration (usecs)"]  },  checkpointMs:s.wiredTiger.transaction["transaction checkpoint most recent time (msecs)"]});

The documented defaults are a 60-second checkpoint/flush interval (syncdelay) and a maximum 100 ms journal commit interval, but the lesson reads the running server rather than assuming a future release keeps those values. A write that includes or implies j:true can trigger journal synchronization sooner.

3. Compare acknowledgement policies with measured distributions

python -m venv /tmp/atlasmart-ch24-journal-venv. /tmp/atlasmart-ch24-journal-venv/bin/activatepython -m pip install --disable-pip-version-check "pymongo==4.17.0"python - <<'PY'from statistics import medianfrom time import perf_counterfrom pymongo import MongoClient, WriteConcernclient=MongoClient("mongodb://127.0.0.1:27187/?directConnection=true")base=client.atlasmart.get_collection("durability_ch24_l2")base.drop()policies={    "w1-jfalse": WriteConcern(w=1,j=False),    "w1-jtrue": WriteConcern(w=1,j=True),    "majority": WriteConcern(w="majority",wtimeout=5000),}for label,wc in policies.items():    c=base.with_options(write_concern=wc); samples=[]    for i in range(40):        t=perf_counter(); c.insert_one({"_id":f"{label}-{i}","policy":label,"seq":i}); samples.append((perf_counter()-t)*1000)    q=sorted(samples)    print(label,{"p50_ms":median(q),"p95_ms":q[int(0.95*(len(q)-1))],"max_ms":max(q)})PY

These tiny local timings are not a production benchmark. They reveal only the acknowledgement cost on this machine/topology. Record disk type, container limits, cache warmth, concurrency, and journal settings before comparing environments.

4. Safe crash-recovery drill: prove the guaranteed case, do not fabricate the uncertain one

from pymongo import MongoClient, WriteConcernclient=MongoClient("mongodb://127.0.0.1:27187/?directConnection=true")base=client.atlasmart.get_collection("crash_ch24_l2")base.drop()base.with_options(write_concern=WriteConcern(w=1,j=True)).insert_one({"_id":"journaled","claim":"must be durable when acknowledgement returns"})base.with_options(write_concern=WriteConcern(w=1,j=False)).insert_one({"_id":"not-proven","claim":"acknowledgement alone did not require a journal sync"})print(list(base.find({}, {"_id":1,"claim":1})))

docker kill --signal=KILL atlasmart-ch24-l2docker start atlasmart-ch24-l2until mongosh "mongodb://127.0.0.1:27187/admin?directConnection=true" --quiet --eval 'db.runCommand({ping:1}).ok' 2>/dev/null | grep -q 1; do sleep 1; donemongosh "mongodb://127.0.0.1:27187/atlasmart?directConnection=true" --quiet --eval '  printjson(db.crash_ch24_l2.find().sort({_id:1}).toArray());  printjson(db.getSiblingDB("admin").serverStatus().wiredTiger.log);' 

5. Checkpoint pressure and journal pressure are related but distinguishable

A rising transaction checkpoint most recent time (msecs) under steady load can indicate storage saturation, but a single slow checkpoint is not enough evidence. Journal sync time reflects synchronous durability work; dirty cache growth and pages written from cache provide context for checkpoint/reconciliation pressure. Host-level device latency and queue depth are still required to prove a disk bottleneck.

6. Production judgment

Do not tune syncdelay or journalCommitInterval to solve generic slowness. MongoDB documentation explicitly advises leaving the checkpoint interval alone in almost every production situation. Choose write concern from the business invariant and failure model, then measure the resulting latency. A checkpoint is a local storage consistency mechanism, not a backup or replica. The next lesson compares compression strategies and shows why smaller storage can reduce I/O yet still increase CPU.

Authoritative references

WiredTiger metrics and internal field names are implementation- and version-sensitive. The lesson uses documented MongoDB interfaces for evidence and requires re-checking the current server manual before relying on exact metric names or defaults in a later release.

• WiredTiger Storage Engine
• Storage FAQ
• serverStatus
• Self-Managed Diagnostics FAQ
• Journaling
• Configure Journaling
• Write Concern
• Write Operation Performance
• Configuration File Options
• Server Parameters
• mongod Options
• db.collection.stats()
• $collStats
• dbStats
• db.createCollection() Storage Engine Options
• Create Indexes and Storage Engine Options
• Performance Tuning
• Production Notes
• Log Messages
• MongoDB 8.3 Release Notes
• mongosh Release Notes
• PyMongo Release Notes