Roll out MongoDB validators safely across old/new AtlasMart writers using strict/moderate levels, error/warn actions, observable debt, backfill, cutover, and rollback.
Validation Levels and Actions: strict/moderate, error/warn, and Safe Rollout of New Rules
Batch heterogeneous writes safely, interpret partial success, compare ordered and unordered execution, and use modern cross-namespace bulk APIs without assuming all-or-nothing behavior.
Learning outcomes
AtlasMart wants to require a two-letter shipping country code.
The schema rule is easy; the deployment is not. Two application
versions will overlap during a rolling release, and thousands of
historical orders lack shipping.countryCode.
validation level controls which inserts/updates
are subject to the rule, while
validation action controls whether a failing
write is rejected or allowed and logged. Those knobs are
compatibility tools, not substitutes for migration planning.
Distinguish strict, moderate, and
off validation levels from error,
warn, and current-version
errorAndLog actions.
Predict how a newly-invalid historical document behaves under strict versus moderate validation.
Design an expand-observe-backfill-enforce rollout that keeps old and new clients compatible during a controlled window.
Use server logs plus explicit invalid-document queries as rollout evidence rather than assuming warn mode means success.
Define a rollback posture before switching from warning to rejection.
This lesson pins MongoDB Community Server
8.3.8 with image
mongodb/mongodb-community-server:8.3.8-ubuntu2204-slim, a disposable standalone mongod bound only to
127.0.0.1:27048, mongosh 2.10.0, and
PyMongo 4.17.0 where driver behavior is shown.
The database is atlasmart; the collection is
orders_ch07_l2. Authentication and TLS are
disabled only because this is an isolated loopback learning
container. The lab reads FCV but never changes it. Default
read/write concern and primary read preference are used on
this standalone. Atlas, Search, KMS, Enterprise Advanced, and
paid services are not required. Product commands below were
reviewed against current official documentation; they were not
executed in this generation environment because
Docker/mongod/mongosh/PyMongo are unavailable here.
Result fragments labeled “expected” are contract-oriented shapes derived from the documented command semantics, not copied from a generation-time MongoDB run. Field order, error wording, log attributes, object identifiers, timing, and some diagnostic detail are version/environment dependent. Verify the invariant or count being taught rather than comparing output byte-for-byte.
1. Level answers “which writes are checked?”; action answers “what happens if they fail?”
Under strict (the default), validation applies to
inserts and updates, including updates to existing documents
that no longer satisfy the current rule. Under
moderate, inserts are still validated, and updates
to currently valid documents are validated, but updates to
existing invalid documents are not required to become valid.
This makes moderate useful during repair windows—but it can also
let legacy debt continue changing, so you must measure that debt
explicitly.
| Setting | Server behavior | Rollout implication |
|---|---|---|
validationLevel:"strict" |
Validate all inserts and all updates. | Best final posture when the fleet and historical data satisfy the rule; risky as a first rollout step. |
validationLevel:"moderate" |
Validate inserts and updates to existing valid documents; existing invalid documents can still be updated without passing the new rule. | Useful compatibility bridge while old invalid data is being repaired. |
validationLevel:"off" |
Do not apply validation to inserts or updates. | An emergency/maintenance posture, not a migration plan; it removes the integrity gate. |
validationAction:"error" |
Reject a noncompliant write. | Enforcement. Default action. |
validationAction:"warn" |
Allow the write but log the validation violation. | Observation/compatibility mode; the collection can accumulate invalid documents. |
validationAction:"errorAndLog" |
Reject and also log the violation. New in MongoDB 8.1. | Current optional posture when both enforcement and a log trail are required; verify exact server version before using it. |
Do not read “moderate” as “validate only the fields I changed.” MongoDB evaluates document validity. The distinction is whether the existing document was valid before the update.
2. The unsafe cutover: strict + error before old writers are gone
If AtlasMart deploys the new required field to the collection before the old order service is upgraded, the database will correctly reject that service's inserts. The validator is doing its job; the deployment contract is wrong. That is why schema enforcement must be coordinated with application compatibility, not merely reviewed as a database change.
A robust sequence is: make new readers understand both shapes; make new writers produce the expanded shape; add the candidate validator in observation mode; measure violations; backfill historical data; verify old writers are retired; then contract to strict rejection. Keep the previous validator/application behavior recoverable until the rollback window closes.
3. Run the staged rollout and observe both data and logs
docker rm -f atlasmart-mongo-ch07-l2 2>/dev/null || truedocker volume rm atlasmart-mongo-ch07-l2-data 2>/dev/null || truedocker run -d --name atlasmart-mongo-ch07-l2 \ -p 127.0.0.1:27048:27017 \ -v atlasmart-mongo-ch07-l2-data:/data/db \ mongodb/mongodb-community-server:8.3.8-ubuntu2204-slimmongosh "mongodb://127.0.0.1:27048/atlasmart?directConnection=true" --quiet --eval \'printjson({server:db.version(), hello:db.hello().isWritablePrimary}); printjson(db.getSiblingDB("admin").runCommand({getParameter:1,featureCompatibilityVersion:1}))'
const orders = db.getCollection("orders_ch07_l2");orders.drop();orders.insertMany([ { _id:"o-7201", customerId:"c-1", totalCents:NumberInt(4500), shipping:{ country:"United States", city:"Seattle" } }, { _id:"o-7202", customerId:"c-2", totalCents:NumberInt(9200), shipping:{ country:"United States", city:"Boston" } }, { _id:"o-7203", customerId:"c-3", totalCents:NumberInt(3100), shipping:{ country:"Canada", countryCode:"CA", city:"Toronto" } }]);const v2 = { bsonType:"object", required:["_id","customerId","totalCents","shipping"], properties:{ _id:{bsonType:"string"}, customerId:{bsonType:"string"}, totalCents:{bsonType:["int","long"], minimum:0}, shipping:{ bsonType:"object", required:["country","countryCode","city"], properties:{ country:{bsonType:"string"}, countryCode:{bsonType:"string", pattern:"^[A-Z]{2}$"}, city:{bsonType:"string"} } } }};print("candidate invalid:", orders.countDocuments({ $nor:[{ $jsonSchema:v2 }] }));// Deliberately unsafe rollout: old writer immediately breaks.printjson(db.runCommand({ collMod:"orders_ch07_l2", validator:{ $jsonSchema:v2 }, validationLevel:"strict", validationAction:"error" }));try { orders.insertOne({ _id:"o-old-client", customerId:"c-old", totalCents:NumberInt(1000), shipping:{country:"United States", city:"Austin"} });} catch (e) { print("old writer blocked:", e.code); }// Repair the rollout posture first: observe, do not block.printjson(db.runCommand({ collMod:"orders_ch07_l2", validator:{ $jsonSchema:v2 }, validationLevel:"moderate", validationAction:"warn" }));orders.insertOne({ _id:"o-old-during-rollout", customerId:"c-old2", totalCents:NumberInt(1200), shipping:{country:"United States", city:"Denver"} });orders.insertOne({ _id:"o-new-during-rollout", customerId:"c-new", totalCents:NumberInt(1300), shipping:{country:"United States", countryCode:"US", city:"Denver"} });// Deterministic backfill for the lab's known countries.printjson(orders.updateMany( { "shipping.countryCode": { $exists:false }, "shipping.country":"United States" }, { $set:{ "shipping.countryCode":"US" } }));print("remaining invalid:", orders.countDocuments({ $nor:[{ $jsonSchema:v2 }] }));// Enforce only after old writers are upgraded and debt reaches zero.printjson(db.runCommand({ collMod:"orders_ch07_l2", validator:{ $jsonSchema:v2 }, validationLevel:"strict", validationAction:"error" }));try { orders.insertOne({ _id:"o-old-after-cutover", customerId:"c-x", totalCents:NumberInt(900), shipping:{country:"United States", city:"Miami"} });} catch (e) { print("old writer rejected after cutover:", e.code); }orders.insertOne({ _id:"o-new-after-cutover", customerId:"c-y", totalCents:NumberInt(900), shipping:{country:"United States", countryCode:"US", city:"Miami"} });print("new writer accepted:", orders.countDocuments({_id:"o-new-after-cutover"}));
Expected result shape
candidate invalid: 2{ ok: 1 }old writer blocked: 121{ ok: 1 }{ acknowledged: true, matchedCount: 3, modifiedCount: 3, ... }remaining invalid: 0{ ok: 1 }old writer rejected after cutover: 121new writer accepted: 1
During warn mode the old-client write is accepted,
so an acknowledgement is not proof of schema compliance. The
corresponding mongod log contains a “Document would
fail validation” event. Log payload detail is version-sensitive
and can contain document/field values, so treat diagnostic logs
as governed data—not a free telemetry channel.
docker logs atlasmart-mongo-ch07-l2 2>&1 | grep -F "Document would fail validation" || true
docker logs atlasmart-mongo-ch07-l2 2>&1 | Select-String "Document would fail validation"
4. Rollback is a state, not a command
A safe rollback is not simply “set warn again.”
Decide what happens if the new application is rolled back after
it has written new-only fields. AtlasMart's expansion keeps the
old fields readable until the old application is retired. If a
cutover causes unexpected rejection, the database team can
restore the previous validator or move back to
moderate/warn, but only if the application still
understands the data written during the experiment.
Backfill rollback is a separate concern. For reversible transformations, retain the source fields or record enough provenance to reconstruct them. For destructive transformations, require a tested backup/restore path. Never claim that a validator can reverse a migration; it only gates future writes.
Track candidate-invalid count, validation warning/rejection rate, old-client version share, write-error rate, p95/p99 write latency, replication lag on replica sets, CPU/cache pressure during backfill, and the number of documents still requiring repair. No single universal threshold is appropriate; compare against the workload’s own SLOs and healthy baseline.
5. Verification, cleanup, and production judgment
Verification checklist
- The candidate query reports two historical orders without a country code.
- Direct strict/error enforcement blocks the synthetic old writer, demonstrating the deployment incompatibility.
- Moderate/warn permits the old writer while creating observable warning evidence.
- The deterministic backfill reaches zero candidate-invalid documents in the lab.
- Strict/error is enabled only after data and writer compatibility are verified.
- After cutover, the old writer is rejected and the new writer succeeds.
Use warn only when you have a consumer for the
warning signal and a plan to drive debt down; otherwise it can
turn schema validation into decorative logging. Use
moderate deliberately when existing invalid records
must remain mutable during migration. Final strict enforcement
is strongest when old clients are gone, all historical documents
pass, and bypass privileges are controlled. None of these
settings change transaction isolation, durability, replica
elections, shard targeting, or tenant authorization.
On a large replica set or sharded cluster, backfill writes
replicate and consume storage/cache/network budget; validator
changes affect every writer quickly. Test under production-like
load, throttle from measured pressure, and keep rollback
reversible. The next lesson adds a first-class
schemaVersion field and compares lazy migration
with eager backfills.
docker rm -f atlasmart-mongo-ch07-l2docker volume rm atlasmart-mongo-ch07-l2-data
Check your understanding
-
Why can an acknowledgement in
validationAction:"warn"not prove that a document satisfies the schema? -
Under
moderate, what happens when an already-invalid historical document is updated? - Why can direct strict/error rollout break an otherwise healthy old application?
- What must reach zero (or an explicitly accepted exception set) before final strict enforcement?
- Why is switching the validator back not, by itself, a data-migration rollback?
Review the answers
Warn mode intentionally allows a violating write and records a log warning. The write can be acknowledged while the resulting document is noncompliant.
Existing invalid documents are not required to pass the rule on update under moderate validation. That compatibility behavior is useful but must be measured so debt does not persist silently.
The old application may omit a field the new validator now requires. The database correctly rejects its write even though the application has not yet been upgraded.
Candidate-invalid historical documents should be repaired or explicitly exempted, and old writer versions that cannot satisfy the contract should be retired. The exact acceptance gate belongs in the rollout plan.
Changing the validator only changes future write checks. It does not restore fields already transformed or deleted, so data rollback requires retained source data, reversible transforms, or a tested restore path.
Authoritative references
- MongoDB 8.3 release notes — Current server-series and patch-release status; re-check before reproducing the lab.
- MongoDB versioning — Explains major/minor release series and why the latest stable patch should be used.
- mongosh release notes — Current mongosh release and version-sensitive shell behavior.
- PyMongo release notes — Current Python driver line used when application-side migration behavior is demonstrated.
- Specify validation level — Defines strict and moderate behavior for existing valid/invalid documents.
- Handle invalid documents — Defines error, warn, and current errorAndLog actions and shows warning log evidence.
-
Modify schema validation
— Official
collModprocess and effect of tightening rules on previously valid documents. - collMod command — Command-level validator, validationLevel, and validationAction options.
-
Query valid or invalid documents
— Supports preflight debt measurement with
$jsonSchema. - Log messages — Context for treating validation warnings as operational log data.