Chapter 16 · Backup, Export, Recovery, Integrity, and Disaster Preparedness
integrity_check, quick_check, foreign_key_check, and Corruption Diagnosis
Separate structural database health from referential and business correctness, run SQLite integrity checks safely, interpret representative results, and build an operational health report.
Learning outcomes
Corruption is a structural problem in the database image, but many serious data problems are not corruption at all. A database can be structurally perfect and still contain an orphaned foreign key, an impossible business state, stale external data, or the wrong application schema version. Operational checks therefore need layers.
Distinguish structural corruption, foreign-key violations, and application/business-rule errors.
Use PRAGMA quick_check and integrity_check with current documented differences.
Use PRAGMA foreign_key_check because integrity_check deliberately does not perform that job.
Interpret representative success/failure results without attempting unsupported byte-level repair.
Add metadata and business-invariant checks to create a useful operational report.
Preserve suspect files and restore from known-good backups instead of experimenting on the only copy.
Four layers of “healthy”
| Layer | Question | Example check |
|---|---|---|
| File/database structure | Are pages, records, indexes, and core constraints internally coherent? | PRAGMA integrity_check |
| Fast routine structure | Can we run a cheaper broad structural check? | PRAGMA quick_check |
| Referential integrity | Do declared foreign keys currently have valid parents? | PRAGMA foreign_key_check |
| Application meaning | Does this data make sense for FieldNotes right now? | Domain queries, version checks, expected critical rows. |
quick_check versus integrity_check
quick_check performs most of the checks of integrity_check but skips UNIQUE verification and table/index content-consistency verification. Current SQLite documentation describes quick_check as O(N) while a full integrity_check can require O(N log N) work.
PRAGMA quick_check;-- Healthy expected output:-- okPRAGMA integrity_check;-- Healthy expected output:-- okBoth can accept limits/partial-table forms in current SQLite, but a partial check is intentionally narrower than a whole-database check. Do not report “database fully healthy” after checking one table.
Foreign keys are checked separately
SQLite explicitly documents that integrity_check does not detect foreign-key violations. This surprises people who read “integrity” as “all application integrity.” Use foreign_key_check separately.
PRAGMA foreign_keys = OFF;CREATE TABLE parent(id INTEGER PRIMARY KEY);CREATE TABLE child( id INTEGER PRIMARY KEY, parent_id INTEGER REFERENCES parent(id));INSERT INTO parent VALUES(1);INSERT INTO child VALUES(1,1);INSERT INTO child VALUES(2,999); -- deliberate orphan while FK enforcement is offPRAGMA foreign_keys = ON;PRAGMA integrity_check; -- can still report: okPRAGMA foreign_key_check; -- reports the orphanThe local acceptance run returned ok from both structural checks and one row from foreign_key_check: child table child, rowid 2, parent table parent, FK index 0.
PRAGMA foreign_keys=ON prevents new violations on that connection; it does not retroactively repair old rows. foreign_key_check is how you find existing violations.
Business-rule checks belong beside SQLite checks
Even perfect SQL constraints cannot encode every rule that depends on external systems, time, policy, or aggregate state. A health report should therefore include explicit queries for critical assumptions.
-- Empty text may be syntactically legal but operationally useless.SELECT note_idFROM maintenance_noteWHERE trim(note_text) = '';-- Verify that every active device is attached to an active/known site-- according to the actual schema policy used by your application.SELECT d.device_id, d.device_codeFROM device AS dLEFT JOIN site AS s ON s.site_id = d.site_idWHERE s.site_id IS NULL;-- File identity and migration level.PRAGMA application_id;PRAGMA user_version;Do not blindly copy these as universal rules. The point is to encode your invariants in a repeatable report with an owner and expected result.
A health-check script should fail clearly
from pathlib import Pathimport sqlite3, syspath = Path(sys.argv[1])con = sqlite3.connect(f"file:{path.as_posix()}?mode=ro", uri=True)try: quick = con.execute("PRAGMA quick_check").fetchall() integrity = con.execute("PRAGMA integrity_check").fetchall() fk = con.execute("PRAGMA foreign_key_check").fetchall() app_id = con.execute("PRAGMA application_id").fetchone()[0] user_version = con.execute("PRAGMA user_version").fetchone()[0] empty_notes = con.execute( "SELECT count(*) FROM maintenance_note WHERE trim(note_text)=''" ).fetchone()[0] report = { "quick": quick, "integrity": integrity, "foreign_key_violations": len(fk), "application_id": app_id, "user_version": user_version, "empty_notes": empty_notes, } print(report) healthy = ( quick == [("ok",)] and integrity == [("ok",)] and not fk and empty_notes == 0 ) raise SystemExit(0 if healthy else 2)finally: con.close()Opening read-only is useful for an observation job, but remember Chapter 9/12 caveats for read-only WAL databases and immutable mode. Your operational tool must use a connection mode compatible with how the deployment stores live state.
Representative failures: preserve evidence first
| Finding | Meaning | Safe next move |
|---|---|---|
integrity_check returns page/index errors | Possible structural corruption or damaged index/content relationship | Stop avoidable writes; preserve a copy/snapshot; investigate storage and restore from known-good backup. |
quick_check=ok, integrity_check fails | The skipped deeper checks found a problem | Treat full-check result as authoritative for the condition it reports. |
integrity_check=ok, FK rows returned | Structure is coherent but referential constraints are violated | Repair through controlled SQL/business logic after understanding origin. |
| All SQLite checks pass, business query fails | Not corruption; application invariant violated | Use application repair/migration logic and investigate write path. |
Wrong application_id/user_version | Potential wrong file or incompatible application schema | Do not run migrations blindly; identify the file/version first. |
What not to do during corruption diagnosis
Do not edit database pages with a hex editor, delete WAL/journal files by hand, run random PRAGMAs from forum snippets, or repeatedly write to the only damaged copy. Those actions can destroy recoverable evidence. Make a protected copy first, identify whether a good backup exists, and perform recovery experiments on disposable duplicates.
Corruption can come from failing storage, RAM, filesystem bugs, unsafe synchronization, rogue processes, or unsupported filesystem semantics. Restoring onto the same failing medium without investigating the underlying cause can simply corrupt the replacement.
Routine cadence versus incident checks
| Context | Reasonable pattern |
|---|---|
| Fast frequent monitoring | Use lightweight application checks and perhaps quick_check when cost is acceptable. |
| Backup verification | Open restored/backup copy; run full integrity_check + foreign_key_check + business invariants. |
| Release/migration rehearsal | Run structural, FK, schema/version, and migration-specific checks. |
| Suspected corruption | Preserve evidence and run deeper checks on a copy; avoid destructive “repair” experiments. |
| Disaster-recovery drill | Verify the entire restore path, not merely the source backup file. |
Checkpoint
Classify the failure
Name the layer before choosing a response.
integrity_checksays ok butforeign_key_checkemits two rows.quick_checksays ok andintegrity_checkreports an index inconsistency.- Every SQLite check passes but an application says a device has two mutually exclusive active assignments.
- A disk begins returning I/O errors.
- A restored backup passes all checks but has the wrong
user_versionfor the application binary.
Review the answers
These are respectively referential, deeper structural, business-rule, storage/infrastructure, and application compatibility problems. None should be treated as the same generic “database corruption” incident.
Bridge to salvage and restore drills
When a known-good backup exists, restore is usually safer than salvage. When it does not, SQLite’s .recover can attempt to extract surviving content from damaged pages—but recovery output must be treated as suspect data until rebuilt and validated.