Chapter 17 · Security, Privacy, Governance, and Auditability
Audit Trails, Provenance, and Immutable History
Build trustworthy audit trails and provenance records that answer who changed what, when, why, and from which source without confusing audit history with mutable business state.
Learning outcomes
An audit trail should answer who changed what, when, from where, and why. Provenance goes further by recording where data originated and how it was transformed. These are governance features, but they also support debugging, incident response, reconciliation, and trust.
Distinguish business history from audit history.
Design append-oriented audit records.
Capture actor, source, correlation, and before/after context appropriately.
Model provenance for imports and derived data.
Business history versus audit history
Business history:
WorkOrderStatusEvent
Audit history:
user 912 changed status from scheduled to closedvia API request abc123at 2026-08-10T12:00ZThey overlap but serve different purposes.
Audit-event shape
AuditEvent( audit_event_id, occurred_at, actor_type, actor_id, action_code, entity_type, entity_id, request_id, source_system, reason_code, metadata_json)Actor identity
An actor may be:
- human user;
- service account;
- scheduled job;
- migration;
- external integration.
“system” is often too vague.
Correlation IDs
One request may update many rows. A stable request/correlation ID lets investigators reconstruct the full operation across services and audit events.
Before/after values
Options:
- store full before/after snapshots;
- store changed fields only;
- store business event semantics;
- store hashes plus source history.
Full snapshots are easy to understand but may duplicate sensitive data heavily.
An audit log should not become an uncontrolled second copy of every sensitive field.
Immutable or append-oriented
Audit records should generally not be silently updated. Corrections can be represented by additional events rather than rewriting history.
Tamper evidence
High-assurance systems may use:
- restricted append-only storage;
- separate audit database;
- cryptographic hashes/chains;
- WORM/object retention controls;
- external log shipping.
Database triggers versus application audit
Triggers capture changes regardless of application path but may lack rich business context. Application events know intent but can be bypassed by direct SQL. Critical systems may combine layers.
Migration audit
Schema/data migrations should identify:
migration versionoperator/pipelinedeployment IDrows affectedstarted/completed timeProvenance for imported data
DataImport( import_id, source_name, source_file_hash, received_at, imported_by, schema_version)Imported rows can reference import_id.
Field-level provenance
For critical derived fields, record:
source systemsource record IDtransformation versioncalculated_atThis helps explain why a derived result has its current value.
Lineage versus provenance
Provenance often focuses on origin and transformation of a specific data item. Lineage often describes the broader flow:
Lesson 5 develops lineage further.
Who viewed data?
Some domains require access auditing, not only change auditing. Reading highly sensitive records may need an access log with actor, purpose, and time.
Reason codes
For administrative overrides:
reason_code = 'customer_correction'ticket_id = 'SUP-1842'can make privileged changes accountable.
Audit retention
Audit records may need longer retention than operational rows, but privacy laws/policies may still require deletion or minimization. “Audit” does not automatically mean “keep everything forever.”
WorkshopHub audit examples
| Action | Audit fields |
|---|---|
| Close WorkOrder | actor, old/new status, request, time |
| Change Asset owner | old/new customer, effective date, actor |
| Override inventory | quantity delta, reason, ticket, actor |
| Import Part catalog | source file/hash, import ID, pipeline version |
Practice: audit the override
Manual inventory correction
A supervisor changes inventory from 8 to 12 after a physical count. What should be captured?
Review answer
Capture the affected Part/location, previous and new quantity or delta, supervisor identity, timestamp, reason code, related count/ticket/reference, request/correlation ID, and source application. Avoid storing unrelated sensitive data in the audit record.
Summary and next lesson
Auditability and provenance make changes explainable and attributable. The final lesson expands governance to the entire lifecycle: retention, deletion, lineage, and data quality.
References
- NIST audit/logging guidance.
- OWASP Logging Cheat Sheet.
- Database and platform documentation for audit extensions/logging features.