Execute an end-to-end AtlasMart impact analysis for a breaking source-column rename, inventory affected models, metrics, reports, and consumers, and produce a migration/deprecation decision before the source contract changes.
Build an Impact Analysis for a Source Column Change and Identify Every Model, Metric, Report, and Consumer at Risk
Build an identity and access model for AtlasMart that separates humans from services, eliminates shared credentials, enforces environment boundaries, and proves least privilege with allow/deny evidence.
Learning outcomes
Execute a source-change impact analysis before pipeline development.
Name affected models, metrics, marts, reports, consumers, owners, and migration actions.
Use deprecation/compatibility windows instead of destructive renames.
Separate “registered blast radius” from unknown shadow dependencies.
Produce evidence that supports approve/block/rollback decisions.
Chapter 23 begins from the governed and secured AtlasMart state produced by earlier chapters: 10 current paid lines, 8 orders, 12 units, 820 USD gross revenue, 495 USD cost, and 325 USD gross profit. The fact grain remains one current paid order line; the Chapter 20 metric contracts remain authoritative; Chapter 21 certified marts remain dependent on those contracts; and Chapter 22 security/privacy controls remain in force. Metadata and lineage describe, govern, and make change safer. They do not silently create a second business definition.
Runtime: Python 3.13.5 + SQLite 3.46.1. Mode: local, in-memory SQLite plus deterministic Python, no cloud account and no paid feature. Time: UTC. Currency: USD. Security: synthetic identities and non-secret metadata only; metadata can itself reveal sensitive topology, so production catalog access still requires policy. History: catalog entries and deprecation notices are versioned/effective-dated rather than overwritten without evidence. Non-guarantee: the lab proves graph/catalog logic for this fixture; it does not prove any vendor's automatic lineage completeness.
1. The realistic problem: accept the ERP rename on October 1—or block it?
The ERP producer proposes replacing
line_amount_usd with
gross_line_amount_usd on 2026-10-01 and removing
the old field after 2026-10-15. The values retain gross USD
semantics. This sounds like a cosmetic rename, but the old field
name is embedded across ingestion mappings, warehouse models,
semantic metrics, marts, reports, and consumer workflows. The
decision is whether AtlasMart is ready to accept the change
without breaking certified data products.
2. Register the change before changing code
UPDATE schema_fieldSET status = 'deprecated'WHERE schema_id = 'erp.order_lines.v3' AND field_name = 'line_amount_usd';INSERT INTO schema_field(...) VALUES ( 'erp.order_lines.v4', 'gross_line_amount_usd', 'DECIMAL(12,2)', 0, 'Replacement for line_amount_usd; same gross USD semantics', 'active', '2026-10-01T00:00:00Z');INSERT INTO deprecation_notice(...) VALUES ( 'DEP-ERP-2026-09-21', 'src.erp.order_lines.line_amount_usd', '2026-09-21T00:00:00Z', '2026-10-01T00:00:00Z', '2026-10-15T00:00:00Z', 'src.erp.order_lines.gross_line_amount_usd', 'Source renamed field; downstream mappings must migrate.');The exact dates belong to this fixture, not a universal policy. The mechanism is the important part: announce, model replacement, inventory impact, migrate, validate in parallel, then retire only when the consumer/compatibility evidence allows it.
3. Full executed blast radius
15 impacted assetssource_column 1raw_column 1job 1model_column 1measure 1metric 2mart 2report 3consumer 3Named consumers/use cases:exec_team -> Weekly executive revenue reviewfinance_team -> Ad-hoc finance reconciliationfinance_team -> Daily margin closemarketing_ops -> Campaign revenue reportingThe graph names both code/data assets and accountable consumers. That allows a migration plan to assign work and communication rather than merely producing a list of SQL objects.
4. Why both revenue and profit metrics are affected
gross_revenue_usd.v1 depends on the governed
paid-line amount. gross_profit_usd.v1 also depends
on that amount as the revenue component of revenue minus cost.
The source cost column itself is not affected. Fine-grained
lineage avoids incorrectly marking
cost_amount_usd as changed while still discovering
that the gross-profit metric is downstream of the renamed
revenue input.
5. Controlled failure: rename now, discover consumers later
| Bad sequence | Observable consequence |
|---|---|
| Producer deletes old field first | Raw ingestion/mapping fails at the boundary |
| Warehouse hotfixes mapping only | Metrics may work, but reports/consumers were never notified/tested |
| Lineage stops at warehouse | 11 of 15 registered impacted assets are missed |
| No compatibility window | Rollback requires emergency producer change |
| Metric semantics changed during rename | A schema migration becomes an undocumented business restatement |
The repair is a versioned migration that preserves semantics and runs golden/reconciliation tests against both old and replacement fields while both are available.
6. Migration acceptance plan
| Step | Owner | Evidence to proceed |
|---|---|---|
| Register v4 source contract + deprecation | commerce-platform / orders-steward | Replacement field/type/semantics reviewed |
| Update raw mapping + integration job | data-platform | Both old/new fixture inputs produce same accepted facts |
| Update model/semantic dependencies | warehouse + finance-analytics | 820 revenue / 495 cost / 325 profit unchanged |
| Refresh dependent marts | finance + marketing analytics | Certified mart conformance tests pass |
| Regression test reports | BI owners | Golden report outputs match |
| Notify named consumers | BI/product owners | Consumer inventory acknowledgements/migration notes |
| Retire old field | producer + governance | No registered active dependency remains; rollback window decision recorded |
7. Approval state: BLOCK until migration evidence exists
The initial readiness decision is BLOCK, not because the new field is bad, but because 15 registered downstream assets are affected. After dual-field compatibility, code changes, semantic/reconciliation tests, report regressions, consumer migration, and updated lineage all pass, the change can become APPROVE FOR CUTOVER. A governance system should record that transition and its evidence instead of replacing the earlier blocked state.
8. What the impact result does not prove
It does not prove there are no unmanaged spreadsheet exports, hard-coded queries, local notebook copies, or external integrations. It proves the blast radius of the registered metadata graph. Unknown consumers are reduced by query logs, access telemetry, catalog adoption, ownership requirements, and deprecation monitoring—not by pretending the catalog is omniscient.
9. Correctness, rollback, and bridge to Chapter 24
Correctness: this rename must preserve 10 current paid lines, 8 orders, 12 units, 820 USD gross revenue, 495 USD cost, and 325 USD gross profit. History: retain old/new contract versions and effective dates. Idempotency: rerunning impact analysis or catalog updates must not duplicate notices/assets. Security: impact output may reveal sensitive consumers/resources; restrict access appropriately. Performance/cost: graph traversal here is tiny; production catalog scale needs measurement. Rollback: compatibility with the old field is the rollback lever until retirement. Next: Chapter 24 turns these contracts into systematic schema, quality, SCD/CDC, reconciliation, and regression tests.
10. Verification checklist
- Run the fixture and confirm controls remain 10 current paid lines, 8 orders, 12 units, 820 USD gross revenue, 495 USD cost, and 325 USD gross profit.
- Confirm the owner gate fails before the sandbox repair and passes after it.
- Confirm the source rename returns exactly 15 registered impacted assets.
- Confirm warehouse-only lineage sees only 4 of 15.
- Confirm the consumer inventory names executive, finance, and marketing uses.
-
Confirm the post-registration metadata fingerprint is
2819b6f63a8f9801d9092556d5e90b10039c0faa049ec23df48c0f734884db83. - Reset and rerun; results must be deterministic.
Knowledge check
Acceptance questions
- Why can a rename be breaking even when values/types are unchanged?
-
Why should
gross_profit_usd.v1appear in impact? - What evidence is required before retiring the old field?
- Why is “15 assets” a lower bound rather than an absolute truth?
Review the answers
1. Consumers bind to names/contracts; compatibility is syntactic and semantic, not only value equality.
2. Its formula includes the revenue measure derived from the renamed source field.
3. Updated mappings/models/lineage, reconciliation/golden tests, report regressions, consumer migration, and rollback decision.
4. The graph only knows registered/emitted dependencies; shadow consumers may exist.
Authoritative references
- OpenLineage — Lineage Dataset FacetCurrent specification for expressing dataset/job/field dependencies; useful for portable lineage concepts without making OpenLineage a prerequisite.
- OpenLineage — Column Level Lineage Dataset FacetShows fine-grained field dependencies and distinguishes identity, transformation, aggregation, join, filter, sort, window, and conditional influence.
- W3C — Data Catalog Vocabulary (DCAT) Version 3A standard vocabulary for describing cataloged datasets/data services and their metadata; referenced as an interoperability model, not as a required implementation.
- W3C — PROV-OStable provenance vocabulary for entities, activities, and agents; useful for reasoning about lineage/provenance boundaries.
- SQLite — WITH / recursive common-table expressionsThe local lab uses a recursive CTE to traverse downstream lineage deterministically.
- Python — hashlibUsed to fingerprint the deterministic metadata state after catalog repair and change registration.
11. Lab cleanup/reset
Close the process to discard the in-memory catalog. Rerun the deterministic fixture to reproduce the failure, repair, deprecation registration, impact traversal, and metadata fingerprint without touching any production system.