Make authority, serving, and movement boundaries explicit before migration.
Design a Hybrid Lakehouse+Warehouse Architecture with Clear System of Record, Serving, and Data-Movement Boundaries
Build a hybrid analytical architecture whose storage, transaction, semantic, and serving responsibilities remain independently testable.
Learning outcomes
Design a hybrid AtlasMart architecture with explicit operational system of record, replay source, analytical integration state, warehouse serving state, and semantic ownership.
Document every data movement/federation boundary, copy, freshness contract, security path, lineage edge, and rollback path.
Use open-table storage where interoperability/replay benefits justify it and dimensional serving where governed BI behavior justifies it.
Prevent duplicate semantic ownership when the same data appears in lakehouse, warehouse, marts, and BI tools.
Create an acceptance checklist that remains valid if the physical engine or table format changes.
Chapter 28 begins from the accepted state through Chapter 27: 10 current paid lines, 8 orders, 12 units, 820 USD gross revenue, 495 USD cost, and 325 USD gross profit, with source progress through sequence 208. The declared sales grain remains one paid order line. The ERP remains the operational system of record for order state; bronze stores received replay evidence; silver holds validated analytical atomic state; the dimensional warehouse serves governed BI; and the semantic layer still owns metric meaning. A lakehouse or open table format changes storage/transaction/interop mechanisms, not those business contracts.
Mandatory work is local, free, synthetic, and executed with Python. It writes JSONL object-store fixtures plus synthetic metadata examples explicitly labeled not real Iceberg/Delta/Hudi tables. All copies reconcile to 820 / 495 / 325 USD. The federation/materialization comparison is an explicit calculation: 120 dashboard queries/day × 320 MiB remote scan = 37.5 GiB/day federated remote reads; hourly materialization moves 28.8 GiB/day and scans 4.688 GiB/day locally. Modeled latency is 2,500 ms vs 180 ms and freshness is 2 minutes vs at most 60 minutes. These are teaching assumptions, not cloud measurements.
1. Problem frame
AtlasMart now needs a final architecture decision. Data science wants open object-store tables, finance wants predictable dimensional queries, and operations wants near-current incident investigation. The design must say which system is authoritative for which state, when data is copied or federated, how semantic definitions propagate, and how to roll back if a format/engine migration fails.
A good hybrid architecture has explicit authority boundaries; it is not “use every platform at once.”
2. Hybrid target architecture
ERP / CRM / WMS operational systems | CDC/batch with source identity + contracts vBRONZE object storage -- replay evidence / received source history | validate, dedupe, delete/correction handling, schema contract vSILVER open-table atomic integration -- one accepted paid order line |\ | \---- federated audit / DS reads (snapshot/version rules) | +---- publish/reconcile ----> DIMENSIONAL WAREHOUSE SERVING fact_sales + conformed dimensions aggregates/materializations | v SEMANTIC CONTRACTS / CERTIFIED MARTS | v BI / finance / operations consumersSystem-of-record does not mean “the place analysts query most.” ERP owns operational order state. Bronze owns the received-event audit/replay record. Silver owns the validated analytical atomic interpretation at source sequence 208. The dimensional warehouse owns the certified serving representation. The semantic layer owns metric definition/version/filter/time semantics. Each authority is scoped.
3. Copy and ownership inventory
| Surface | Role | Authoritative for | Derived? | Primary consumers |
|---|---|---|---|---|
| ERP | Operational SoR | Order state | No | Operations/apps |
| Bronze | Replay evidence | Received source events | Yes | Data engineering/audit |
| Silver | Validated integration | Accepted atomic analytical state | Yes | Engineering/DS/federated audit |
| Warehouse | Dimensional serving | Certified query state | Yes | BI/marts |
| Semantic | Metric contract | Metric meaning | Yes | All analytical consumers |
4. Movement contracts
Every movement edge needs an identity, freshness target, idempotency rule, schema contract, quality gate, lineage record, security principal, and replay/rollback procedure. A copy is acceptable when it has a purpose. Uncontrolled copies—CSV exports, personal extracts, duplicate marts—are dangerous because they have no invalidation or privacy-deletion propagation path.
movement_contract: from: silver.sales_line_atomic to: warehouse.fact_sales grain: one paid order line watermark: source_sequence 208 metric_controls: {revenue_usd: 820, cost_usd: 495, gross_profit_usd: 325} publish: only after reconciliation + history/key tests idempotency: replace/merge one logical grain deterministically rollback: previous certified warehouse version security: service identity writes; BI roles cannot read bronze/silver raw PII5. Controlled failure: two “gold” systems both own revenue
Wrong approach: lakehouse gold computes
net_sales with one filter while the warehouse
semantic layer computes gross_revenue_usd.v1. Both
are labeled “Revenue.” Dashboards disagree and lineage cannot
identify which definition is certified.
Repair: assign semantic ownership once. If lakehouse gold materializes a revenue aggregate, it references the same versioned metric contract and is certified only after reconciliation. A storage engine may execute the formula in multiple places, but the definition, tests, owner, and version remain one governed contract.
6. Acceptance test: physical architecture can change while controls do not
expected = (10, 8, 12, 820, 495, 325)bronze_to_silver = (10, 8, 12, 820, 495, 325)warehouse_serving = (10, 8, 12, 820, 495, 325)assert bronze_to_silver == expectedassert warehouse_serving == expected# A storage/table-format migration is incomplete until:# - old/new query results reconcile,# - lineage/security/freshness remain valid,# - reader/writer compatibility is tested,# - rollback is rehearsed.7. Architecture decision matrix
| Workload | Chosen path | Why | Primary risk |
|---|---|---|---|
| Finance dashboard | Warehouse materialized serving | Repeated, strict latency, governed hourly certification | Staleness / duplicate copy |
| Audit source trace | Federated silver snapshot | Low frequency, need latest accepted atomic history | Remote dependency / consistency |
| Data science feature exploration | Silver/open table | Open engine access and atomic detail | Cost / uncontrolled derived copies |
| Executive metric | Semantic contract over certified serving | One definition across tools | Bypass/dashboard-local logic |
8. Production judgment and bridge to migration
A hybrid is justified when it gives materially different workloads appropriate access without splitting ownership of data meaning. Keep the architecture boringly explicit: who owns source truth, what is replayable, what is certified, which snapshot/version a federated query reads, where data is copied, how long copies may be stale, which credentials can bypass semantic controls, and which tests prove reconciliation. Chapter 29 will use these same contracts to modernize legacy warehouses without preserving technical debt or breaking metric behavior.
Knowledge check
Can both silver and the warehouse be “authoritative”?
Show answer
Yes, but only for different scoped responsibilities. Silver can be authoritative for accepted analytical atomic state while the warehouse is authoritative for the certified BI-serving representation. The scope must be explicit.
What should happen when a serving copy fails reconciliation?
Show answer
Do not certify/publish it. Preserve the prior certified version, diagnose the smallest cause, replay idempotently, and communicate consumer impact.
What makes a copy governed rather than accidental?
Show answer
A declared purpose, owner, lineage, freshness/retention/security contracts, reconciliation tests, invalidation/deletion propagation, and rollback/cleanup path.
Why does Chapter 29 care about this boundary map?
Show answer
Migration must preserve meaning and operational contracts, not just rows. The boundary map identifies which behaviors and authority relationships must survive platform changes.
Authoritative references
- Apache Iceberg — Table specificationOfficial specification for snapshot metadata, schema/partition evolution, manifests, and table-state commits.
- Apache Iceberg — Queries and metadata tablesOfficial documentation for snapshots, history, metadata logs, and time-travel query behavior.
- Delta Lake — Official documentationOfficial project documentation for Delta transaction-log tables, ACID semantics, schema enforcement/evolution, history, and time travel.
- Delta Lake — Table utility commandsOfficial details for table history, table metadata, restore, and retention-sensitive cleanup behavior.
- Apache Hudi — TimelineOfficial description of Hudi table actions/instants and the timeline as table-state metadata.
- Apache Hudi — Schema evolutionOfficial project documentation for supported schema-evolution behavior and version-specific caveats.
- Apache Hudi — Table and query typesOfficial description of Copy-on-Write and Merge-on-Read tradeoffs.
- Databricks — Medallion architectureA current product documentation example of bronze/silver/gold layering; used here as one implementation pattern, not as a prerequisite or universal definition.
- Kimball Group — Dimensional modeling techniquesReference for dimensional grain, fact/dimension modeling, conformance, and BI-serving semantics that remain separate from table-format mechanics.
9. Lab cleanup/reset
Delete atlasmart_ch28_lab/ to remove the synthetic
bronze/silver/warehouse/metadata/report files. The lab creates
no cloud resources and stores no credentials.