Treat medallion refinement and dimensional modeling as complementary coordinate systems.
Bronze/Silver/Gold Layering vs Dimensional Models: Complementary Concepts, Not Automatic Replacements
Build a hybrid analytical architecture whose storage, transaction, semantic, and serving responsibilities remain independently testable.
Learning outcomes
Distinguish pipeline-quality layers (bronze/silver/gold) from dimensional business modeling (facts, dimensions, grain, conformance, SCD).
Map AtlasMart bronze receipts, silver validated atomic state, and gold/warehouse serving without collapsing distinct responsibilities.
Explain where deduplication, late-data repair, conformance, dimensional history, and metric ownership belong.
Diagnose the failure mode “gold = one wide table” and repair it with explicit consumer contracts.
Design replay and promotion paths so medallion layers improve recoverability without creating a second semantic system.
Chapter 28 begins from the accepted state through Chapter 27: 10 current paid lines, 8 orders, 12 units, 820 USD gross revenue, 495 USD cost, and 325 USD gross profit, with source progress through sequence 208. The declared sales grain remains one paid order line. The ERP remains the operational system of record for order state; bronze stores received replay evidence; silver holds validated analytical atomic state; the dimensional warehouse serves governed BI; and the semantic layer still owns metric meaning. A lakehouse or open table format changes storage/transaction/interop mechanisms, not those business contracts.
Mandatory work is local, free, synthetic, and executed with Python. It writes JSONL object-store fixtures plus synthetic metadata examples explicitly labeled not real Iceberg/Delta/Hudi tables. All copies reconcile to 820 / 495 / 325 USD. The federation/materialization comparison is an explicit calculation: 120 dashboard queries/day × 320 MiB remote scan = 37.5 GiB/day federated remote reads; hourly materialization moves 28.8 GiB/day and scans 4.688 GiB/day locally. Modeled latency is 2,500 ms vs 180 ms and freshness is 2 minutes vs at most 60 minutes. These are teaching assumptions, not cloud measurements.
1. Problem frame
The data platform team proposes bronze → silver → gold and declares that facts, dimensions, SCDs, conformed dimensions, and metric contracts are now unnecessary. Finance still needs revenue by historical customer segment, operations needs fulfillment, and marketing needs shared customer/product definitions. The mistake is treating pipeline-stage labels as a semantic model.
Medallion describes quality/refinement progression; dimensional modeling describes analytical meaning and query structure.
2. Two coordinate systems
Bronze/silver/gold answers “how processed/trusted is this dataset?” Dimensional modeling answers “what business event does a row represent, what context describes it, how does history work, and how can measures aggregate safely?” They can intersect, but neither automatically determines the other.
| AtlasMart layer | Primary responsibility | Possible objects | Not automatically guaranteed |
|---|---|---|---|
| Bronze | Preserve received source evidence | Raw CDC/events/files | Conformance, dedupe, BI semantics |
| Silver | Validate, type, dedupe, integrate | Atomic order lines, clean customers | Presentation grain, dashboard metrics |
| Gold / serving | Consumer-ready products | Star schema, aggregates, features | One universal shape for all consumers |
3. AtlasMart mapping
bronze.erp_order_lines_received grain: one source change event identity: source sequence / event idsilver.sales_line_current grain: one accepted paid order line dedupe/delete/schema rules appliedwarehouse.fact_sales grain: one paid order line surrogate keys + historical dimension lookupsemantic.gross_revenue_usd.v1 SUM(line_amount_usd) over governed paid-line filter/time semanticsNotice the grain changes from source event in bronze to current accepted business line in silver. A Type 2 customer dimension can also exist in silver or the serving layer depending on architecture. What matters is that the as-of lookup and non-overlapping effective windows remain explicit and tested.
4. Replay and idempotency
Bronze earns its keep when silver/gold can be rebuilt deterministically from preserved source evidence. Replay must preserve event identity, ordering, deletes/corrections, schema versions, and watermarks. “Reprocess from bronze” is unsafe if bronze silently overwrites files or drops tombstones. Likewise, a gold rebuild must not create a second version of revenue logic: it should compile or consume the governed semantic definition and reconcile to atomic facts.
5. Controlled failure: gold as a permanent denormalized API
Wrong approach: create
gold_everything with customer, product, marketing,
order, shipment, and inventory columns at mixed grains because
“gold is business-ready.” Sales lines fan out against
campaigns/shipments; revenue no longer reconciles.
Repair: keep explicit process grains. Build a sales star, a fulfillment fact, inventory periodic snapshots, and governed aggregates/data products. Use conformed dimensions and drill-across rather than one mixed-grain table. If a wide table is justified for one consumer, document its grain, source dependencies, refresh, metric versions, and non-reusable assumptions.
6. Local lab: validate layer roles
layer_contract = { "bronze": {"grain":"one received source event", "mutable":False}, "silver": {"grain":"one accepted paid order line", "revenue":820}, "warehouse": {"grain":"one paid order line", "revenue":820}, "semantic": {"metric":"gross_revenue_usd.v1", "expected":820}}assert layer_contract["silver"]["revenue"] == layer_contract["warehouse"]["revenue"]assert layer_contract["semantic"]["expected"] == 820print("replay / serving controls aligned")This does not dictate whether silver/gold are implemented as managed warehouse tables, open tables, or views. Physical placement remains an engine/platform decision.
7. Production judgment
Medallion layers are valuable when they clarify replayability, quality gates, ownership, and progressive refinement. They become harmful when teams use the labels instead of declaring grain, history, keys, metric contracts, and serving SLOs. Preserve an explicit source → bronze → silver → dimensional/semantic consumer lineage and test each boundary independently.
Knowledge check
Does “gold” imply star schema?
Show answer
No. Gold generally signals business-ready/refined data in a medallion pattern. A star schema is one possible serving model and must still declare grain, facts, dimensions, and history.
Where should late data be handled?
Show answer
At the earliest governed stage that has enough identity/order/history context to do it correctly, with downstream rebuild/reconciliation. The layer name alone does not determine the rule.
Why preserve bronze?
Show answer
It can provide replay/audit evidence, but only if event identity, schema/version, deletes, ordering, and retention are sufficient for deterministic reconstruction.
Authoritative references
- Apache Iceberg — Table specificationOfficial specification for snapshot metadata, schema/partition evolution, manifests, and table-state commits.
- Apache Iceberg — Queries and metadata tablesOfficial documentation for snapshots, history, metadata logs, and time-travel query behavior.
- Delta Lake — Official documentationOfficial project documentation for Delta transaction-log tables, ACID semantics, schema enforcement/evolution, history, and time travel.
- Delta Lake — Table utility commandsOfficial details for table history, table metadata, restore, and retention-sensitive cleanup behavior.
- Apache Hudi — TimelineOfficial description of Hudi table actions/instants and the timeline as table-state metadata.
- Apache Hudi — Schema evolutionOfficial project documentation for supported schema-evolution behavior and version-specific caveats.
- Apache Hudi — Table and query typesOfficial description of Copy-on-Write and Merge-on-Read tradeoffs.
- Databricks — Medallion architectureA current product documentation example of bronze/silver/gold layering; used here as one implementation pattern, not as a prerequisite or universal definition.
- Kimball Group — Dimensional modeling techniquesReference for dimensional grain, fact/dimension modeling, conformance, and BI-serving semantics that remain separate from table-format mechanics.
8. Lab cleanup/reset
Remove the synthetic layer-contract files and rerun from the deterministic source fixture. No proprietary medallion product is required.