Data Engineering · lesson 8 of 23
Beginner
8- 1 Data Engineering Fundamentals
- 2 File Formats: CSV, JSON, Parquet
- 3 Idempotency and Safe Retries
- 4 Batch Pipelines and Time Windows
- 5 Partitioning and Physical Layout
- 6 Batch vs. Streaming: The Real Trade-off
- 7 Data Lake vs. Data Warehouse vs. Lakehouse
- 8 Data Modeling Basics: Star Schemas and Denormalization
Intermediate
4Advanced
11- 13 Streaming: Exactly-Once Semantics
- 14 Schema Evolution and Data Contracts
- 15 Streaming Architecture: Lambda and Kappa
- 16 Data Lineage: Tracing a Column Back to Its Source
- 17 Data Governance: Catalogs, Classification, and Access Policy
- 18 Data Security and PII: Hashing, Masking, and Tokenization
- 19 Pipeline Observability: Tracing, Anomaly Detection, and Alerting
- 20 Cost Optimization: Measuring What Partition and Column Pruning Actually Save
- 21 CI/CD for Data: Gating Promotion with Real Tests
- 22 Disaster Recovery: RPO, RTO, and What a Backup Actually Loses
- 23 Data Platform Architecture: Wiring the Pieces Together
Also part of Data Engineering
9- + Your First Airflow DAG from Airflow
- + Kafka Topics, Partitions, and Offsets from Kafka
- + Spark Architecture: Driver, Executors, and Cluster Managers from Apache Spark
- + Models, ref, and the DAG from dbt
- + Delta Lake: The Transaction Log from Databricks
- + The Medallion Architecture from Databricks
- + File I/O, pathlib, and JSON in Python from Python
- + Error Handling in Python from Python
- + Terraform Fundamentals (IaC) from Terraform
Data Modeling Basics: Star Schemas and Denormalization
Load the same order data into a normalized schema and a star schema, then run the same query to see why analytics tables trade storage for join-free reads.
By SkillByExample · Published
Load the same three orders into a normalized schema and a star schema, then run the identical aggregate query against both.
Normalized: every fact stored exactly once
import duckdb
con = duckdb.connect()
con.execute("create table customers (customer_id int, name varchar, country varchar)")
con.execute("create table products (product_id int, name varchar, category varchar)")
con.execute("create table orders (order_id int, customer_id int, order_date date)")
con.execute("create table order_items (order_id int, product_id int, quantity int, unit_price decimal(10,2))")
con.execute("insert into customers values (1,'Ada','GB'), (2,'Grace','US')")
con.execute("insert into products values (10,'Widget','Hardware'), (11,'Gadget','Hardware'), (12,'Manual','Media')")
con.execute("insert into orders values (100,1,'2026-01-04'), (101,2,'2026-01-05')")
con.execute("""insert into order_items values
(100,10,2,9.99), (100,12,1,4.50),
(101,11,1,19.99)""")
print(con.execute("""
select p.category, sum(oi.quantity * oi.unit_price) as revenue
from order_items oi
join orders o on o.order_id = oi.order_id
join products p on p.product_id = oi.product_id
join customers c on c.customer_id = o.customer_id
group by p.category
order by p.category
""").df().to_string(index=False))
$ python data_modeling.py
== normalized: revenue by category needs a 4-table join ==
category revenue
Hardware 39.97
Media 4.50
Four tables, three joins, for one grouped sum. Every one of customers, products, and
orders has to be touched just to label order_items rows with a category and confirm they
belong to a real order.
Star schema: pre-joined once, queried many times
con.execute("""
create table fact_order_items as
select oi.order_id, o.order_date, c.customer_id, c.name as customer_name, c.country,
p.product_id, p.name as product_name, p.category,
oi.quantity, oi.unit_price, oi.quantity * oi.unit_price as revenue
from order_items oi
join orders o on o.order_id = oi.order_id
join products p on p.product_id = oi.product_id
join customers c on c.customer_id = o.customer_id
""")
print(con.execute("""
select category, sum(revenue) as revenue
from fact_order_items
group by category
order by category
""").df().to_string(index=False))
== star schema: revenue by category needs zero joins ==
category revenue
Hardware 39.97
Media 4.50
Same answer, no join. The join happened exactly once, when fact_order_items was built — every
downstream query pays for it zero more times, no matter how many analysts run this query today.
What that costs in storage
row_count_normalized = sum(con.execute(f"select count(*) from {t}").fetchone()[0]
for t in ["customers", "products", "orders", "order_items"])
row_count_star = con.execute("select count(*) from fact_order_items").fetchone()[0]
print(f"normalized: {row_count_normalized} rows spread across 4 tables, customer 'Ada' stored once")
print(f"star schema: {row_count_star} rows in 1 fact table, customer name 'Ada' repeated on every row of her orders")
normalized: 10 rows spread across 4 tables, customer 'Ada' stored once
star schema: 3 rows in 1 fact table, customer name 'Ada' repeated on every row of her orders
fact_order_items repeats Ada, GB, Widget, and Hardware on every line item they touch.
That repetition is the actual trade: storage and write complexity (update Ada’s name in a
million historical fact rows if she changes it) for read simplicity. Transactional systems
avoid it because they update constantly and want each fact to exist once; analytics tables
embrace it because they are written once (or once a day) and read thousands of times.
Practice
1. Add a fourth order for a new customer and confirm the star schema query still needs no join.
Insert into customers, orders, and order_items, then rebuild fact_order_items (a
create or replace table from the same query) — the read side of the star schema never
changes; only the build step re-runs.
2. What happens to `fact_order_items` if a customer's country is corrected after the fact table is built?
Nothing, until the fact table is rebuilt — the star schema holds a snapshot of the join at
build time, not a live reference. This is exactly the “slowly changing dimension” problem the
incremental-loads-and-cdc lesson’s SCD1/SCD2 section covers: do you overwrite the old
country everywhere (type 1) or keep both versions with an effective-date range (type 2)?
3. Build a second fact table, `fact_daily_revenue`, pre-aggregated by day and category.
create table fact_daily_revenue as
select order_date, category, sum(revenue) as revenue
from fact_order_items
group by order_date, category
A query against this table is even cheaper than querying fact_order_items — the aggregation
happened once at build time instead of once per query. This is the same “pay once, read many”
trade taken one level further.
4. Identify which columns in `fact_order_items` are facts (summed/averaged) and which are dimension attributes (grouped/filtered).
Facts: quantity, unit_price, revenue — numeric, additive. Dimension attributes:
customer_name, country, product_name, category, order_date — descriptive, used to
group or filter, never summed.
Next: data quality — checks that run on every load rather than a dashboard nobody reads.