dbt · lesson 2 of 11
Also part of: Data Engineering
Models, ref, and the DAG
How ref builds a dependency graph, why layering into staging and marts pays off, and the selector syntax that runs exactly the models you changed.
By SkillByExample · Published
ref is the whole idea. Every time one model reads another through ref, dbt records an
edge — and from those edges it derives build order, parallelism, and what to rebuild when
something changes.
Two models, one edge
-- models/staging/stg_customers.sql
select
id as customer_id,
first_name || ' ' || last_name as full_name,
upper(country) as country_code
from {{ ref('raw_customers') }}
-- models/marts/customer_orders.sql
select
c.customer_id,
c.full_name,
c.country_code,
count(o.order_id) as order_count,
coalesce(sum(o.amount), 0) as lifetime_value,
min(o.ordered_at) as first_order_at
from {{ ref('stg_customers') }} as c
left join {{ ref('stg_orders') }} as o
on o.customer_id = c.customer_id
group by 1, 2, 3
dbt run
09:02:11 Running with dbt=1.9.1
09:02:11 Found 3 models, 2 seeds, 431 macros
09:02:11
09:02:11 Concurrency: 4 threads (target='dev')
09:02:11
09:02:11 1 of 3 START sql view model main.stg_customers ................. [RUN]
09:02:11 2 of 3 START sql view model main.stg_orders .................... [RUN]
09:02:11 1 of 3 OK created sql view model main.stg_customers ............ [OK in 0.04s]
09:02:11 2 of 3 OK created sql view model main.stg_orders ............... [OK in 0.04s]
09:02:11 3 of 3 START sql table model main.customer_orders .............. [RUN]
09:02:11 3 of 3 OK created sql table model main.customer_orders ......... [OK in 0.07s]
09:02:11
09:02:11 Finished running 2 view models, 1 table model in 0 hours 0 minutes and 0.24 seconds (0.24s).
09:02:11
09:02:11 Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3
Read the interleaving. Models 1 and 2 both start before either finishes — they are
independent, so dbt ran them on separate threads. Model 3 waited for both. Nothing in the
project file declares that order; it came from two ref calls.
duckdb bookshop.duckdb -c "select * from main.customer_orders order by customer_id"
┌─────────────┬──────────────────┬──────────────┬─────────────┬────────────────┬────────────────┐
│ customer_id │ full_name │ country_code │ order_count │ lifetime_value │ first_order_at │
├─────────────┼──────────────────┼──────────────┼─────────────┼────────────────┼────────────────┤
│ 1 │ Ada Lovelace │ GB │ 2 │ 65.5 │ 2026-01-04 │
│ 2 │ Grace Hopper │ US │ 1 │ 12.0 │ 2026-01-05 │
│ 3 │ Alan Turing │ GB │ 1 │ 8.75 │ 2026-01-09 │
│ 4 │ Katherine Johnson│ US │ 0 │ 0.0 │ │
│ 5 │ Edsger Dijkstra │ NL │ 0 │ 0.0 │ │
└─────────────┴──────────────────┴──────────────┴─────────────┴────────────────┴────────────────┘
Customer 9’s order has vanished — the left join starts from customers, and there is no
customer 9. That is a silent data loss bug, and the point of the tests in lesson 4.
What ref protects you from
from bookshop.main.stg_orders -- hard-coded
from {{ ref('stg_orders') }} -- resolved
The hard-coded version compiles to the same string in dev and in production, so a developer
run reads production tables. ref resolves against the current target’s schema:
dbt run --target prod --select customer_orders
09:07:52 1 of 1 START sql table model analytics.customer_orders ......... [RUN]
09:07:52 1 of 1 OK created sql table model analytics.customer_orders .... [OK in 0.11s]
main.customer_orders in dev, analytics.customer_orders in prod, from one unchanged file.
The layering that keeps this maintainable
models/
├── staging/ one model per source table: rename, cast, filter. Views.
│ ├── stg_customers.sql
│ └── stg_orders.sql
├── intermediate/ joins and reshaping nobody queries directly. Often ephemeral.
│ └── int_orders_enriched.sql
└── marts/ what analysts and dashboards read. Tables.
├── customer_orders.sql
└── daily_revenue.sql
The rule that makes it work: only staging models may read a source. Everything else reads other models. When a source column is renamed, exactly one file changes.
-- models/intermediate/int_orders_enriched.sql
{{ config(materialized='ephemeral') }}
select
o.order_id,
o.ordered_at,
o.amount,
c.country_code,
o.amount >= 25 as is_large_order
from {{ ref('stg_orders') }} as o
inner join {{ ref('stg_customers') }} as c
on c.customer_id = o.customer_id
-- models/marts/daily_revenue.sql
select
ordered_at,
country_code,
count(*) as orders,
sum(amount) as revenue,
sum(case when is_large_order then 1 else 0 end) as large_orders
from {{ ref('int_orders_enriched') }}
group by 1, 2
dbt run
09:14:38 Found 5 models, 2 seeds, 431 macros
09:14:38
09:14:38 1 of 4 START sql view model main.stg_customers ................. [RUN]
09:14:38 2 of 4 START sql view model main.stg_orders .................... [RUN]
09:14:38 1 of 4 OK created sql view model main.stg_customers ............ [OK in 0.04s]
09:14:38 2 of 4 OK created sql view model main.stg_orders ............... [OK in 0.04s]
09:14:38 3 of 4 START sql table model main.customer_orders .............. [RUN]
09:14:38 4 of 4 START sql table model main.daily_revenue ................ [RUN]
09:14:38 3 of 4 OK created sql table model main.customer_orders ......... [OK in 0.08s]
09:14:38 4 of 4 OK created sql table model main.daily_revenue ........... [OK in 0.09s]
09:14:38
09:14:38 Done. PASS=4 WARN=0 ERROR=0 SKIP=0 TOTAL=4
Five models found, four built. The ephemeral one was never created as a relation — dbt
inlined it as a CTE inside daily_revenue:
head -12 target/compiled/bookshop/models/marts/daily_revenue.sql
with __dbt__cte__int_orders_enriched as (
select
o.order_id,
o.ordered_at,
o.amount,
c.country_code,
o.amount >= 25 as is_large_order
from "bookshop"."main"."stg_orders" as o
inner join "bookshop"."main"."stg_customers" as c
on c.customer_id = o.customer_id
)
select
Ephemeral models keep the warehouse clean of intermediate objects nobody queries. The cost
is debuggability: there is no relation to select * from when the numbers look wrong.
Selecting what to build
dbt ls --select customer_orders+
bookshop.marts.customer_orders
dbt run --select stg_orders+
09:19:05 Found 5 models, 2 seeds, 431 macros
09:19:05
09:19:05 1 of 3 START sql view model main.stg_orders .................... [RUN]
09:19:05 1 of 3 OK created sql view model main.stg_orders ............... [OK in 0.04s]
09:19:05 2 of 3 START sql table model main.customer_orders .............. [RUN]
09:19:05 3 of 3 START sql table model main.daily_revenue ................ [RUN]
09:19:05 2 of 3 OK created sql table model main.customer_orders ......... [OK in 0.08s]
09:19:05 3 of 3 OK created sql table model main.daily_revenue ........... [OK in 0.08s]
09:19:05
09:19:05 Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3
stg_orders plus everything that depends on it, transitively — stg_customers was
correctly left alone.
| Selector | Builds |
|---|---|
my_model | just that model |
my_model+ | it and everything downstream |
+my_model | it and everything upstream |
+my_model+ | the full lineage through it |
2+my_model | two generations upstream, no further |
staging | everything in models/staging/ |
tag:nightly | models tagged nightly |
source:raw+ | every model built from the raw source |
--exclude my_model | everything except it |
The combination that matters in daily work is “what I changed and everything it feeds”:
dbt run --select stg_orders+ --exclude tag:expensive
09:21:44 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
Tag models in dbt_project.yml or per model:
{{ config(tags=['nightly', 'expensive']) }}
Cycles fail at parse time
Make stg_orders read customer_orders, which already reads stg_orders:
09:24:16 Running with dbt=1.9.1
09:24:16 Encountered an error:
Compilation Error
Found a cycle: model.bookshop.stg_orders --> model.bookshop.customer_orders --> model.bookshop.stg_orders
No model ran. dbt builds the graph before executing anything, so a cycle costs you a second
rather than a half-built warehouse — the same reason a typo’d ref fails immediately:
Compilation Error in model customer_orders (models/marts/customer_orders.sql)
Model 'model.bookshop.customer_orders' (models/marts/customer_orders.sql)
depends on a node named 'stg_order' which was not found
Documenting the graph
# models/marts/_marts.yml
version: 2
models:
- name: customer_orders
description: One row per customer with lifetime order metrics.
columns:
- name: customer_id
description: Primary key, from the source customer table.
- name: lifetime_value
description: Sum of all non-pending order amounts, zero for customers with none.
dbt docs generate
09:28:03 Found 5 models, 2 seeds, 4 data tests, 431 macros
09:28:03 Building catalog
09:28:04 Catalog written to /home/you/bookshop/target/catalog.json
Those descriptions and the DAG become a browsable site with dbt docs serve, covered in
lesson 9.
Practice
1. Add a country_revenue mart and confirm dbt builds it after its parents.
-- models/marts/country_revenue.sql
select
country_code,
sum(revenue) as revenue,
sum(orders) as orders
from {{ ref('daily_revenue') }}
group by 1
09:32:50 3 of 5 START sql table model main.daily_revenue ................ [RUN]
09:32:50 3 of 5 OK created sql table model main.daily_revenue ........... [OK in 0.08s]
09:32:50 4 of 5 START sql table model main.country_revenue ............. [RUN]
09:32:50 4 of 5 OK created sql table model main.country_revenue ........ [OK in 0.06s]
09:32:50 Done. PASS=5 WARN=0 ERROR=0 SKIP=0 TOTAL=5
country_revenue started only after daily_revenue reported OK — one ref was enough to
order them.
2. Use dbt ls to list everything downstream of stg_customers.
dbt ls --select stg_customers+ --resource-type model
bookshop.staging.stg_customers
bookshop.intermediate.int_orders_enriched
bookshop.marts.customer_orders
bookshop.marts.daily_revenue
bookshop.marts.country_revenue
dbt ls answers “what breaks if I change this” without building anything. It is the fastest
impact analysis you have before touching a shared staging model.
3. Replace a ref with a hard-coded table name. Does it still work?
09:36:12 1 of 4 START sql table model main.customer_orders .............. [RUN]
09:36:12 1 of 4 OK created sql table model main.customer_orders ......... [OK in 0.07s]
It works — and it built first, before stg_orders existed in a clean warehouse. Without
the ref edge, dbt has no reason to wait, so this passes locally and fails on a fresh build
in CI. Hard-coded names are not a style problem, they are an ordering bug.
4. Make int_orders_enriched a view instead of ephemeral and compare the compiled SQL.
{{ config(materialized='view') }}
09:39:41 3 of 5 START sql view model main.int_orders_enriched ........... [RUN]
09:39:41 3 of 5 OK created sql view model main.int_orders_enriched ...... [OK in 0.04s]
select
ordered_at,
country_code,
...
from "bookshop"."main"."int_orders_enriched"
The CTE is gone and daily_revenue now selects from a real relation. Views are the better
default while developing — you can query the intermediate result — and ephemeral is worth
switching to once the model is stable and nobody needs to inspect it.
Next: sources — declaring where raw data comes from, and catching it when it stops arriving.