dbt — SQL with Software Engineering Habits

Models, refs, tests, snapshots, contracts. Why dbt won the transform layer.

0/2 done

Theory

dbt's bet: transformation is software

dbt does one thing: it compiles your .sql files into warehouse SQL and runs them in dependency order. The leverage comes from the conventions:

  • Models — one SQL SELECT per table/view. ref('stg_orders') wires the DAG so dbt builds them in the right order.
  • Testsunique, not_null, accepted_values, relationships, plus arbitrary SQL tests. They fail the build, not just a dashboard.
  • Snapshots — automatic SCD-2 for source tables that lack history.
  • Contracts (dbt 1.5+) — declare a model's column names, types and constraints; consumers depend on the contract, not the implementation.

Combined, dbt brings code review, CI, version control and testing — the basics of software engineering — to a layer that used to live in stored procedures and tribal knowledge.

Analogy

Before dbt, warehouse SQL lived like recipes shouted across a kitchen — undocumented, untested, surviving only in the head of the one chef who knew the order to cook things in. dbt turns those recipes into a published cookbook under version control: each dish (model) lists its ingredients with ref() so the kitchen automatically cooks them in the right order; each recipe ships with a taste-test (test) that fails the whole service if the sauce is off; and a contract is the printed nutrition label diners can rely on even when you change the recipe behind it. Same SQL — now with the discipline of software.

Reflect

dbt's biggest cultural win is moving SQL into the same world as application code: PRs, code review, CI, semantic versioning. The biggest trap is treating it as the only tool — dbt is excellent at SQL transforms, mediocre at Python and ML, and not an orchestrator.

  • Which transforms in your platform still live outside dbt — and is that a deliberate boundary or accidental drift?
  • How would you stage a dbt contract rollout for a high-traffic table without breaking existing consumers?

Reading in progress · 0 of 2 activities done