Strategic advisors

Kent Graziano

Kent Graziano

The Data Warrior, Strategic Advisor, Data Vault Master, Author, Speaker, and Tae Kwon Do Grandmaster

Gordon Wong

Gordon Wong

Leading organizations through analytics transformations, preference for social missions, healthcare, energy, education, and civic engagement

Model in SqlDBM, build in dbt: Methods and tools of scalable data pipelines

Ask a data team where their data model lives, and you’ll often hear a confident answer: “we model in dbt.” As a long-time user and fan, I understand where that confidence comes from. Dbt brought version control, testing, and modularity to data transformation and earned its place at the center of the modern stack. Yet many data teams recognize (or intuit) that dbt alone isn’t the answer to an enterprise data solution that scales and keeps up with an organization’s growth. 

This article explores how modeling and building are complementary practices, each covering ground the other was never designed for. It then offers an overview of the features SqlDBM provides to support dbt users in any scenario: designing something new, making sense of a project you inherited, or keeping design and code in sync as both evolve.

You don’t model in dbt, you build in it

For the uninitiated, let’s address the elephant in the room. A dbt model is a materialized SELECT statement. A data model is a blueprint for what your data means and how it connects. By overloading the term “model”, dbt led a generation of engineers astray. The work that happens in dbt is best described not as modeling, but as building on top of an existing model. The question is, is that model formal or implicit? Agreed or assumed? 

Because dbt outputs something with a defined structure (i.e., tables and columns) and gives engineers speed, testing, and version control, it feels complete. But transformation is not modeling, nor are they at odds. Transformation and modeling are two halves of one cycle: plan and build. That is the governing principle of scalable data, now let’s look at some specifics. 

Three reasons engineering teams still need modeling 

Building in dbt is coding, and engineers love to code. It’s kinda their whole thing. So anything outside of coding often gets overlooked or dismissed as the cost of doing business. Let’s start with perhaps the biggest blind spot: “writing a join is easy”. Granted, but knowing what to join is hard. This latter part is rarely factored into the equation. The time spent asking around, organizing meetings, and sending gentle reminders could have been avoided with a glance at a diagram. 

The second mistake is confusing a DAG (Directed Acyclic Graph) for the landscape itself. Ref’ing in dbt creates the DAG which then serves as a terrific resource for lineage and debugging. But lineage alone only tells you how code executes; it doesn’t tell you how your business connects.

SqlDBM diagram showing a relationship line between dim_date and OPERATIONS_RAW.SUPPLIER compared to a dbt ref() line into DBT_SERGE_STAGE.DIM_SUPPLIER

Inverting the process makes this obvious: say a DAG shows CUSTOMER joining to ORDERS to create CUSTOMER_ORDER_ANALYSIS. Before the DAG existed, how was one to know that CUSTOMER and ORDERS could be joined in the first place, and before they were declared as sources, where customer and order information is even stored?

To drive the point home, think of directions on Google Maps: the blue line is the DAG, the map is the data model—once you’ve mapped the streets, you can trace the route.

Shift happens, and pretending otherwise is mistake number three. When structure inevitably shifts, changing a diagram is easy; refactoring a DAG is hard. In dbt, structure hardens through ref chains. But without a curated data model, that unplanned structure compounds into ungoverned sprawl, with symptoms including (but not limited to) competing versions of entities, faulty join explosions, multiple names for conformed dimensions, and the associated cognitive load.

Now that we’ve seen the pitfalls of neglecting modeling at the building stage, let’s explore how great things can be when the two work in concert. 

Division of labor

To get the most out of data modeling and transformation, let’s first unpack their areas of responsibility. The data model owns the what and why: entities, grain, keys, definitions, standards—artifacts the business can easily review and agree on. Dbt owns the how: transformational logic, tests, and orchestration, to name a few. In a simple example, this might look like the data team proposing a new entity and validating it with stakeholders, using a visual guide to ensure provenance, consistency, and expectations align. Agreeing this beforehand greatly minimizes the changes that an entire DAG will have to shift several months hence when inconsistencies are discovered. 

This process resembled that of a construction project. An architect lays down the blueprint; the crew builds to spec. Except, unlike in data engineering, nobody argues that having a construction crew replaces the need for an architect. 

With this in mind, let’s look at how SqlDBM’s dbt features help this coordination work seamlessly. 

SqlDBM and dbt in action

The workflow starts on the canvas. Entities, relationships, keys, and grain get settled visually, where a mistake costs a redraw, and everyone can participate, including stakeholders who never open dbt projects. Descriptions, ownership, and standards attach directly to objects and columns, so documentation accumulates as a byproduct of design instead of trailing behind it.

When the design settles, the same objects that produce warehouse DDL also produce dbt-ready YAML. Switch the output format on the Forward Engineering screen, choose between source and model YAML, and SqlDBM generates output complete with source headers, object names, columns, and any descriptions maintained in the project. Beyond the defaults, standard dbt properties like tests, freshness, tags, and versions are maintained as fields in Database Documentation, with pre-configured templates that spare you the syntax, and custom metadata such as owner or medallion layer flows into the meta tag. Best of all, since data types and constraints already live in the relational definition, including them in the output is a checkbox. That last point deserves a pause: dbt model contracts, which normally require hand-declaring every column, type, and constraint, come with no extra effort. 

SqlDBM’s Forward Engineering screen generating dbt YAML for selected model objects, alongside the resulting YAML script

The workflow also runs in reverse. Upload your project’s manifest.json, and SqlDBM matches its sources and models to the objects on your diagram, classifies each one accordingly, records its dbt project, package, and file path, and fills in any descriptions the model is missing, without ever overwriting the ones you already have. It also reads the ref() calls in the manifest and draws them as dependency relationships on the diagram, turning your project’s lineage into something you can see alongside the relational design. 

Once the assets are generated, SqlDBM’s push-to-git functionality removes that last manual step. The model writes its generated files directly to the repository and follows the same pull request workflow as the code it describes. At that point, the round trip is complete: design in the model, lock in with (optional) dbt contracts, then build in dbt. Each side keeps the other honest.

Those are the tools; now let’s run through some practical use cases that put them to use. 

Putting the tools to work

Different challenges call for different approaches. This section will cover three common scenarios where each puts the functionality outlined in the previous section to work.

Diagram of the SqlDBM and dbt workflow: sources feed SqlDBM for entity design, Forward Engineering produces DDL and dbt YAML pushed to git, dbt builds and tests the warehouse, and manifest.json reverse-engineers back into the model

The business asks for a new mart, say subscription revenue. The model-first team opens a diagram before writing any code. They collaboratively decide on the entities, grain, and keys while including finance representatives in the discussion. This approach creates a reviewable diagram everyone can reference, since their input shaped it. Once the design is agreed, Forward Engineering produces the dbt YAML complete with the tests, properties, and metadata maintained alongside the design. With the handoff, engineers begin with scaffolding rather than a blank file, and their effort goes where it belongs: transformation logic. By the time anyone opens an editor, the hard question of what to join has already been answered.

Next, a new lead takes over a dbt project with 500 models and three generations of departed authors. Dbt docs will happily render the DAG, and the DAG is honest about execution order while staying silent about the domain. One manifest upload later, the picture changes: objects are classified as sources and models, descriptions flow in where the diagram had none, and every ref() appears as a visible dependency sitting alongside the relational structure. Orphaned models also surface by remaining unlabeled. Redundant work stops hiding, and the five versions of customer finally stand in the same frame. That’s the power of relational and transformational in a single frame, and SqlDBM is the only tool that can render it. 

Let’s end with a major change. Finance changes how revenue is recognized, a source system gets replaced, or the company moves from licenses to subscriptions. In a code-only shop, impact analysis begins with grep and ends with hope. With an up-to-date model, the change happens on the diagram first: the ripple through related entities is visible before a line of code moves, stakeholders review the impact in a form they understand, and only then does regenerated DDL and YAML head for the repository. This is battle-tested insight: the cheapest refactor is the one that happens before the code exists.

Objection!

Even when presented with a better alternative, people are often reluctant to change their behavior. Here are some of the most common arguments I hear from teams that haven’t yet realized that a minute of planning is worth an hour of rework. 

“Modeling slows us down.”

Modeling costs hours; rework costs sprints. The slow path is discovering the wrong grain in production and refactoring every model downstream. Scaffolded YAML, pre-answered join questions, and impact analysis before code changes all remove work from the sprint. Iterate on the model the same way you iterate on code, just earlier, where iteration is nearly free. Teams that skip design don’t go faster; they pay for the same decisions later, with interest.

“Our dbt docs are the model.”

Docs describe what was built, not what should exist. They’re generated after the fact, from the code, so they inherit every accident in it. A rendered DAG of five customer models is documentation of a problem, not a design.

“We’re an ELT shop, not a warehouse team.”

Your medallion layers still have entities, grain, and keys, whether anyone drew them or not. Every team has a data model; some just haven’t seen theirs.

Conclusion – start with the map

The final takeaway isn’t an either-or. Dbt earned its place at the center of the stack, and a well-built transformation layer is worth protecting. This article makes a narrower claim: building answers how, and something still has to answer what. Teams that pair the two get designs the business can review, scaffolding engineers never have to write by hand, and a shared map that stays current as the project grows.

Design, review, build, deploy. Each half keeps the other honest, and the ultimate takeaway is value: faster time to market, fewer mistakes, and lower long-term cost of ownership.

If you haven’t paired your data model with dbt, make that your next move. Grab a manifest.json from dbt’s target directory, and upload it to SqlDBM. What comes back is the landscape you’ve been navigating from memory: sources and models classified, descriptions filled in, every ref() drawn as a relationship you can finally see. Writing the join was never the hard part. What to join is now one upload away.

Continue reading: From No Model to AI Context Layer: The Data Modeling Maturity Ladder →