Ghost models and spooky manifests in dbt

It's February, but every day is Halloween in the data warehouse. Do you have ghosts in your dbt project? If so, they could be costing you days of lost productivity and thousands of dollars in data warehouse compute. However, the problem is also very easy to fix!

The ghost model scenario

For the rest of the post, suppose we have the following scenario:

• Production job 1 runs dbt build --select tag:tag1 on a regular schedule


• Production job 2 runs dbt build --select tag:tag2 on a regular schedule


• There is a model ghost in the production Git branch that's not tagged tag1 or tag2 - so it never runs on either schedule and never gets materialized at all.

Spooky manifests

The file manifest.json is regenerated every time you run a dbt command (with some exceptions, see the that allow your team to automatically test proposed changes, but those jobs can easily become very slow and very expensive if not managed properly.

The best way to manage those costs while still getting the full benefits of CI is by combining state selection and deferral into something called has this nice diagram. The diagram shows that if you just modified model_c, you don't need to rebuild its unmodified ancestors model_a and model_b, and instead just point to the production version:

to build all ancestors of all descendants of the selected model: dbt build --select @state:modified. What could be just running a handful of models with deferral could become hundreds of models without deferral.

Summary: Ghost models make your CI expensive and slow

Models that exist in your dbt project's primary branch but don't get materialized in the data warehouse are ghost models. Your production job manifests will include them as if they really exist, but slim CI pipelines will crash and burn when they try to defer to them. This will force you to move away from deferral, which could cause an exponential increase of your CI runtime and costs.

An easy fix!

The fix to the problem is hinted at in the them. It won't affect production since they don't get materialized anyway. Then they'll stop showing up in the manifest and slim CI will work again! When you need the ghost model in the future, you can enable it, and then it will run as part of your CI pipeline. Just remember to make it run on a schedule before merging to production!