dbt Models

Census supports connecting to an existing dbt project, which allows you to select models you want to make available to sync into all your business tools. This means you can keep all your source code & transforms in a single repository. 
Census compiles your models on the fly whenever a sync is scheduled so your data and your models are always up to date. And Census is designed to work hand-in-hand with dbtCloud or any other dbt runner.
Setting it up
Here are the instructions for connecting a dbt project:
1.Connect to your GitHub repository. We currently only support dbt projects stored in GitHub. If you’d prefer to use a different service, please let us know!
2.Select the branch (if any) you’d like Census to use. Census will refresh the project on a regular basis and detect any changes to your models. You can force a refresh at any point from the models' page.
3.Customize the Census model selector. Any model exposed to Census becomes available as a source for syncing your data to external tools. By default, Census looks for models with the census tag but you can customize the filter.
Example Selectors
All models with a tag: tag:census
All models in a directory: path/to/models
All models: *
For more information on selector syntax, see dbt’s Model Selector Syntax.
Finally, specify where intermediate models are materialized. If the models you expose in Census have dependencies, we will attempt to use these materialized tables. You may need to ensure that our database connection has read access to these tables.
Once you’ve configured your project repository, Census will analyze your project and display the models you’ve made available. You’re now ready to start using these models as part of Census syncs!
Coordinating with dbt Cloud
If you're using dbt Cloud to run your dbt project, our integration goes even further. You can configure Census to automatically run syncs whenever your models have been rebuilt. See our documentation on connecting and configuring dbt Cloud.
Required data warehouse permissions
Census doesn't necessarily require the same permissions your dbt project needs because Census only runs the models you've exposed to Census during set up. Census only requires read access to your selected models and any of their materialized dependencies. That means you can use dbt's materialize configuration flag to create permissions boundaries. Once materialized dependencies are generated by dbt runner, Census will reference the materialized results when accessing your models.
dbt version and unsupported features
Our dbt integration is designed to pair nicely with your existing dbt runner, whether dbt Cloud or self-hosted. We do this by using the dbt compile command rather than the typical dbt run and then make use of the compiled output only. 
As a result, there's several dbt features that Census does not make use of. These include:
Materialization directives. Census doesn’t currently materialize your tables back to your data warehouse. Census will however use materialized tables by your dbt runner to speed up the execution
Changing the default schema behavior by overriding the generate_schema_name macro​
Pre and post hooks
Non-public packages
Our dbt integration currently supports version 1.0. We also post version support in our changelog.
Upgrading to dbt 1.0
Thankfully, upgrading to dbt 1.0 is pretty straightforward for most projects. Here's the most common changes we see for most Census + dbt users:
Upgrade your dbt packages by updating their version in packages.yml and then running dbt deps. 
The most common compatibility issue is dbt_utils . To fix, update your packages.yml with the following reference:

  - package: dbt-labs/dbt_utils
    version: 0.7.6 

(This version is also compatible back to dbt 0.20)
You'll also start to get warnings renaming  source-paths to model-paths and data-paths to seed-paths with a default value of seeds. Both of these can be ignored for now and will be removed in a future version.
For more information, dbt has written up a detailed list of the changes.
​

Models - Previous

Census Models

Next - Models

Looker

Last modified 6mo ago

Copy link

Outline

Setting it up

Coordinating with dbt Cloud

Required data warehouse permissions

dbt version and unsupported features