Data Storage
How we store your data
We store the data you sync using Census, sometimes for the lifetime of your Census workspace, in three ways:
Your connected warehouse: for Basic Datasets shared from your warehouse, we write transformations and advanced sync engine logs back to the warehouse itself. This data includes:
Census Store: this is an Iceberg-based managed storage. Items stored in Census Store can be accessed by external compute, including warehouses. This data includes:
Entity Resolution datasets
Census store is also the storage location for AI Columns, Enrichment Columns, and Warehouse Writeback logs for any of these dataset types.
General Object Storage: this is used to house data related to sync execution. This data includes:
Snapshots of the data you sync using Basic Sync Engine
Sync Tracking logs of the data you sync
Temporary files used to load and unload data
Census Store & General Object Storage data is stored in cloud-based object storage providers. Census provides object storage locations in multiple clouds in the US and the EU that you can use to store this data, and your Census organization is configured to use these locations by default. You can also add your own object storage locations to Census.
Census-provided object storage
New Census organizations are configured to use Census-provided object storage by default. Census manages object storage locations in multiple public clouds, including AWS (S3), GCP (GCS), and Azure (Blob Storage). This includes managing credentials with narrow permissions and short lifetimes, encrypting data at rest and in transit, automatically removing old data once it is no longer in use, and ensuring your workspace is always storing data in the correct region (US or EU). This approach is secure and is used by the vast majority of Census customers.
Customer-provided object storage
When you bring your own object storage, you are responsible for managing the retention of data in the object storage location, and for ensuring the object storage is located in an appropriate region for your requirements.
Last updated
Was this helpful?