LogoLogo
  • 🦩Overview
  • 💾Datasets
    • Overview
    • Core Concepts
      • Columns & Annotations
      • Type & Property Mappings
      • Relationships
    • Basic Datasets
      • dbt Integration
      • Sigma Integration
      • Looker Integration
    • SaaS Datasets
    • CSV Datasets
    • Streaming Datasets
    • Entity Resolution
    • AI Columns
      • AI Prompts Recipe Book
    • Enrichment Columns
      • Quick Start
      • HTTP Request Enrichments
    • Computed Columns
    • Version Control
  • 📫Syncs
    • Overview
    • Triggering & Scheduling
    • Retry Handling
    • Live Syncs
    • Audience Syncs
    • Observability
      • Current Sync Run Overview
      • Sync History
      • Sync Tracking
      • API Inspector
      • Sync Alerts
      • Observability Lake
      • Datadog Integration
      • Warehouse Writeback
      • Sync Lifecycle Webhooks
      • Sync Dry Runs
    • Structuring Data
      • Liquid Templates
      • Event Syncs
      • Arrays and Nested Objects
  • 👥Audience Hub
    • Overview
    • Creating Segments
      • Segment Priorities
      • Warehouse-Managed Audiences
    • Experiments and Analysis
      • Audience Match Rates
    • Activating Segments
    • Calculated Columns
    • Data Preparation
      • Profile Explorer
      • Exclusion Lists
  • 🧮Data Sources
    • Overview
    • Available Sources
      • Amazon Athena
      • Amazon Redshift
      • Amazon S3
      • Azure Synapse
      • ClickHouse
      • Confluent Cloud
      • Databricks
      • Elasticsearch
      • Kafka
      • Google AlloyDB
      • Google BigQuery
      • Google Cloud SQL for PostgreSQL
      • Google Pub/Sub
      • Google Sheets
      • Greenplum
      • HTTP Request
      • HubSpot
      • Materialize
      • Microsoft Fabric
      • MotherDuck
      • MySQL
      • PostgreSQL
      • Rockset
      • Salesforce
      • SingleStore
      • Snowflake
      • SQL Server
      • Trino
  • 🛫Destinations
    • Overview
    • Available Destinations
      • Accredible
      • ActiveCampaign
      • Adobe Target
      • Aha
      • Airship
      • Airtable
      • Algolia
      • Amazon Ads DSP (AMC)
      • Amazon DynamoDB
      • Amazon EventBridge
      • Amazon Pinpoint
      • Amazon Redshift
      • Amazon S3
      • Amplitude
      • Anaplan
      • Antavo
      • Appcues
      • Apollo
      • Asana
      • AskNicely
      • Attentive
      • Attio
      • Autopilot Journeys
      • Azure Blob Storage
      • Box
      • Bloomreach
      • Blackhawk
      • Braze
      • Brevo (formerly Sendinblue)
      • Campaign Monitor
      • Canny
      • Channable
      • Chargebee
      • Chargify
      • ChartMogul
      • ChatGPT Retrieval Plugin
      • Chattermill
      • ChurnZero
      • CJ Affiliate
      • CleverTap
      • ClickUp
      • Constant Contact
      • Courier
      • Criteo
      • Crowd.dev
      • Customer.io
      • Databricks
      • Delighted
      • Discord
      • Drift
      • Drip
      • Eagle Eye
      • Emarsys
      • Enterpret
      • Elasticsearch
      • Facebook Ads
      • Facebook Product Catalog
      • Freshdesk
      • Freshsales
      • Front
      • FullStory
      • Gainsight
      • GitHub
      • GitLab
      • Gladly
      • Google Ads
        • Customer Match Lists (Audiences)
        • Offline Conversions
      • Google AlloyDB
      • Google Analytics 4
      • Google BigQuery
      • Google Campaign Manager 360
      • Google Cloud Storage
      • Google Datastore
      • Google Display & Video 360
      • Google Drive
      • Google Search Ads 360
      • Google Sheets
      • Heap.io
      • Help Scout
      • HTTP Request
      • HubSpot
      • Impact
      • Insider
      • Insightly
      • Intercom
      • Iterable
      • Jira
      • Kafka
      • Kevel
      • Klaviyo
      • Kustomer
      • Labelbox
      • LaunchDarkly
      • LinkedIn
      • LiveIntent
      • Loops
      • Mailchimp
      • Mailchimp Transactional (Mandrill)
      • Mailgun
      • Marketo
      • Meilisearch
      • Microsoft Advertising
      • Microsoft Dynamics
      • Microsoft SQL Server
      • Microsoft Teams
      • Mixpanel
      • MoEngage
      • Mongo DB
      • mParticle
      • MySQL
      • NetSuite
      • Notion
      • OneSignal
      • Optimizely
      • Oracle Database
      • Oracle Eloqua
      • Oracle Fusion
      • Oracle Responsys
      • Orbit
      • Ortto
      • Outreach
      • Pardot
      • Partnerstack
      • Pendo
      • Pinterest
      • Pipedrive
      • Planhat
      • PostgreSQL
      • PostHog
      • Postscript
      • Productboard
      • Qualtrics
      • Radar
      • Reddit Ads
      • Rokt
      • RollWorks
      • Sailthru
      • Salesforce
      • Salesforce Commerce Cloud
      • Salesforce Marketing Cloud
      • Salesloft
      • Segment
      • SendGrid
      • Sense
      • SFTP
      • Shopify
      • Singular
      • Slack
      • Snapchat
      • Snowflake
      • Split
      • Sprig
      • Statsig
      • Stripe
      • The Trade Desk
      • TikTok
      • Totango
      • Userflow
      • Userpilot
      • Vero Cloud
      • Vitally
      • Webhooks
      • Webflow
      • X Ads (formerly Twitter Ads)
      • Yahoo Ads (DSP)
      • Zendesk
      • Zoho CRM
      • Zuora
    • Custom & Partner Destinations
  • 📎Misc
    • Credits
    • Census Embedded
    • Data Storage
      • Census Store
        • Query Census Store from Snowflake
        • Query Census Store locally using DuckDB
      • General Object Storage
      • Bring Your Own Bucket
        • Bring your own S3 Bucket
        • Bring your own GCS Bucket
        • Bring your own Azure Bucket
    • Developers
      • GitLink
      • Dataset API
      • Custom Destination API
      • Management API
    • Security & Privacy
      • Login & SSO Settings
      • Workspaces
      • Role-based Access Controls
      • Network Access Controls
      • SIEM Log Forwarding
      • Secure Storage of Customer Credentials
      • Digital Markets Act (DMA) Consent for Ad Platforms
    • Health and Usage Reporting
      • Workspace Homepage
      • Product Usage Dashboard
      • Observability Toolkit
      • Alerts
    • FAQs
Powered by GitBook
On this page
  • Prerequisites
  • Step 1: Create a Google Cloud Storage Connection
  • Step 2: Grant Access to GCS
  • Step 3: Create your first sync
  • Supported Sync Behaviors
  • File Path
  • Variables
  • Advanced Configuration
  • Other Things to Know
  • Need help connecting to Google Cloud Storage?

Was this helpful?

  1. Destinations
  2. Available Destinations

Google Cloud Storage

This page describes how to use Census with Google Cloud Storage.

PreviousGoogle Campaign Manager 360NextGoogle Datastore

Last updated 8 months ago

Was this helpful?

Prerequisites

  • Have your Census account ready. If you need one, now.

  • Have your Google Cloud Storage Account in the on the bucket that you want to sync to.

  • Have the proper credentials to access to your data source. See our docs for each supported data source for further information:

Step 1: Create a Google Cloud Storage Connection

Our Google Cloud destination behaves a little differently than other Census destinations. Instead of going through an OAuth connection flow, we provide you a Google Identity address to which you grant Storage Object permissions.

  • Click the New Destination button, and select Google Cloud Storage

  • Paste the name of the GCS bucket, name the connection, and click the Save Connection button

Step 2: Grant Access to GCS

  • Now head to the Google Cloud Storage bucket that you want to sync to, and click into that bucket's details.

  • Click the Permissions tab on the Bucket, then click "Add Permissions".

  • Paste the credentials in the New Principals portion and select "Storage Object Admin."

You will be able to send a successful sync if the file path variables are unique per sync run by only granting this service account "Storage Object Creator", but the Test Connection in the next step will fail.

  • Click save to save the permission.

Step 3: Create your first sync

The sync will move data from your warehouse to your GCS bucket. In this step, you'll define how that will work.

  1. Under What data do you want to sync?, choose your data warehouse as the Connection and your source: either a Model from the Census Models tab or a database table.

  2. Under How should changes to the source be synced?, Replace will be automatically selected. This is the only supported sync behavior for GCS.

  3. Under Which properties should be updated?, choose whether to sync only Selected Properties or Sync All Properties. Syncing all properties will automatically add new properties to the sync if the model or database table changes.

  4. To test your sync without actually syncing data, click Run Test and verify the results.

  5. Click the Next button to see the final preview which will have a recap of what will happen when you start the sync. If you're happy, check the Sync Now checkbox and save the sync.

  6. Confirm the data arrives in the GCS bucket!

Supported Sync Behaviors

Behavior

Supported?

Objects

Replace

✅

All

Learn more about all of our sync behaviors on our Core Concepts page.

File Path

When setting up a sync to GCS, you can provide a file path for the file name Census will create/replace. The file path can include folders. Data arrives in one file to the designated bucket and file path.

Variables

When defining the File Path, you can use variables that will be set when the sync runs. This allows you to create and sync to new files that reflect the date and time of the sync.

Variable

Description

Example Values

%Y

4-digit year

1997

%y

2-digit year

97

%m

month with zero padding

07, 12

%-m

month without zero padding

7, 12

%d

day with zero padding

03, 23

%-d

day without zero padding

3, 23

%H

24 hour with zero padding

08, 18

%k

24 hour without zero padding

8, 18

%I

12 hour with zero padding

08, 12

%l

12 hour without zero padding

8, 12

%M

minute with zero padding

04, 56

%S

second with zero padding

06, 54

Advanced Configuration

In addition to the file path, you can configure how the data is encoded as it is written. Primarily this is a question of file format:

  • CSV - The standard comma separated values file. You can optionally specify an alternative delimeter such as |*, and you can enable/disable the header row.

  • TSV - The tab separated values file. You can enable/disable the header row.

  • JSON - A single JSON arraay of objects

  • NDJSON - New line-delimited list of JSON objects

  • Parquet - A columnar storage format that is more efficient for certain types of data.

  • If your configured delimiter is present in data values, Census will automatically add double quotes around the value. Example: Hello, world is written as as "Hello, world" if the chosen delimiter is a comma.

In addition to file format, you can also provide a PGP Public Key to encrypt the data before it is written to the file. This is useful for ensuring that the data is secure in transit and at rest.

Other Things to Know

  • Currently, the destination only supports syncing for files up to 5GB.

Need help connecting to Google Cloud Storage?

In Census, navigate to

Your new Google Cloud Storage connection will include a GCP service account for that specific Census connection. Click the copy button () to save it to your clipboard.

Then navigate back to the page and click "Test" on the service.

From inside your Census account, navigate to the page and click Add Sync.

Under Where do you want to sync data to?, choose the name you assigned in Step 1 (we used GCS) as the Connection. Enter the File Path for the file where data will sync. The path can accept variables that will populate when the sync runs. See . Confirm the file path in the Template Preview field.

__ If the service account only has "Storage Object Creator" permissions, this file path needs to be unique per sync run

If you need any help configuring GCS, contact the to get some help.

if you want Census to support additional sync behaviors for Google Cloud Storage.

if your use cases don't work with these limitations. We plan on addressing at least a few of these in the future!

You can send our at support@getcensus.com or start a conversation from the in-app chat.

🛫
💡
create a Free Trial Census account
Browser tab
Azure Synapse
Databricks
Elasticsearch
Google BigQuery
Google Sheets
MySQL
Postgres
Redshift
Snowflake
SQL Server
Destinations
Destinations
Syncs
Census support team
Let us know
Contact us
support team an email
File Path Variables
You are good to go!