This page describes how to use Census with S3.

πŸƒβ€β™€οΈ Getting Started

β€Œ In this guide, we will show you how to connect S3 to Census and create your first sync.


1. Create an S3 Destination in Census

    Once you are in Census, Navigate to Connections​
    Click the Add Service button
    Select S3 in the dropdown list
    Name your Destination (for example, s3) and enter the access key ID and secret key of an AWS account that has write credentials to your S3 bucket. Also enter the name and region of your AWS bucket. πŸ‘‡
Your AWS credentials will automatically be tested to ensure they are associated with a valid AWS account.

2. Connect your Data Warehouse

Please follow one of our short guides depending on your data warehouse technology:
After setting up your warehouse, your Census Connections Page should look like this:

3. Create your first Sync

Now head to the Sync page and click the Add Sync button
In the " What data do you want to sync?" section
    For the Connection, select the data warehouse you connected in step 2
    For the Source, select a model or table in your warehouse
Next up is the "Where do you want to sync data to?" section
    Pick your S3 destination (for us it is S3) as the Connection.
    A File Path text field populated with a placeholder file path will pop up below the connection:
    Change the file path to reflect the name and place of the csv file you want to send data to.
    File paths accept variables that are set when the sync is run, and you can see the preview of your file path if you were to run the sync right now in the Template Preview section.
    The full list of variables you can use to template your file path is:
    %Y => 4 number year like 1997
    %y => 2 number year like 97
    %m => zero padded month like 07
    %-m => month like 7
    %d => zero padded day like 03
    %-d => day like 3
    %H => zero padded 24 hour like 08 or 18
    %k => 24 hour like 8 or 18
    %I => zero padded 12 hour like 08 or 12
    %l => 12 hour like 8 or 12
    %M => zero padded minute like 04 or 56
    %S => zero padded second like 05 or 55
For the " How should changes to the source be synced?" section
    Mirror should automatically be selected.
Finally, select the fields you want to update in the Mapper in the "Which Fields should be updated?" section
    By default, all of your table columns will be added to the mapper. If you don't want that, simply remove the columns/fields you don't want to sync in the mapper.
The end result should look something like this:
Click the Next button to see the final preview which will have a recap of what will happen when you start the sync

4. Confirm the data was sent to your S3 Bucket

Now go back to your destination or service, in our case we will go to our s3 bucket and we should see a new file with the specified file path πŸŽ‰
That's it, in 4 steps, you used Census to sync data from your warehouse or source to your S3 bucket πŸŽ‰

πŸ”„ Supported Sync Behaviors

β€Œ πŸ”‹ Contact us if you want Census to support more Sync Behaviors for this destination

πŸ’‘ Things to know about the S3 connector

    Currently we only support syncing a file of up to 5 gigabytes to S3.
    Data arrives in one file to the designated S3 bucket/file path.
    Files are written as a csv with headers.
​Contact us if your use cases don't work with these current limitations, we plan on addressing at least a few of these limitations in the future!

πŸš‘ Need help connecting to S3?

β€Œ Contact us via [email protected] or start a conversion via the in-app chat.
Last modified 8d ago