Have an S3 bucket ready for use, along with the credentials of an AWS user that can write to your S3 bucket. You can check out the AWS docs here to set up an S3 bucket, and you can manage AWS IAM credentials here.
Have the proper credentials to access to your data source. See our docs for each supported data source for further information:
Name your Destination (for example, s3) and enter the access key ID and secret key of an AWS account that has write credentials to your S3 bucket. Also enter the name and region of your AWS bucket. 👇
Your AWS credentials will automatically be tested to ensure they are associated with a valid AWS account.
2. Connect your Data Warehouse
Please follow one of our short guides depending on your data warehouse technology:
After setting up your warehouse, your Census Connections Page should look like this:
3. Create your first Sync
Now head to the Sync page and click the Add Sync button
In the " What data do you want to sync?" section
For the Connection, select the data warehouse you connected in step 2
For the Source, select a model or table in your warehouse
Next up is the "Where do you want to sync data to?" section
Pick your S3 destination (for us it is S3) as the Connection.
A File Path text field populated with a placeholder file path will pop up below the connection:
Change the file path to reflect the name and place of the csv file you want to send data to.
File paths accept variables that are set when the sync is run, and you can see the preview of your file path if you were to run the sync right now in the Template Preview section.
The full list of variables you can use to template your file path is:
%Y => 4 number year like 1997
%y => 2 number year like 97
%m => zero padded month like 07
%-m => month like 7
%d => zero padded day like 03
%-d => day like 3
%H => zero padded 24 hour like 08 or 18
%k => 24 hour like 8 or 18
%I => zero padded 12 hour like 08 or 12
%l => 12 hour like 8 or 12
%M => zero padded minute like 04 or 56
%S => zero padded second like 05 or 55
For the " How should changes to the source be synced?" section
Mirror should automatically be selected.
Finally, select the fields you want to update in the Mapper in the "Which Fields should be updated?" section
By default, all of your table columns will be added to the mapper. If you don't want that, simply remove the columns/fields you don't want to sync in the mapper.
The end result should look something like this:
Click the Next button to see the final preview which will have a recap of what will happen when you start the sync
4. Confirm the data was sent to your S3 Bucket
Now go back to your destination or service, in our case we will go to our s3 bucket and we should see a new file with the specified file path 🎉
That's it, in 4 steps, you used Census to sync data from your warehouse or source to your S3 bucket 🎉
🔄 Supported Sync Behaviors
Behaviors
Supported?
Objects?
Mirror
✅
All
🔋 Contact us if you want Census to support more Sync Behaviors for this destination
💡 Things to know about the S3 connector
Currently we only support syncing a file of up to 5 gigabytes to S3.
Data arrives in one file to the designated S3 bucket/file path.
Files are written as a csv with headers.
Contact us if your use cases don't work with these current limitations, we plan on addressing at least a few of these limitations in the future!