HTTP Request

Send data anywhere that accepts standard HTTP requests.

Overview

Our HTTP Request destination is a Swiss Army knife. It allows you to send data to any service that accepts standard HTTP requests—including standard SaaS APIs and internal systems—and only takes a few minutes to set up via our no-code interface.

To demonstrate how it works, let's look at a real-world example of creating contacts in Brevo when new rows appear in your source data. If you want to follow along, you can create a new Brevo account here.

Create a destination

• Navigate to the Destinations page in Census.

• Click New Destination and select HTTP Request from the menu.

• Name your destination and enter the Base URL for the Brevo API (https://api.brevo.com).

• Generate a new API key in Brevo and copy it to your clipboard.

• Click Add Header, name the header api-key (per Brevo's docs), paste your API key under Value, and check the box indicating it's a secret (this will hide it in our UI if you or anyone else attempts to edit this connection later on).

• Click Connect to continue.

Create your HTTP Request destination.

Create a sync

Navigate to the Syncs page in Census and click New Sync.

Select your source

In this example, we'll use data from a demo Redshift cluster but this could be any source that we support.

Select your destination

Select the HTTP Request destination you just created and input the specific endpoint we'll be using to create contacts in Brevo.

Note: in this example, the endpoint is static (i.e. /v3/contacts), but we also support using the Liquid template language to populate values dynamically. For example, if we were sending requests to update existing contacts, we might need to include contact IDs in the endpoint like this: /v3/contacts/{{ record["id"] }}.

Choose your request trigger

Choose the trigger that you want Census to monitor in your source data.

Specify the number of records per request

Census allows you to either send one record at a time or batch records for better performance when the destination allows for it. In this case, we'll keep things simple and send one record at a time (which is also what this particular endpoint expects).

If you are using multiple records per request together with a JSON template payload, you will commonly need to use a Liquid for loop like this using the special records:

[
  {% for record in records %}
  {
    "id": "{{ record['ID'] }}",
    "name": "{{ record['NAME'] }}"
  },
  {% endfor %}
]

The records will be set to an array containing all of the records to be sent in the current request.

Select a sync key

This helps Census identify unique records in your source data.

Choose a request method

Census supports all major HTTP request methods. In this case, the endpoint we're syncing to expects a POST request.

Choose your payload type

We currently support the two most common payload types—empty and JSON. If you need more (e.g. XML, SOAP), please send us an email at [e-mail]!

This endpoint expects JSON, so we'll go with that.

Define your payload

Census provides three different ways of defining a payload.

• Template editor: using the Liquid template language to define an entirely custom payload. This is the most flexible option but requires a bit of elbow grease. See our full documentation to Liquid templates for more information.
• Single source column: select a single column from your source data that contains the entire JSON payload you want to send.

• Multiple source columns: select multiple columns from your source data. Census will combine them into a JSON payload with each column represented as a top-level key-value pair.

We'll use the template editor for this example so we can construct a payload that exactly matches what the Brevo API expects.

Configure rate limits

Brevo's contact endpoints have a rate limit of 10 requests per second (RPS). We can specify that here so that Census respects that limit when sending data.

Save and run your sync 👏

Once you've saved and run your sync, head over to the API Inspector tab in your sync configuration to confirm the data was sent as expected. In this case, it was and we can confirm the record made it to our contact list in Brevo!

Error Handling

Census will use the HTTP response status code to determine whether records were accepted by the target service. Here is how various response status codes are interpreted:

• 2xx - Success. All records sent in the request were accepted by the destination.

• 4xx - Client failure in the batch. All records sent in the request were rejected by the destination. The sync will continue, but the records sent in the batch will be marked as rejected and reattempted next time the sync is run.

  • NOTE: a 429 status code will trigger up to 7 request retries with exponential backoff

• 5xx - Server failure. The request will be retried up to 7 times with exponential backoff. If the request is unsuccessful within 7 retries, the entire sync will be failed.

FAQs

How do I configure a HTTP Destination to use basic auth?

Some destinations may use basic authorization instead of an API key.

This can be done by base64 encoding the username and password with the following format:

<username>:<password>

Then adding that as a header to the HTTP Request destination which should look like this: Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

Need help using HTTP Request?

Contact us via [e-mail] or start a conversation via the in-app chat.