Liquid Templates

Census uses the Liquid template language to give you extra control over the data you send.

These are tiny programs that let you change the format of data using a simple snippet of code. For example, you can use Liquid to...

• Add prefixes or suffixes

• Change to upper or lower case

• Format dates

• Combine columns together

• Conditionally use alternate values

• Build JSON objects or arrays

Liquid is easy to learn and also very powerful. We look forward to seeing the templates you build!

Census supports the full standard set of Liquid 5.4 template features, filters, and tags (with the minor exception of the include and render tags for external files).

Census does not support any Shopify- or Jekyll-specific extensions to Liquid templates, so be sure you're referencing the standard docs at shopify.github.io/liquid.

Getting started

Your most basic operation in Liquid is to reference a record column, like this:

{{ record['NAME'] }}

View result

Data

NAME: Boris Jabes

Result

Boris Jabes

You can add text before or after a variable reference by putting it outside the {{ and }} markers:

Customer #{{ record['ID'] }} (new)

View result

Data

ID: 1234

Result

Customer #1234 (new)

By using two variable references in one template, you can combine columns together:

{{ record['LNAME'] }}, {{ record['FNAME'] }}

View result

Data

FNAME: Jane
LNAME: Smith

Result

Smith, Jane

Another very common job for Liquid is transforming text. Liquid filters are activated with the | (pipe) symbol:

{{ record['EMAIL'] | downcase }}

View result

Data

EMAIL: [email protected]

Result

[email protected]

Column references

All of the columns on your source record are available using the record['COLUMN_NAME'] syntax. When Census sees this syntax, it knows this column is required from the source.

The exact column name must be used in the record property accessor.

For example, if your source column name is COMPANY_ID, then you must use record['COMPANY_ID'] in your Liquid template. Any other variations such as company_id, companyid, or companyID will be raised as validation errors.

Any columns not explicitly referenced are not retrieved from the source, which ensures that syncs using Liquid templates run just as fast as all your other syncs on Census.

Standard Filters

There are over 40 standard Liquid filters to be found in the left sidebar in the official documenation. Here are a few common ones...

downcase

Convert text to lowercase:

{{ record['CUSTOMER_ID'] | downcase }}

View result

Data

CUSTOMER_ID: ABc123

Result

abc123

upcase

Convert text to uppercase:

{{ record['STATE_CODE'] | upcase }}

View result

Data

STATE_CODE: hi

Result

HI

replace

Substitute all example text with the provided result text:

{{ record['PRODUCT_ID'] | replace: "_", "-" }}

View result

Data

PRODUCT_ID: abc_xyz

Result

abc-xyz

strip

Remove space characters from the start and end of the text:

{{ record['COMPANY_NAME'] | strip }}

View result

Data

COMPANY_NAME: "  Acme Inc.    "

Result

Acme Inc.

date

Format a timestamp as a text date:

{{ record['PURCHASED_AT'] | date: "%Y-%m-%d %H:%M" }}

View result

Data

PURCHASED_AT: 1698166115

Result

2023-10-24 16:48

Census Filters

Census also includes a number of custom filters specifically designed to help you with data transforms...

json

Encode a value into JSON syntax:

{{ record["EXTRA_DATA"] | json }}

View result

Data

EXTRA_DATA:
  a: 1
  b: true
  c: "text"

Result

{"a":1,"b":true,"c":"text"}

This can be useful when working with APIs or structured data.

except

Get a subset of an object with some keys removed:

{{ record["EXTRA_DATA"] | except: "b", "c" | json }}

View result

Data

EXTRA_DATA:
  a: 1
  b: 2
  c: 3
  d: 4

Result

{"a":1,"d":4}

Often used together with json. Also accepts an array or an object as a single argument.

only

Get a subset of an object containing just some keys:

{{ record["EXTRA_DATA"] | only: "b", "c" | json }}

View result

Data

EXTRA_DATA:
  a: 1
  b: 2
  c: 3
  d: 4

Result

{"b":2,"c":3}

Often used together with json. Also accepts an array or an object as a single argument.

keys

Get all of the keys of an object:

{{ record["EXTRA_DATA"] | keys | json }}

View result

Data

EXTRA_DATA:
  a: 1
  b: 2
  c: 3
  d: 4

Result

["a","b","c","d"]

sha256

Get the SHA-256 hash of a value and return the hex representation:

{{ record['EMAIL'] | sha256 }}

View result

Data

EMAIL: [email protected]

Result

388c735eec8225c4ad7a507944dd0a975296baea383198aa87177f29af2c6f69

Can be helpful for generating hashed identifiers for advertising destinations like Google Ads or LinkedIn.

base64

Encode a value using Base64:

{{ record['VALUE'] | base64 }}

View result

Data

VALUE: "<b>hello\nworld</b>"

Result

PGI+aGVsbG8Kd29ybGQ8L2I+

May be useful to work around fields that don't allow some special characters.

Validation

As you write your Liquid templates, it's possible that you may make a typo or don't quite have the right syntax. Census will catch all these errors as you type, and you'll know about them well before your sync starts running.

Here are some invalid template examples that will be caught by Census:

• {{ record['ID' }} → Liquid syntax error: Expected close_square but found end_of_string in "{{ record['ID' }}"

• {{ record['NAMEE'] }} → Column being referenced in template can't be found in data source [NAMEE]

• {% assign category = record['CAT'] %}{{ categoryy.ratio | times: 100 }} → [categoryy.ratio | times: 100] unknown variable `categoryy`

Compared to some other implementations of Liquid, Census runs using strict parsing and rendering. This means that invalid templates won't run, and any errors encountered when running a valid template won't cause bad data to be sent to your destination.

Advanced

The examples above just scratch the surface of what can be done with Liquid templates. The language exposes enough capabilities to define complex conditions, process structured data, and do some really advanced data manipulation.

Custom variables

If you're writing advanced expression and want to store an intermediate result, you can create your own Liquid variables to help (official docs).

assign

Make a new variable from a value:

{% assign name = record['NAME'] | upcase %}

capture

Make a new variable from captured text:

{% capture full_name -%}
  {{ record['FNAME'] }} {{ record['LNAME'] }}
{%- endcapture %}

This is great for text variables that combine multiple columns or make use of multiple Liquid filters or tags.

Conditionals

You can build advanced conditional code with Liquid, for example, to use one variable or another, or to "translate" values for your destination (official docs).

if

Construct a condition using boolean operators:

{% if record['LTV'] > 1000 -%}
  diamond
{%- elsif record['LTV'] > 100 -%}
  gold
{%- else -%}
  N/A
{%- endif %}

case/when

Construct conditions based on matching values:

{% case record['TYPE'] -%}
  {%- when "customer" -%}
     {{ record['CNAME'] }}
  {%- when "prospect" -%}
     {{ record['PNAME'] }}
  {%- else -%}
     anon
{%- endcase %}

Looping

You can also iterate over arrays to build text with repeating elements (official docs).

for

Generate text for each item:

[
  {% for tag in record %}
  "{{ tag | upcase }}",
  {% endfor %}
]

Commonly used in JSON mode and with batched records.

Whitespace

If you're using spaces in a complex template and notice your output now includes unwanted spaces, this is a sign that you need to use Liquid whitespace control syntax.

Liquid is basically text building engine. You can think of {{ ... }} and {% ... %} as "insertion points" into a text file...

• The code inside the brackets will get run by Liquid, and it doesn't make a difference if there's whitespace inside the code, only the code's output will be inserted.

• On the other hand, any whitespace outside of those insertion points will be preserved by Liquid, just like any other text.

Preserved whitespace won't be an issue for text formats like HTML and JSON where whitespace is collapsed or ignored, but for many other use cases, whitespace is significant and causes problems.

To deal with this, Liquid uses the - (hyphen) symbol for whitespace control, stripping out whitespace at the insertion point where the symbol is used (official docs).

Here are a few examples of whitespace inside vs. outside Liquid code blocks (using {id: 1} as the record data)...

Template	Result	Note
"id{{ record['id'] }}"	"id1"	1
" {{record['id']}} "	" 1 "	2
"{% if true %}{{ record['id'] }}{% endif %}"	"1"	3
"{% if true %} {{ record['id'] }} {% endif %}"	" 1 "	4
"{% if true %} {{- record['id'] -}} {% endif %}"	"1"	5
"{% if true -%} {{ record['id'] }} {%- endif %}"	"1"	6

Notes:

1. Whitespace inside code has no effect

2. Whitespace outside code is preserved

3. No whitespace between tags gives no result whitespace

4. Whitespace between tags is preserved

5. Whitespace control symbol strips whitespace around variable

6. Whitespace control symbol strips whitespace inside tags

Whitespace includes single spaces, line breaks, and tabs—any of the blank spaces you see in your template code.

JSON mode

One special use for Liquid templates is to build JSON documents. This can be great for advanced API usage with templated fields or even building custom integrations with the HTTP Request destination.

Two special features get activated when in JSON mode:

Escaped strings

JSON strings are escaped by default, which means that you don't need to worry about source data causing JSON syntax errors.

{
  "comment": "{{ record['COMMENT'] }}"
}

View result

Data

COMMENT: 'They said "hello"'

Result

{
  "comment": "They said \"hello\""
}

This behavior can be overridden using either the json filter (to encode a value into valid JSON) or the raw filter (when your source data is already valid JSON).

{
  "json_data": {{ record['OBJECT'] | json }},
  "raw_data": {{ record['RAW] | raw }}
}

View result

Data

OBJECT:
  x: 1
  y: 2
RAW: '{"a":3,"b":4}'

Result

{
  "json_data": {
    "x": 1,
    "y": 2
  },
  "raw_data": {
    "a": 3,
    "b": 4
  }
}

Flexible syntax

When generating JSON data using for loops, this process will often leave a , after the last item in an array or object, which is invalid JSON.

{
  "products": [
    {% for product in record["PRODUCTS"] %}
    {
      "id": {{ product.id }},
      "name": "{{ product.name }}"
    },
    {% endfor %}
  ]
}

View result

Data

PRODUCTS:
  - id: 1
    name: a
  - id: 2
    name: b

Intermediate YAML

{
  "products": [
    {
      "id": 1,
      "name": "a"
    },
    {
      "id": 1,
      "name": "b"
    },
  ]
}

Final JSON

{
  "products": [
    {
      "id": 1,
      "name": "a"
    },
    {
      "id": 1,
      "name": "b"
    }
  ]
}

Instead of triggering an error, Census allows this syntax and converts it into valid JSON. It does this by parsing the template output as YAML, which is a strict superset of JSON (allows all valid JSON) that accepts these "trailing commas".

If you want, you're free to use any other YAML features, like block-style collections, and the result will still be converted to valid JSON:

products:
  {% for product in record["PRODUCTS"] %}
  - id: {{ product.id }}
    name: "{{ product.name }}"
  {% endfor %}

View result

Data

PRODUCTS:
  - id: 1
    name: a
  - id: 2
    name: b

Intermediate YAML

products:
  - id: 1
    name: "a"
  - id: 2
    name: "b"

Final JSON

{
  "products": [
    {
      "id": 1,
      "name": "a"
    },
    {
      "id": 1,
      "name": "b"
    }
  ]
}