JavaScript Transform

Use the power of Javascript to modify incoming event object, replace it with a completely new event or produce multiple events based on incoming data. Also, you can use Transform to assign Data Warehouse specific SQL types for object fields, set the destination table name for each event or to skip the event altogether.

Overview

You can use modern javascript language features and built-in functions to transform an incoming event. Jitsu puts incoming events as a global variable: $

Provided javascript must return:

• single object - modified incoming event or completely new object

• array of objects - a single incoming event will result in multiple events in destinations

• null - to skip events from processing

To override the destination table, you need to add a special property named Luden_TABLE_NAME to the resulting events.

To override the destination SQL column type for a specific object field, you need to add an extra property with the special prefix __sql_type_ added to the name of a field to resulting events. E.g.: __sql_type_utc_time: "date" sets SQL type date for column utc_time

NOTE Luden uses Node.js under the hood for JavaScript execution. This requires that node and npm executables are present in $PATH. Furthermore, Jitsu needs node-fetch@2.6.7 and vm2@3.9.9 NPM modules in order to run JavaScript transformations and plugins. These modules are installed in a temporary directory automatically on Jitsu Server startup, but you may set NODE_PATH environment variable with the location of node_modules directory with node-fetch and vm2 installed in order to use pre-installed modules. You can also use our Docker image which is already set up to run JavaScript transformations and plugins.

Server YAML Configuration

Transform is controlled via data_layout.transform_enabled and data_layout.transform properties of the destination object.

Example:

destinations:
  example:
    type: google_analytics
    mode: stream
    data_layout:
      transform_enabled: true
      transform: |-
        const context = $.eventn_ctx || $

        return {
            ...$,
            dl: context.url,
            dh: context.doc_host,
            dp: context.doc_path,
            dt: context.page_title,
        }

Modify Incoming Event

Javascript spread operator allows making a copy of an incoming event while applying some changes in just a few lines of code:

return {...$,
    new_property: $.event_type
}

Add property to user object:

return {...$,
    user: {...$.user, state: "active"}
}

Build New Event

Collect some user properties to the new object:

return {
    properties: [
        {
            property: "email",
            value: $.user?.email
        },
        {
            property: "language",
            value: $.user_language
        }
    ]
}

Put an original event as a string payload:

return {
    "event_type": "POST event",
    "payload": JSON.stringify($)
}

Override Destination Table

Using Javascript spread operator:

return {...$, JITSU_TABLE_NAME: "new_table_name"}

Conventional way:

$.JITSU_TABLE_NAME = "new_table_name"
return $

Override SQL Column Type

Set simple SQL types:

return {...$, 
    event_date: $.utc_time,                               
    __sql_type_event_date: "date",
    event_time: $.utc_time,
    __sql_type_event_time: "time"
}

SQL types with extra parameters:

Some Data Warehouses support extra parameters for column types during table creation. For such cases, Transform uses the following syntax to provide data type and column type separately:

return {...$,
    title: $.page_title,
    __sql_type_title: ["varchar(256)", "varchar(256) encode zstd"]
}