Jan 26, 2021

Jolt transform a default value dependent on another field

Jolt is a JSON to JSON transformation library where the transform is defined in JSON

Sometimes you need to add a default value along with a data transformation. The straight forward way to do this is to define a “shift” operation to handle the data movement and a “default” operation to handle the default value. For example if I want to copy the “state” field into the “value” field of a “region” object with a “type” of “STATE” the Jolt transform would look like this

Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": "region.value"
    }
  },
  {
    "operation": "default",
    "spec": {
      "region": {
        "type": "STATE"
      }
    }
  }
]
Input
{
  "state": "Nebraska"
}
Output
{
  "region": {
    "value": "Nebraska",
    "type": "STATE"
  }
}

This works, however there is an issue if we don’t send a state value the default still gets applied. So our output looks like this.

Blank State Output
{
  "region": {
    "type": "STATE"
  }
}

Fortunately we can solve this issue by defining a block of transform for the “state” field instead of just a just mapping to “region.value”. First we use “@(1, state)” –go up a level and get the state field– to map to the “region.value” and then we use “#STATE” –literal string STATE– to map to “region.type”

Fixed Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": {
        "@(1,state)": "region.value",
        "#STATE": "region.type"
      }
    }
  }
]

This will solve our problem and only apply a type if there is a state field. In fact we could take this a step further and handle a province field as well.

State/Providence Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": {
        "@(1,state)": "region.value",
        "#STATE": "region.type"
      },
      "province": {
        "@(1,province)": "region.value",
        "#PROVINCE": "region.type"
      }
    }
  }
]

About the Author

Scott Bock profile.

Scott Bock

Principal Technologist

Scott is a Senior Software Engineer with over 12 years of experience using Java, and 5 years experience in technical leadership positions. His strengths include troubleshooting and problem solving abilities, excellent repertoire with customers and management, and verbal and written communication. He develops code across the entire technology stack including database, application, and user interface.

One thought on “Jolt transform a default value dependent on another field

  1. Scott Bock says:

    See full code example with test at https://github.com/scottbock/Jolt

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
Feature Flags in Terraform
Feature flagging any code can be useful to developers but many don’t know how to or even that you can do it in Terraform. Some benefits of Feature Flagging your code You can enable different […]
Infrastructure as Code – The Wrong Way
You are probably familiar with the term “infrastructure as code”. It’s a great concept, and it’s gaining steam in the industry. Unfortunately, just as we had a lot to learn about how to write clean […]
Snowflake CI/CD using Jenkins and Schemachange
CI/CD and Management of Data Warehouses can be a serious challenge. In this blog you will learn how to setup CI/CD for Snowflake using Schemachange, Github, and Jenkins. For access to the code check out […]
How to get your pull requests approved more quickly
TL;DR The fewer reviews necessary, the quicker your PR gets approved. Code reviews serve an essential function on any software codebase. Done right, they help ensure correctness, reliability, and maintainability of code. On many teams, […]