Jan 26, 2021

Jolt transform a default value dependent on another field

Jolt is a JSON to JSON transformation library where the transform is defined in JSON

Sometimes you need to add a default value along with a data transformation. The straight forward way to do this is to define a “shift” operation to handle the data movement and a “default” operation to handle the default value. For example if I want to copy the “state” field into the “value” field of a “region” object with a “type” of “STATE” the Jolt transform would look like this

Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": "region.value"
    }
  },
  {
    "operation": "default",
    "spec": {
      "region": {
        "type": "STATE"
      }
    }
  }
]
Input
{
  "state": "Nebraska"
}
Output
{
  "region": {
    "value": "Nebraska",
    "type": "STATE"
  }
}

This works, however there is an issue if we don’t send a state value the default still gets applied. So our output looks like this.

Blank State Output
{
  "region": {
    "type": "STATE"
  }
}

Fortunately we can solve this issue by defining a block of transform for the “state” field instead of just a just mapping to “region.value”. First we use “@(1, state)” –go up a level and get the state field– to map to the “region.value” and then we use “#STATE” –literal string STATE– to map to “region.type”

Fixed Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": {
        "@(1,state)": "region.value",
        "#STATE": "region.type"
      }
    }
  }
]

This will solve our problem and only apply a type if there is a state field. In fact we could take this a step further and handle a province field as well.

State/Providence Transform
[
  {
    "operation": "shift",
    "spec": {
      "state": {
        "@(1,state)": "region.value",
        "#STATE": "region.type"
      },
      "province": {
        "@(1,province)": "region.value",
        "#PROVINCE": "region.type"
      }
    }
  }
]

About the Author

Scott Bock profile.

Scott Bock

Principal Technologist

Scott is a Senior Software Engineer with over 12 years of experience using Java, and 5 years experience in technical leadership positions. His strengths include troubleshooting and problem solving abilities, excellent repertoire with customers and management, and verbal and written communication. He develops code across the entire technology stack including database, application, and user interface.

One thought on “Jolt transform a default value dependent on another field

  1. Scott Bock says:

    See full code example with test at https://github.com/scottbock/Jolt

Leave a Reply

Your email address will not be published.

Related Blog Posts
Building Better Data Visualization Experiences: Part 1 of 2
Through direct experience with data scientists, business analysts, lab technicians, as well as other UX professionals, I have found that we need a better understanding of the people who will be using our data visualization products in order to build them. Creating a product utilizing data with the goal of providing insight is fundamentally different from a typical user-centric web experience, although traditional UX process methods can help.
Kafka Schema Evolution With Java Spring Boot and Protobuf
In this blog I will be demonstrating Kafka schema evolution with Java, Spring Boot and Protobuf.  This app is for tutorial purposes, so there will be instances where a refactor could happen. I tried to […]
Redis Bitmaps: Storing state in small places
Redis is a popular open source in-memory data store that supports all kinds of abstract data structures. In this post and in an accompanying example Java project, I am going to explore two great use […]
Let’s build a WordPress & Kernel updated AMI with Packer
First, let’s start with What is an AMI? An Amazon Machine Image (AMI) is a master image for the creation of virtual servers in an AWS environment. The machine images are like templates that are configured with […]