CODEWITHSUNDEEP
jsontransformationjolt
Chris Bedford
Chris Bedford

Reputation: 2692

is there any way to write a null json transformation (passes through orig document) using Jolt?

You know how XSLT and other XML processing languages support the "null transformation" which passes a document through unmodified ?

I would like to do the same thing for Jolt (a very nice JSON transformation library used in Apache Camel and other places).

I could use JOLT's "insert default" feature and stick some harmless JSON tag and value at the top level of the document.. which is almost what want. But I couldnt' figure out how to pass through the document through JOLT but leave it untouched.

Why do i want to do this you ask ? We are developing a streaming data pipeline and I have to validate incoming strings as valid JSON... Jolt does that for me for free, but in some cases I don't want to monkey with the document. So, I want to use JOLT as a step in the pipeline, but (in some cases) have it do nothing to the input JSSON doc.

Upvotes: 0

Views: 1120

Answers (2)

Alex Block
Alex Block

Reputation: 21

Another option is to create a custom transformer.

package com.example;
public class NullTransform implements Transform{

    @Override
    public Object transform(Object input) {
        return input;
    }

}

then reference it from the chainr jolt as below

[
  {
  "operation": "com.example.NullTransform"
  }
]

You'll still incur the deserializaion/serialization overhead but no other code is run.

Upvotes: 2

Milo S
Milo S

Reputation: 4586

OOTB Jolt contains 5 "operations" that can be applied to the input hydrated Json. 4 of those (default, remove, sort, cardinality) are mutation operations / modify the supplied hydrated Json. I you gave those 4 an empty "spec", they would do nothing, and your data would "pass thru".

The "shift" operation does not mutate the input it is given. Instead it "copies" data from the "input" to a new "output" map/list. If you don't give "shift" a spec, then it copies nothing across.

Thus, from you question, it sounds like you are talking about "shift". With shift you have to explicitly pass thru all the things you "want to keep".

Depending on your data this may be terrible or easy, as you can have shift copy very large chunks of data across.

Example, for the "inception" example on the jolt demo site. http://jolt-demo.appspot.com/#inception

This Spec basically passes thru the input, coping the whole nested map that is "rating" thru to the output.

[
  {
    "operation": "shift",
    "spec": {
      "rating": "rating"
    }
  }
]

It can be generalized with wildcards to : Spec

[
  {
    "operation": "shift",
    "spec": {
      "*": "&"
    }
  }
]

Upvotes: 0

Related Questions