CODEWITHSUNDEEP
jsontalend
aymane hmidich
aymane hmidich

Reputation: 23

How to parse a dynamic JSON schema in Talend

I have a Rest API that returns a JSON, but the same field is sometimes returned as an object and sometimes as an array based on available data, I'm using tExtractJsonFields component and it can't handle this change in the schema, let's take the "stage_is" field for example: for one call the API returns this :

{ "stage_id":[ 1, "New" ] }

to get the ID it should be parsed this way stage_id[0]

for another call I would get this :

{ "stage_id":1 }

and now it is parsed like this : stage_id

enter image description here

is there any known ways to handle this ? I'm trying to look for the simplest solution possible before jumping into using something like the tSystem component and use an external tool or a bash script to solve this, thank you guys for your help in advance.

Upvotes: 0

Views: 1464

Answers (2)

Ibrahim Mezouar
Ibrahim Mezouar

Reputation: 4051

Unfortunately, Talend doesn't handle dynamic json or at least I am not aware of any solution using native components.
I suggest you modify your json before parsing it, so that stage_id is always an array, that way you can parse it with stage_id[0], like so:
enter image description here

By placing a tReplace component before tExtractJSONFields, and using this regex

(\"stage_id\"\\s*:\\s*)(\\d+)

And as a remplacement string: $1\\[$2\\]

This will search for occurences of stage_id followed by a number, and surround that number with brackets.
So, { "stage_id":1 } becomes { "stage_id":[1] }
If you get { "stage_id":[1, "New"] } it will not match the regex and won't be modified.

tExtractJSONFields with stage_id[0] will then parse it in both cases.

Upvotes: 2

Power Mouse
Power Mouse

Reputation: 1441

since you have a broken example just giving you an idea.

void Main()
{
    //array
    var j = "{\"stage_id\":[{\"id\":1,\"val\":\"New\"}]}";
    Test res = JsonConvert.DeserializeObject<Test>(j).Dump();
    //one rec
    j = "{\"stage_id\": 1 }";
    res = JsonConvert.DeserializeObject<Test>(j).Dump();
}

public class Test
{
    public object stage_id {get;set;}
}

enter image description here

Upvotes: 0

Related Questions