Rahul Varma
Reputation: 550
convert complex nested json array using jolt expression in nifi
I have a complex Json where i wanted to flatten the Json using Jolt or any other ways to flatten in Nifi is also fine can anyone please help to flatten the below JSON
{
"veread": "ndjdjjywuieopppqpppwaghhzuii",
"debittanceInformation": [
{
"internalDocumentInformation": [
{
"number": "8",
"lineDetails": [
{
"identifications": [
{
"number": "44",
"model": {
"code": "xdd",
"proprietary": "ddy"
},
"relatedDate": "3/4"
}
],
"quantity": 136,
"unitOfMeasure": "each",
"money": {
"duePayablemoney": 44,
"discountAppliedmoneys": [
{
"model": {
"code": "dddddd",
"proprietary": "rddddy"
},
"money": 216.2
}
],
"valuemoneys": [
{
"model": {
"code": "ddddd",
"proprietary": "dddd"
},
"money": 391.41
}
],
"adjustmentmoneyAndReasons": [
{
"money": 824.54,
"reason": "dd"
}
],
"debittedmoney": 660.7,
"creditNotemoney": 985.63
}
}
],
"internalDocumentmoney": {
"duePayablemoney": 567.45,
"discountAppliedmoneys": [
{
"model": {
"code": "DD",
"proprietary": "EE"
},
"money": 5.64
}
],
"valuemoneys": [
{
"model": {
"code": "z",
"proprietary": "a"
},
"money": 145
}
],
"adjustmentmoneyAndReasons": [
{
"money": 678,
"reason": "tyuiop"
}
],
"debittedmoney": 123,
"creditNotemoney": 456
}
}
],
"supplierReferenceInformation": {
"model": {
"code": "x",
"proprietary": "y"
},
"issuer": "issuer",
"reference": "hfhjfdozkdfl;fdko;k"
},
"billr": {
"name": "jdjaiofjiosJOfjaiodjoiwaj",
"companyIdentifications": [
{
"identification": "74747435438",
"model": "jdidfjdsjf"
}
]
},
"bille": {
"name": "djhsaijdwjadfja;dddF",
"companyIdentifications": [
{
"identification": "dddddd",
"model": "kksjdadawx"
}
]
},
"additionaldebittanceInformation": [
"dd",
"xnjasndfjafnlkNDflkhlfkacmakjhfncasklnf"
]
}
],
"chargesInformation": []
}
The above is My Json which I need to flatten in to simple JSON so that I can process it in Hive . i have tried using Flatten JSON processor but it is not working so now I an trying using jolt spec can any one please guide me with above issue using either Jolt transformation or in Nifi.
my expected output should be as below
{
"veread": "ndjdjjywuieopppqpppwaghhzuii",
"ddddwww_internalDocumentInformation_number": "8",
"ddddwww_internalDocumentInformation_lineDetails_identifications_number": "1025",
"ddddwww_internalDocumentInformation_lineDetails_identifications_model.code": "x",
"ddddwww_internalDocumentInformation_lineDetails_identifications_model.proprietary": "y",
"ddddwww_internalDocumentInformation_lineDetails_identifications_relatedDate": "3/23",
"ddddwww_internalDocumentInformation_lineDetails_quantity": 136,
"ddddwww_internalDocumentInformation_lineDetails_unitOfMeasure": "each",
"ddddwww_internalDocumentInformation_lineDetails_money.duePayablemoney": 957.86,
"ddddwww_internalDocumentInformation_lineDetails_money.discountAppliedmoneys_model.code": "dvbnqwe",
"ddddwww_internalDocumentInformation_lineDetails_money.discountAppliedmoneys_model.proprietary": "rqazloy",
"ddddwww_internalDocumentInformation_lineDetails_money.discountAppliedmoneys_money": 216.2,
"ddddwww_internalDocumentInformation_lineDetails_money.valuemoneys_model.code": "YUIO",
"ddddwww_internalDocumentInformation_lineDetails_money.valuemoneys_model.proprietary": "BHRTY",
"ddddwww_internalDocumentInformation_lineDetails_money.valuemoneys_money": 391.41,
"ddddwww_internalDocumentInformation_lineDetails_money.adjustmentmoneyAndReasons_money": 824.54,
"ddddwww_internalDocumentInformation_lineDetails_money.adjustmentmoneyAndReasons_reason": "BONUS",
"ddddwww_internalDocumentInformation_lineDetails_money.debittedmoney": 660.7,
"ddddwww_internalDocumentInformation_lineDetails_money.creditNotemoney": 985.63,
"ddddwww_internalDocumentInformation_internalDocumentmoney.duePayablemoney": 567.45,
"ddddwww_internalDocumentInformation_internalDocumentmoney.discountAppliedmoneys_model.code": "DD",
"ddddwww_internalDocumentInformation_internalDocumentmoney.discountAppliedmoneys_model.proprietary": "EE",
"ddddwww_internalDocumentInformation_internalDocumentmoney.discountAppliedmoneys_money": 5.64,
"ddddwww_internalDocumentInformation_internalDocumentmoney.valuemoneys_model.code": "z",
"ddddwww_internalDocumentInformation_internalDocumentmoney.valuemoneys_model.proprietary": "a",
"ddddwww_internalDocumentInformation_internalDocumentmoney.valuemoneys_money": 145,
"ddddwww_internalDocumentInformation_internalDocumentmoney.adjustmentmoneyAndReasons_money": 678,
"ddddwww_internalDocumentInformation_internalDocumentmoney.adjustmentmoneyAndReasons_reason": "tyuiop",
"ddddwww_internalDocumentInformation_internalDocumentmoney.debittedmoney": 123,
"ddddwww_internalDocumentInformation_internalDocumentmoney.creditNotemoney": 456,
"ddddwww_supplierReferenceInformation.model.code": "x",
"ddddwww_supplierReferenceInformation.model.proprietary": "y",
"ddddwww_supplierReferenceInformation.issuer": "issuer",
"ddddwww_supplierReferenceInformation.reference": "hfhjfdozkdfl;fdko;k",
"ddddwww_billr.name": "jdjaiofjiosJOfjaiodjoiwaj",
"ddddwww_billr.companyIdentifications_identification": "74747435438",
"ddddwww_billr.companyIdentifications_model": "jdidfjdsjf",
"ddddwww_bille.name": "djhsaijdwjadfja;lMEjknEAKDLJOJRF",
"ddddwww_bille.companyIdentifications_identification": "mkskfsajdwiojdijRUWa",
"ddddwww_bille.companyIdentifications_model": "kksjdadawx",
"ddddwww_additionalddddwww[0]": "JDSJAHDJKLJHRFQWJESJqiorwjd",
"ddddwww_additionalddddwww[1]": "xnjasndfjafnlkNDflkhlfkacmakjhfncasklnf",
"chargesInformation": []
}
I have updated the request
Upvotes: 0
Views: 372
Answers (3)
LeoTaylor
Reputation: 11
The Java code of flattening JSON records of an indefinite number of levels to a two-dimensional table is extremely long.
You can use SPL, an open-source Java package, to get this done. It is convenient and four lines of code are sufficient: img
SPL offers JDBC driver to be invoked by Java. Just store the above SPL script as flattenjson.splx and invoke it in Java as you call a stored procedure:
…
Class.forName("com.esproc.jdbc.InternalDriver");
con= DriverManager.getConnection("jdbc:esproc:local://");
st = con.prepareCall("call flattenjson()");
st.execute();
…
Upvotes: 1
Barbaros Özhan
Reputation: 65105
You can use the shift transformation below in a JoltTransformJSON processor
[
{
"operation": "shift",
"spec": {
"*": "&",
"debittanceInformation": {
"*": {
"internalDocumentInformation": {
"*": {
"number": "&4_&2_&",
"lineDetails": {
"*": {
"identifications": {
"*": {
"*": {
"@": "&9_&7_&5_&3_&"
},
"model": {
"*": {
"@": "&10_&8_&6_&4_&2\\.&"
}
}
}
},
"quantity": "&6_&4_&2_&",
"unitOfMeasure": "&6_&4_&2_&",
"money": {
"*": {
"@": "&8_&6_&4_&2\\.&"
},
"discountAppliedmoneys": {
"*": {
"model": {
"*": {
"@": "&11_&9_&7_&5\\.&4_&2\\.&"
}
},
"*": "&9_&7_&5_&3\\.&2_&"
}
},
"valuemoneys": {
"*": {
"model": {
"*": {
"@": "&11_&9_&7_&5\\.&4_&2\\.&"
}
},
"*": "&9_&7_&5_&3\\.&2_&"
}
},
"adjustmentmoneyAndReasons": {
"*": {
"*": "&9_&7_&5_&3\\.&2_&"
}
}
}
}
},
"internalDocumentmoney": {
"*": {
"@": "&6_&4_&2\\.&"
},
"discountAppliedmoneys": {
"*": {
"model": {
"*": "&8_&6_&4_&3_&1\\.&"
},
"*": "&7_&5_&3\\.&2_&"
}
},
"valuemoneys": {
"*": {
"model": {
"*": "&8_&6_&4\\.&3_&1\\.&"
},
"*": "&7_&5_&3\\.&2_&"
}
},
"adjustmentmoneyAndReasons": {
"*": {
"*": "&7_&5_&3\\.&2_&"
}
}
}
}
},
"supplierReferenceInformation": {
"model": {
"*": "&4_&2\\.&"
},
"*": "&3_&1\\.&"
},
"bill*": {
"companyIdentifications": {
"*": {
"*": "&5_&3\\.&2_&"
}
},
"*": "&3_&1\\.&"
},
"additionaldebittanceInformation": {
"*": "&3_&1\\[&\\]"
}
}
}
}
}
]
Upvotes: 1
tonykoval
Reputation: 1230
Use FlattenJson processor with properties:
Separator:_Flatten Mode:normal
Output json:
{
"transactionId" : "ndjdjjywuieopppqpppwaghhzuii",
"debittanceInformation[0]_internalDocumentInformation[0]_number" : "8",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_identifications[0]_number" : "1025",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_identifications[0]_model_code" : "x",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_identifications[0]_model_proprietary" : "y",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_identifications[0]_relatedDate" : "3\/23",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_quantity" : 136,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_unitOfMeasure" : "each",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_duePayablemoney" : 957.86,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_discountAppliedmoneys[0]_model_code" : "dvbnqwe",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_discountAppliedmoneys[0]_model_proprietary" : "rqazloy",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_discountAppliedmoneys[0]_money" : 216.2,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_valuemoneys[0]_model_code" : "YUIO",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_valuemoneys[0]_model_proprietary" : "BHRTY",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_valuemoneys[0]_money" : 391.41,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_adjustmentmoneyAndReasons[0]_money" : 824.54,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_adjustmentmoneyAndReasons[0]_reason" : "BONUS",
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_debittedmoney" : 660.7,
"debittanceInformation[0]_internalDocumentInformation[0]_lineDetails[0]_money_creditNotemoney" : 985.63,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_duePayablemoney" : 567.45,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_discountAppliedmoneys[0]_model_code" : "DD",
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_discountAppliedmoneys[0]_model_proprietary" : "EE",
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_discountAppliedmoneys[0]_money" : 5.64,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_valuemoneys[0]_model_code" : "z",
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_valuemoneys[0]_model_proprietary" : "a",
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_valuemoneys[0]_money" : 145,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_adjustmentmoneyAndReasons[0]_money" : 678,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_adjustmentmoneyAndReasons[0]_reason" : "tyuiop",
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_debittedmoney" : 123,
"debittanceInformation[0]_internalDocumentInformation[0]_internalDocumentmoney_creditNotemoney" : 456,
"debittanceInformation[0]_supplierReferenceInformation_model_code" : "x",
"debittanceInformation[0]_supplierReferenceInformation_model_proprietary" : "y",
"debittanceInformation[0]_supplierReferenceInformation_issuer" : "issuer",
"debittanceInformation[0]_supplierReferenceInformation_reference" : "hfhjfdozkdfl;fdko;k",
"debittanceInformation[0]_billr_name" : "jdjaiofjiosJOfjaiodjoiwaj",
"debittanceInformation[0]_billr_companyIdentifications[0]_identification" : "74747435438",
"debittanceInformation[0]_billr_companyIdentifications[0]_model" : "jdidfjdsjf",
"debittanceInformation[0]_bille_name" : "djhsaijdwjadfja;lMEjknEAKDLJOJRF",
"debittanceInformation[0]_bille_companyIdentifications[0]_identification" : "mkskfsajdwiojdijRUWa",
"debittanceInformation[0]_bille_companyIdentifications[0]_model" : "kksjdadawx",
"debittanceInformation[0]_additionaldebittanceInformation[0]" : "JDSJAHDJKLJHRFQWJESJqiorwjd",
"debittanceInformation[0]_additionaldebittanceInformation[1]" : "xnjasndfjafnlkNDflkhlfkacmakjhfncasklnf",
"chargesInformation" : [ ]
}
Upvotes: 1
Related Questions
- How do I use Jolt to json array to nested json?
- Convert Flat JSON to Nested JSON with arrays
- Converting flat json to nested json for multiple payloads using jolt transform
- Convert Flat json to Nested Json with multiple arrays using Jolt transform
- transformation of nested array in array by jolt
- Convert flat json to nested json using jolt transform
- JOLT concat values from nested array (Apache NiFi)
- Jolt Transform on Nested JSON Object Array
- Jolt Transformation Nested Json to Flat Json
- How to transform a flat JSON to nested JSON using NiFi