CODEWITHSUNDEEP
dataweavemulesoft
kimmy
kimmy

Reputation: 27

Mulesoft integration - Optimising grouping of multiple data

Using Dataweave 1.0

There are multiple sources of data coming in to Mule. We merge payload from each system. Record_A, Record_B, Record_C and Record_D.

ID is used as key to find a match and group the response in the expected output.

Problem is that it functionally work but with the current code, it basically loops through all individual record, comparing the condition that causes PERFORMANCE ISSUE when record set is very high. (also the real payload is way bigger than what is in the example but shares same structure).

I am using filter function to filter and find only matching record.

Again, I do get expected output but not performant.

%dw 1.0
%output application/json

%var payload = {
    "PERSON": [
        {
            "ID": "aaaa",
            "firstname": "Homer",
            "lastname": "Simpson"
        },
        {
            "ID": "bbbb",
            "firstname": "Bart",
            "lastname": "Simpson"
        },
        {
            "ID": "cccc",
            "firstname": "Meggie",
            "lastname": "Yolo"
        }
    ],
    "ADDRESS": [
        {
            "ID": "aaaa",
            "address": "1 hello world",
            "postcode": "1234"
        },
        {
            "ID": "bbbb",
            "address": "13 Nuclear Plant",
            "postcode": "3333"
        }
    ],
    "CAR": [
        {
            "ID": "aaaa",
            "brand": "Mercedes",
            "numberplate": "AAA111"
        },
        {
            "ID": "aaaa",
            "brand": "Toyota",
            "numberplate": "BBB322"
        },
        {
            "ID": "bbbb",
            "brand": "Mercedes",
            "numberplate": "ABC123"
        }
    ]
}

---

payload.PERSON map ((item) -> {
    "PERSON": payload.PERSON filter ($.ID contains item.ID) map (items) -> {
        (items)
    },
    ("ADDRESS": payload.ADDRESS filter ($.ID contains item.ID) map (items) -> {
        (items)
    }) when payload.ADDRESS != null,
    ("CAR": payload.CAR filter ($.ID contains item.ID) map (items) -> {
        (items)
    }) when payload.CAR != null
})

Expected output is something like below.

[
  {
    "PERSON": [
      {
        "ID": "aaaa",
        "firstname": "Homer",
        "lastname": "Simpson"
      }
    ],
    "ADDRESS": [
      {
        "ID": "aaaa",
        "address": "1 hello world",
        "postcode": "1234"
      }
    ],
    "CAR": [
      {
        "ID": "aaaa",
        "brand": "Mercedes",
        "numberplate": "AAA111"
      },
      {
        "ID": "aaaa",
        "brand": "Toyota",
        "numberplate": "BBB322"
      }
    ]
  },
  {
    "PERSON": [
      {
        "ID": "bbbb",
        "firstname": "Bart",
        "lastname": "Simpson"
      }
    ],
    "ADDRESS": [
      {
        "ID": "bbbb",
        "address": "13 Nuclear Plant",
        "postcode": "3333"
      }
    ],
    "CAR": [
      {
        "ID": "bbbb",
        "brand": "Mercedes",
        "numberplate": "ABC123"
      }
    ]
  },
  {
    "PERSON": [
      {
        "ID": "cccc",
        "firstname": "Meggie",
        "lastname": "Yolo"
      }
    ],
    "ADDRESS": [

    ],
    "CAR": [

    ]
  }
]

Upvotes: 1

Views: 105

Answers (1)

Salim Khan
Salim Khan

Reputation: 4303

Try this..

%dw 1.0
%output application/json

%var PERSON = payload.PERSON 
                    groupBy $.ID
                    mapObject ((value, key) -> (key) : value)


%var ADDRESS = payload.ADDRESS 
                    groupBy $.ID
                    mapObject ((value, key) -> (key) : value)

%var CAR = payload.CAR 
                    groupBy $.ID
                    mapObject ((value, key) -> (key) : value)                    
---
payload.PERSON map ((item) -> {
    "PERSON":PERSON[item.ID] map ((persons) -> {
        (persons)
    }),
    "ADDRESS": ADDRESS[item.ID] map ((addresses) -> {
        (addresses)
    }) default [],
     "CAR": CAR[item.ID] map ((cars) -> {
        (cars)
    }) default []
})

Upvotes: 4

Related Questions