Dataweave - Globally Unique Array Elements

Question

Hi I have a dataweave question, I have the sample input and I want to remove the 2nd object in the 2nd array. In short, records should be unique globally. I tried distinct and filter, however filter remove both, distinct doesn't change anything.

Input

[
  {
    "id": 1,
    "books": [
      {
        "bookid": "1234",
        "title": "C#"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  },
  {
    "id": 2,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  }
]

Output

[
  {
    "id": 1,
    "books": [
      {
        "bookid": "1234",
        "title": "C#"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  },
  {
    "id": 2,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      }
    ]
  }
]

Answer 1

I made an assumption that we would be picking the first unique title and discard the remaining. Below I broke down your problem into 4 steps. First, I am adding id with the books, then remove duplicates using distinctBy, then group filtered list by id and finally, write your desired output. I am curious to see other solutions :)

%dw 2.0
output application/json
var globalList =  payload flatMap ((item) -> (
    item.books map (b) -> {id: item.id,(b)}))
    distinctBy ((item) -> item.bookid)
    groupBy ((item) -> item.id) 
---
  keysOf(globalList) map ((item) ->
    {
        id: item,
        books: globalList[item] map ((book) -> book - 'id')
    
} )

Answer 2

Very similar to Imtiyaz's approach , just that its a bit more elaborated upon.

Input

[
  {
    "id": 1,
    "books": [
      {
        "bookid": "1234",
        "title": "C#"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  },
  {
    "id": 2,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      },
      {
        "bookid": "1234",
        "title": "Java"
      }
    ]
  },
  {
    "id": 3,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      },
      {
        "bookid": "5890",
        "title": "Java"
      }
    ]
  }
]

Script

%dw 2.0
output application/json
var allBookandIds = payload flatMap (value1,index1) -> (
     value1.books map (value,index) -> {
         a: {b: value1.id, (value)}
     }.a 
) 
---
allBookandIds distinctBy($.bookid) groupBy($.b) mapObject {
    id: ($$),
    books: $ map {
        bookid: $.bookid,
        title: $.title
    }
}

Output

{
  "id": "1",
  "books": [
    {
      "bookid": "1234",
      "title": "C#"
    },
    {
      "bookid": "5678",
      "title": "Java"
    }
  ],
  "id": "2",
  "books": [
    {
      "bookid": "4321",
      "title": "VB"
    }
  ],
  "id": "3",
  "books": [
    {
      "bookid": "5890",
      "title": "Java"
    }
  ]
}