CODEWITHSUNDEEP
jsonnet
Aleš
Aleš

Reputation: 9028

Remove duplicates from an array of objects in jsonnet

I have an array of objects, I would like to remove duplicates. My array has a common field name that I would like to use for deduplication.

I am trying to convert the array to a map and then from map back to array but map conversions gives me an error duplicate field name: "a":

local arr = [ 
    { "name": "a", "value": 1234},
    { "name": "b", "value": 555},
    { "name": "c", "value": 0}, 
    { "name": "a", "value": 1234} 
];
local map = { [x.name] : x  for x in arr };

Desired output: 

[ 
      { "name": "a", "value": 1234},
      { "name": "b", "value": 555}, 
      { "name": "c", "value": 0} 
]

Upvotes: 1

Views: 3587

Answers (2)

Aleš
Aleš

Reputation: 9028

As @seh pointed out in ksonnet channel, the latest jsonnet release now allows to use std.set() on objects.

 local arr = [
    { name: "a", value: 1234 },
    { name: "b", value: 555 },
    { name: "c", value: 0 },
    { name: "a", value: 1234 },
  ];
 std.set(arr, function(o) o.name)

The std.set() header is documented in jsonnet's std lib implementation.

Upvotes: 2

jjo
jjo

Reputation: 3020

Ignoring original sort order

You can implement it by replacing the comprehension with std.foldl(), do note though the ordering issue:

local arr = [
  { name: "a", value: 4321 },
  { name: "b", value: 555 },
  { name: "c", value: 0 },
  { name: "a", value: 1234 },
];

// Use foldl to iterate from array, can't use comprehension because of dup fields
local map = std.foldl(function(x, y) x { [y.name]: y }, arr, {});

// Re-convert map to array, note that it'll not respect original order
// but fields' (ie 'name' key)
[ map[x] for x in std.objectFields(map)]

Keeping original sort order

If you need to keep original sort order in output array, you can then add an _idx field to use in a final sort():

local arr = [
  { name: "a", value: 4321 },
  { name: "b", value: 555 },
  { name: "c", value: 0 },
  { name: "a", value: 1234 },
];

// Overload array elements with there index (`_idx` field)
local idxArray = std.mapWithIndex(function(i, x) x { _idx:: i }, arr);

// Use foldl to iterate from array, can't use comprehension because of dup fields
local map = std.foldl(function(x, y) x { [y.name]: y }, idxArray, {});

// Re-convert map to array, it'll keep original order via added _idx field
std.sort([map[x] for x in std.objectFields(map)], function(e) e._idx)

Upvotes: 1

Related Questions