CODEWITHSUNDEEP
javascriptjson
Andrew Kind
Andrew Kind

Reputation: 111

Remove duplicate objects from JSON file in JavaScript

This is the current JSON file:

[{
    "name": "Peter",
    "age": 30,
    "hair color": "brown"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}]

I want to remove the duplicate Steve individual from the list. How can I make a new JSON that checks if the object's name matches and remove any duplicates in JavaScript?

Upvotes: 4

Views: 21650

Answers (6)

Frans Sjöström
Frans Sjöström

Reputation: 465

This was the best solution I could find that makes you able to filter on multiple values in you json object. Solution without _ (module)

array.filter((thing, index, self) =>
    index === self.findIndex((t) => (
    t.place === thing.place && t.name === thing.name // you can add more arguments here to filter more
  ))
)

//Other example
array.filter((thing, index, self) =>
    index === self.findIndex((t) => (
    t.place === thing.place && t.name === thing.name && t.time === thing.time
  ))
)

Upvotes: 0

Shijo Rs
Shijo Rs

Reputation: 157

var data = [{
    "name": "Peter",
    "age": 30,
    "hair color": "brown"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}]
 data = this.data.filter((obj, pos, arr) => {
            return arr.map(mapObj =>
                  mapObj.name).indexOf(obj.name) == pos;
            });
          console.log(data);

Upvotes: 5

Andrew Dunai
Andrew Dunai

Reputation: 3129

Python one-liner from CLI:

cat file_with_duplicates.json | python2 -c 'import sys; import json; sys.stdout.write(json.dumps(reduce(lambda x, y: x + [y] if y not in x else x, json.loads(sys.stdin.read()), [])))' > unique_items.txt

Upvotes: -3

thefourtheye
thefourtheye

Reputation: 239613

You must load the JSON data in to the program and parse that with JSON.parse, like this

var array = JSON.parse(content.toString())

To filter out the repeated names from the array of Objects, we use Array.prototype.filter function. You can store the names in an object, and next time when the same name appears we simply filter it out from the result.

var seenNames = {};

array = array.filter(function(currentObject) {
    if (currentObject.name in seenNames) {
        return false;
    } else {
        seenNames[currentObject.name] = true;
        return true;
    }
});

console.log(array);
# [ { name: 'Peter', age: 30, 'hair color': 'brown' },
#   { name: 'Steve', age: 55, 'hair color': 'blonde' } ]

Upvotes: 10

tymeJV
tymeJV

Reputation: 104785

Loop, check, splice, repeat:

var distinctValues = {};
for (var i = 0; i < data.length; i++) {
    if (distinctValues.hasOwnProperty(data[i].name]) {
        //already has it
        data.splice(i, 1);
        i--;
    } else {
        distinctValues[data[i].name] = true;
    }
}

Upvotes: 1

Timothy Shields
Timothy Shields

Reputation: 79561

Using Underscore.js and the uniq function:

_.uniq(array, false, function (item) { return item.name; })

Upvotes: 2

Related Questions