CODEWITHSUNDEEP
javascript
Coyolero
Coyolero

Reputation: 2433

Eliminate duplicates of several arrays

I have 3 arrays:

array1 = [ 'A', 'B', 'A', 'B']
array2 = [   5,   5,   7,   5]
array3 = [true,true,true,true]

I was wondering if there is any easy way (maybe with lodash) to eliminate the duplicates and end with this:

array1 = [ 'A', 'B', 'A']
array2 = [   5,   5,   7]
array3 = [true,true,true]

I know I can do a function and compare the previous value, but is there a more clever way to do it?

Update Please note that I don't need to eliminate the duplicates of each array. What I looking is a way to eliminate the duplicates "vertically"

Update 2 Please note that each "column" is a record. record1 = ['A',5,true] record2 = ['B',5,true] record3 = ['A',7,true] record1 = ['B',5,true]

Upvotes: 2

Views: 136

Answers (6)

A. L
A. L

Reputation: 12639

One method I can think of is using an object to keep track, which will also coincidentally remove any duplicates as keys have to be unique. The only thing is I can think of how to extract it back into an array for now. I will think about it tomorrow.

This utilizes jquery for deep cloning. If you want it only in vanilla javascript, you could probably just implement a deep clone function.

var array1 = [ 'A', 'B', 'A', 'B'];
var array2 = [   5,   5,   7,   5];
var array3 = [true,true,true,true];

all_arrays = [array1, array2, array3];

let obj = {};
for (let i = 0; i < all_arrays[0].length; i++)
{
	let new_obj = recursive_objects(all_arrays, 0, i)
  $.extend(true, obj, new_obj);
}

console.log(obj);

function return_array(array, temp_obj)
{
	let keys = Object.keys(temp_obj);
  for (let key of keys)
  {
  	
  }
}


function recursive_objects(arrays, arrays_index, index)
{
  let obj = {}
	
  if (arrays_index < arrays.length)
  {
    obj[arrays[arrays_index][index]] = recursive_objects(arrays, ++arrays_index, index);  
  }

  return obj;
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

Upvotes: 0

Jordan Running
Jordan Running

Reputation: 106027

TL;DR

const records = array1.map((a, i) => [a, array2[i], array3[i]]);

const index = {};
records.filter(column => {
  const key = JSON.stringify(column);
  return key in index ? false : index[key] = true;
});

Huh?

There are a lot of ways to solve this, with varying degrees of efficiency, and the best solution will depend on the size of your data. A simple but naΓ―ve solution iterates over each "column" and checks all of the preceding columns for equality. It looks like this:

const array1 = [ 'A', 'B', 'A', 'B'];
const array2 = [   5,   5,   7,   5];
const array3 = [true,true,true,true];

const newArray1 = array1.slice(0,1); // column 0 is never duplicate
const newArray2 = array2.slice(0,1);
const newArray3 = array3.slice(0,1);

// loop over columns starting with index 1
outer: for (let i = 1; i < array1.length; i++) {
  const a = array1[i];
  const b = array2[i];
  const c = array3[i];
  
  // check all preceding columns for equality
  for (let j = 0; j < i; j++) {
    if (a === array1[j] && b === array2[j] && c === array3[j]) {
      // duplicate; continue at top of outer loop
      continue outer;
    }
  }

  // not a duplicate; add to new arrays
  newArray1.push(a);
  newArray2.push(b);
  newArray3.push(c);
}

console.log(newArray1);
console.log(newArray2);
console.log(newArray3);
.as-console-wrapper{min-height:100%}

As you can see, we have to check each row within each column for equality, every time. If you're curious, the complexity of this is 𝑂(𝑛(𝑛+1)/2) (technically 𝑂(π‘šπ‘›(𝑛+1)/2), where π‘š is 3 for three columns).

For a larger data sets it's advantageous to keep track of values you've already seen in a data structure that's quick to access: A hash, a.k.a. a JavaScript object. Since all of your values are primitive, a quick way to construct a key is JSON.stringify. Some might consider this a "hack"β€”and it's important to note that it will fail with values that can't be represented in JSON, e.g. Infinity or NaNβ€”but it's a fast and easy one with data this simple.

const array1 = ['A', 'B', 'A', 'B'];
const array2 = [5, 5, 7, 5];
const array3 = [true, true, true, true];

const newArray1 = [];
const newArray2 = [];
const newArray3 = [];

const index = {};

for (let i = 0; i < array1.length; i++) {
  const a = array1[i];
  const b = array2[i];
  const c = array3[i];
  const key = JSON.stringify([a,b,c]);
  
  if (key in index) {
    // duplicate; skip to top of loop
    continue;
  }
  
  // not a duplicate; record in index and add to new arrays
  index[key] = true;
  newArray1.push(a);
  newArray2.push(b);
  newArray3.push(c);
}

console.log(newArray1);
console.log(newArray2);
console.log(newArray3);
.as-console-wrapper{min-height:100%}

The complexity of this is 𝑂(𝑛), or maybe 𝑂(2π‘šπ‘›) where π‘š, again, is 3 for three columns, and the 2 is another π‘š to very roughly account for the cost of JSON.stringify. (Figuring out the cost of hash access is left as an exercise for the pedants among us; I'm content to call it 𝑂(1).)

That's still pretty verbose. Part of the reason is that using three different variables for the dataβ€”which is really a single "table"β€”leads to a lot of repetition. We can preprocess the data to make it easier to deal with. Once it's "transposed" into a single two-dimensional array, we can use Array.prototype.filter with the key technique from above, for some very terse code:

const array1 = ['A', 'B', 'A', 'B'];
const array2 = [5, 5, 7, 5];
const array3 = [true, true, true, true];

// turn "columns" into "rows" of a 2D array
const records = array1.map((a, i) => [a, array2[i], array3[i]]);

const index = {};
const newData = records.filter(column => {
  const key = JSON.stringify(column);
  return key in index ? false : index[key] = true;
});

console.log(newData);
.as-console-wrapper{min-height:100%}

Of course, pre-processing isn't free, so this code isn't any more performant than the more verbose version; you'll have to decide how important that is to you. If you want you can now extract the columns from newData into three variables (newData.forEach(([a,b,c]) => { newArray1.push(a); newArray2.push(b); /* ... */ })), but for many purposes the "transposed" 2D array will be easier to work with.

Upvotes: 4

georg
georg

Reputation: 214949

You're going to need a couple of helper functions (lodash provides them also):

let zip = (...arys) => arys[0].map((_, i) => arys.map(a => a[i]));
let uniq = (ary, key) => uniq2(ary, ary.map(key), new Set);
let uniq2 = (ary, keys, set) => ary.filter((_, i) => !set.has(keys[i]) && set.add(keys[i]))

// test

var array1 = ['A', 'B', 'A', 'B'];
var array2 = [5, 5, 7, 5];
var array3 = [true, true, true, true];

var [x, y, z] = zip(
    ...uniq(
        zip(array1, array2, array3),
        JSON.stringify
    )
);

console.log(x, y, z)

Upvotes: 1

Nenad Vracar
Nenad Vracar

Reputation: 122027

You need to find duplicate elements with same indexes in all arrays and then filter out those elements.

var array1 = ['A', 'B', 'A', 'B'],
  array2 = [5, 5, 7, 5],
  array3 = [true, true, true, true];

var dupes = []
var arrays = [array1, array2, array3];

arrays.forEach(function(arr, i) {
  arr.forEach((e, j) => !this[e] ? this[e] = true : dupes[i] = (dupes[i] || []).concat(j))
}, {})

var index = dupes[0].filter(e => dupes.every(a => a.includes(e)))
var result = arrays.map(e => e.filter((a, i) => !index.includes(i)))

console.log(result)

Upvotes: 1

Mathiasfc
Mathiasfc

Reputation: 1697

Another way, with filter():

array1 = ['A', 'B', 'A', 'B'];
array2 = [5, 5, 7, 5];
array3 = [true, true, true, true];

uniqueArray1 = array1.filter(function(item, pos) {
  return array1.indexOf(item) == pos;
})

uniqueArray2 = array2.filter(function(item, pos) {
  return array2.indexOf(item) == pos;
})

uniqueArray3 = array3.filter(function(item, pos) {
  return array3.indexOf(item) == pos;
})

console.log(uniqueArray1);
console.log(uniqueArray2);
console.log(uniqueArray3);

Upvotes: 0

Geeky
Geeky

Reputation: 7488

You can use ES6 Set https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set

Set -> lets you store unique values of any type, whether primitive values or object references.

and then convert back to an array

check this snippet

const array1 = ['A','B','A','B']
const array2 = [5,5,7,5]
const array3 = [true,true,true,true]

const uniqA1= new Set(array1)
const uniqA2= new Set(array2)
const uniqA3= new Set(array3)

console.log(Array.from(uniqA1))
console.log(Array.from(uniqA2))
console.log(Array.from(uniqA3))

Hope it helps

Upvotes: 2

Related Questions