CODEWITHSUNDEEP
mysqlsqlindexingduplicates
clemlatz
clemlatz

Reputation: 8063

Find duplicates on multiple columns in a SQL table in order to create UNIQUE index

I created a index called abc on a table called table with a PRIMARY KEY called id and three others INT columns a, b and c that can be NULL.

Now I need this index to be UNIQUE, so I tried :

ALTER TABLE table DROP INDEX abc, ADD UNIQUE abc (a, b, c);

But I have duplicates, so MySQL answers :

#1062 - Duplicate entry '1-2-3' for key 'abc'

I probably have a lot of those duplicates, so I'm looking for an easy way to search & destroy them all. My first guess has been : 

SELECT * FROM table GROUP BY abc

But sadly it seems that you can't group by indexes.

Is there an easy way to find those duplicates, keep one line of each duplicate and delete the others ?

EDIT :

Upvotes: 1

Views: 2367

Answers (4)

Joachim Isaksson
Joachim Isaksson

Reputation: 181097

No need to eliminate duplicates first, just use the IGNORE option for ALTER TABLE to do exactly what you want;

ALTER IGNORE TABLE table DROP INDEX abc, ADD UNIQUE abc (a, b, c);

An SQLfiddle to test with.

If IGNORE is not specified, the copy is aborted and rolled back if duplicate-key errors occur. If IGNORE is specified, only one row is used of rows with duplicates on a unique key. The other conflicting rows are deleted.

...and always remember to back up your data before running potentially destructive SQL from random people on the Internet.

Upvotes: 2

neiha
neiha

Reputation: 171

I'm guessing you have several records that aren't in this situation.

To avoid losing data

CREATE table duplicates SELECT MIN(id) as id, a, b, c,  COUNT(1) as nduplicates 
FROM yourtable
GROUP BY a,b,c 
HAVING COUNT(1)>1;

UPDATE yourtable t, duplicates d 
SET t.a='toDelete(or some value that you can easy identify from the rest)' 
WHERE d.a=t.a and d.b=t.b and d.c=t.c 
and d.id!=t.id;

DELETE FROM yourtable WHERE a='toDelete';

and then drop duplicates table.

Upvotes: 0

Charles Bretana
Charles Bretana

Reputation: 146603

try this to find dupes

Select a, b, c
From table 
group By a, b, c
Having Count(*) > 1

If there is already a unique key column (say its pkColumn) on this table, you can do this to delete extra dupes.

Delete table
From table t
Where pkColumn <>
   (Select Min(pkColumn)
    From table
    where a = t.a
      and b = t.b
      and c = t.c)

Upvotes: 1

StanislavL
StanislavL

Reputation: 57421

SELECT a,b,c
FROM table 
GROUP BY a,b,c
HAVING count(*)>1

Upvotes: 1

Related Questions