CODEWITHSUNDEEP
mysqlunique
gunner1095
gunner1095

Reputation: 145

How to change duplicate rows to unique values in mysql?

I have a database table with about 1M records. I need to find all duplicate names in this table and make them unique.

For example...

Id   Name
-----------
1    A
2    A
3    B
4    C
5    C

Should be changed to...

Id   Name
-----------
1    A-1
2    A-2
3    B
4    C-1
5    C-2

Is there an effective way of doing this with a mysql query or procedure?

Thanks in advance!

Upvotes: 5

Views: 3317

Answers (4)

rosh-dev851
rosh-dev851

Reputation: 564

A readable solution

CREATE TEMPORARY TABLE duplicate_names
SELECT name FROM records_table
GROUP BY name
HAVING count(name) > 1;

SET @row_number = 1;
SET @name = '';
CREATE TEMPORARY TABLE duplicates
SELECT 
CASE 
WHEN @name = name THEN @row_number:=@row_number + 1
  ELSE   
    @row_number:=1 
END AS identifier,
@name:=name AS name,id 
FROM
    records_table WHERE name IN (SELECT name FROM duplicate_names)
ORDER BY records_table.name;
    
UPDATE records_table INNER JOIN duplicates 
ON records_table.id = duplicates.id
SET records_table.name = 
CONCAT(duplicates.name,'-',duplicates.identifier);

Upvotes: 0

StackCoder
StackCoder

Reputation: 87

First you should store duplicate Id in a temporary table. 

Drop temporary table if not exist temp;

Create temporary table temp (
Select max(id)'id' from table_x group by Name having count(*)>1
);

Delete from table_x as x,temp as t where x.id = t.id;

Just do this repeatedly... U will get unique rows after that set unique key to name field..

Upvotes: 0

John Ruddell
John Ruddell

Reputation: 25842

this is a little tricky.. I tested on my local host and it does what you want.. let me know if you have any questions. SQL FIDDLE

UPDATE temp t1, 
(
    SELECT 
        id as unique_id, 
        new_name 
    FROM(
        SELECT
          id,
          IF(@ROW = Name, @COUNT, @COUNT := 1),
          CONCAT(Name, ' - ', @COUNT) AS new_name,
          @ROW := Name,
          @COUNT := @COUNT + 1
        FROM temp
        JOIN (SELECT @COUNT := 0, @ROW := "") AS t
        WHERE Name IN(SELECT Name FROM temp
        GROUP BY Name
        HAVING COUNT(Name) > 1)
    ) AS temp_test
) as testing
SET t1.Name = testing.new_name where t1.id = testing.unique_id

Final output looks like this: PICTURE

EDIT: This may work better for performance sake

1. FIRST RUN THIS QUERY

SET SESSION group_concat_max_len = 1000000;  -- longer if needed
SET @query1 := (
SELECT 
    GROUP_CONCAT(DISTINCT unique_name) 
FROM temp
JOIN(
    select Name as unique_name
    FROM temp
    GROUP BY name
    HAVING COUNT(Name) > 1
) as t
);

2. THEN RUN THIS UPDATE

UPDATE temp t1, 
(
    SELECT 
        id as unique_id, 
        new_name 
    FROM(
        SELECT
          id,
          IF(@ROW = Name, @COUNT, @COUNT := 1),
          CONCAT(Name, ' - ', @COUNT) AS new_name,
          @ROW := Name,
          @COUNT := @COUNT + 1
        FROM temp
        JOIN (SELECT @COUNT := 0, @ROW := "") AS t
        WHERE FIND_IN_SET (`name`, @query1)
    ) AS temp_test
) as testing
SET t1.Name = testing.new_name where t1.id = testing.unique_id

I tested this on my local and it works so you should be able to get this to run :)

Upvotes: 3

Mihai Stancu
Mihai Stancu

Reputation: 16107

UPDATE    table_x AS upd
    SET   upd.Name = CONCAT(upd.Name, '-', upd.Id)
    WHERE upd.id IN(
                    SELECT    sel.id
                        FROM  table_x AS sel
                        WHERE sel.Name = upd.Name
                          AND sel.Id != upd.Id
                 );

Upvotes: 0

Related Questions