CODEWITHSUNDEEP
mysqlquery-optimization
GoldenTabby
GoldenTabby

Reputation: 220

Getting duplicate rows by several columns in MySQL

I'm trying to search duplicate rows by several columns in large table (near 18 000 rows). Problem is that queries take a lot of time, I tried this:

 SELECT * FROM table_name a, table_name b 
 WHERE a.col1 = b.col1 
 AND a.col2 = b.col2 
 AND a.col3 = b.col3
 AND a.col4 = b.col4 
 AND a.id <> b.id

and this:

SELECT *
FROM table_name 
WHERE col1 IN (
    SELECT col1 
    FROM table_name 
    GROUP BY col1 
    HAVING count(col1) > 1
    )
AND col2 IN (
    SELECT col2
    FROM table_name 
    GROUP BY col2
    HAVING count(col2) > 1
    )
AND col3 IN (
    SELECT col3
    FROM table_name 
    GROUP BY col3
    HAVING count(col3) > 1
    )
AND col4 IN (
    SELECT col4
    FROM table_name 
    GROUP BY col4
    HAVING count(col4) > 1
    )

they both work, but too slow. Any ideas?

Upvotes: 0

Views: 126

Answers (1)

Siphon
Siphon

Reputation: 1061

You can try using one joint GROUP BY statement like:

SELECT * FROM table_name
    GROUP BY col1, col2, col3, col4
    HAVING count(*) > 1

At the very least, it will look cleaner.

EDIT

To return all results as a sub-set for the previous column:

SELECT *
FROM table_name 
WHERE col4 IN (
  SELECT col4
  FROM table_name 
  WHERE col3 IN (
    SELECT col3
    FROM table_name 
    WHERE col2 IN (
      SELECT col2
      FROM table_name
      WHERE col1 IN (
        SELECT col1
        FROM table_name
        GROUP BY col1
        HAVING count(col1) > 1
        )
      )
    )

This, in concept, should give you all results in a faster execution time.

Upvotes: 1

Related Questions