CODEWITHSUNDEEP
phpmysqlsqloptimizationmariadb
Bud Damyanov
Bud Damyanov

Reputation: 31839

Optimise MySQL query, which uses sub-query; MariaDB much faster

I have the following query (with sub-query):

SELECT  ROUND(SUM(cb.points)) AS points
    FROM  clients_bonuses cb
    WHERE  cb.cb_year = 2016
      AND  cb.account_id IN (
        SELECT  CONCAT('85500/',uc.contract)
            FROM  users_contracts uc , users_details ud
            WHERE  ud.id=uc.users_details_id
              AND  uc.platform_id = 1
              AND  ud.id=6  )

which runs perfectly fine (0.2 sec) on my local MySQL server (10.1.9-MariaDB), however, on my production MySQL server (5.5.46-0ubuntu0.14.04.2) it takes 35 sec. to complete.

My local database is exact copy of the production database, hardware configuration differs only by video adapter (local has build-in Intel Graphics).

My question is - what might be the cause of the problem? Can I (and how to) optimize this query?

Result from EXPLAIN the above query (notice the differences in FirstMatch(cb); Using join buffer (flat, BNL join))

(production server):

"id"    "select_type"   "table" "type"  "possible_keys" "key"   "key_len"   "ref"   "rows"  "Extra"
"1" "PRIMARY"   "cb"    "ALL"   \N  \N  \N  \N  "394886"    "Using where"
"2" "DEPENDENT SUBQUERY"    "ud"    "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" "Using index"
"2" "DEPENDENT SUBQUERY"    "uc"    "ALL"   \N  \N  \N  \N  "12243" "Using where"

(local machine):

"id"    "select_type"   "table" "type"  "possible_keys" "key"   "key_len"   "ref"   "rows"  "Extra"
"1" "PRIMARY"   "ud"    "const" "PRIMARY"   "PRIMARY"   "4" "const" "1" "Using index"
"1" "PRIMARY"   "cb"    "ALL"   \N  \N  \N  \N  "394537"    "Using where"
"1" "PRIMARY"   "uc"    "ALL"   \N  \N  \N  \N  "12238" "Using where; FirstMatch(cb); Using join buffer (flat, BNL join)"

Tables (quotes removed from table columns names, because of this editor):

  1. users_contracts

    CREATE TABLE users_contracts ( id INT(11) UNSIGNED NOT NULL AUTO_INCREMENT, users_details_id INT(11) NOT NULL DEFAULT '0', platform_id INT(11) NOT NULL DEFAULT '0', contract VARCHAR(20) NOT NULL DEFAULT '', createdon TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', createdby INT(11) NOT NULL DEFAULT '0', updatedon TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', updatedby INT(11) NOT NULL DEFAULT '0', portfolio_id INT(11) NULL DEFAULT NULL, PRIMARY KEY (id) ) COLLATE='cp1251_general_ci' ENGINE=MyISAM AUTO_INCREMENT=13617;

  2. clients_bonuses

    CREATE TABLE clients_bonuses ( id INT(11) UNSIGNED NOT NULL AUTO_INCREMENT, account_id VARCHAR(255) NOT NULL DEFAULT '', bi_id INT(11) NOT NULL DEFAULT '0', cb_year YEAR NOT NULL DEFAULT '0000', cb_month TINYINT(1) NOT NULL DEFAULT '0', cb_day TINYINT(1) NOT NULL DEFAULT '0', bonus DECIMAL(10,4) NOT NULL DEFAULT '0.0000', amount DECIMAL(20,4) NOT NULL DEFAULT '0.0000', points DECIMAL(20,4) NOT NULL DEFAULT '0.0000', month_lots DECIMAL(10,4) NOT NULL DEFAULT '0.0000', PRIMARY KEY (id) ) COLLATE='cp1251_general_ci' ENGINE=MyISAM AUTO_INCREMENT=395015;

  3. users_details

    CREATE TABLE users_details ( id INT(11) UNSIGNED NOT NULL AUTO_INCREMENT, email VARCHAR(100) NOT NULL DEFAULT '', phone VARCHAR(32) NULL DEFAULT NULL, interest TINYINT(1) NOT NULL DEFAULT '0', contact TINYINT(1) NOT NULL DEFAULT '0', instrument TINYINT(1) NOT NULL DEFAULT '0', instrument1 TINYINT(1) NOT NULL DEFAULT '0', comments TEXT NOT NULL, reminder TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', reminder1 TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', last_accessed TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', dealer INT(11) NOT NULL DEFAULT '0', dealer1 INT(11) NOT NULL DEFAULT '0', real_client TINYINT(1) NOT NULL DEFAULT '0', real_client_meta TINYINT(1) NOT NULL DEFAULT '0', real_client_bgtrader TINYINT(1) NOT NULL DEFAULT '0', real_client_bmpro TINYINT(1) NOT NULL DEFAULT '0', advmails TINYINT(1) NOT NULL DEFAULT '0', analysis_status TINYINT(4) NULL DEFAULT '0', real_client_date TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', real_client_meta_date TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', real_client_bgtrader_date TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', real_client_bmpro_date TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', reg_status TINYINT(1) NOT NULL DEFAULT '0', password VARCHAR(100) NOT NULL DEFAULT '', real_name VARCHAR(100) NOT NULL DEFAULT '', date_of_first_points TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', trader_points INT(11) NOT NULL DEFAULT '0', metatrader_points INT(11) NOT NULL DEFAULT '0', bgtrader_points INT(11) NOT NULL DEFAULT '0', bmpro_points INT(11) NOT NULL DEFAULT '0', vps_status TINYINT(1) NOT NULL DEFAULT '0', vps_username VARCHAR(50) NOT NULL DEFAULT '', vps_password VARCHAR(50) NOT NULL DEFAULT '', vps_start_date TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', metatrader5_points INT(11) NOT NULL DEFAULT '0', mt_points INT(11) NOT NULL DEFAULT '0', reminder_from INT(11) NOT NULL DEFAULT '0', reminder_mt4 DATE NULL DEFAULT NULL, taken_bonus DATE NOT NULL DEFAULT '0000-00-00', taken_bonus_from DATE NOT NULL DEFAULT '0000-00-00', experience INT(11) NOT NULL DEFAULT '0', invalid_phone TINYINT(4) NOT NULL DEFAULT '0', invalid_email TINYINT(4) NOT NULL DEFAULT '0', number_of_calls INT(11) NOT NULL DEFAULT '0', last_call DATETIME NULL DEFAULT NULL, PRIMARY KEY (id), UNIQUE INDEX email (email), INDEX users_details_email (email(30)) ) COLLATE='cp1251_general_ci' ENGINE=InnoDB AUTO_INCREMENT=63820;

Upvotes: 2

Views: 131

Answers (2)

Rick James
Rick James

Reputation: 142298

IN ( SELECT ... ) optimizes poorly.

Turning it into a JOIN might inflate the aggregate (SUM).

FROM ( SELECT ... ) eliminates the IN problem, but risks the 'inflate' problem.

So, EXISTS ( SELECT * ... ) is probably the best answer:

SELECT  ROUND(SUM(cb.points)) AS points
    FROM  clients_bonuses cb
    WHERE  cb.cb_year = 2016
      AND  EXISTS 
      ( SELECT  *
            FROM  users_contracts uc
            JOIN  users_details ud  ON  ud.id = uc.users_details_id
            WHERE  uc.platform_id = 1
              AND  ud.id=6
              AND  CONCAT('85500/', uc.contract) = cb.account_id 
      )

And you could benefit from:

users_contracts : INDEX(users_details_id, platform_id) -- in either order
clients_bonuses : INDEX(cb_year, account_id, points)  -- in that order

It looks like you could further speed it up by getting rid of users_details:

SELECT  ROUND(SUM(cb.points)) AS points
    FROM  clients_bonuses cb
    WHERE  cb.cb_year = 2016
      AND  EXISTS 
      ( SELECT  *
            FROM  users_contracts uc
            WHERE  uc.platform_id = 1
              AND  uc.users_details_id = 6
              AND  CONCAT('85500/', uc.contract) = cb.account_id 
      )

These formulations will probably work well on all variations of MySQL/MariaDB since 4.1.

Upvotes: 1

StanislavL
StanislavL

Reputation: 57381

You should move the ud.id=uc.users_details_id AND uc.platform_id = 1 AND ud.id=6 from WHERE to INNER JOIN

TO let it work properly add INDEX for uc.users_details_id ud.id is primary key so it's already indexed.

Upvotes: 1

Related Questions