D_R
D_R

Reputation: 4972

Make Query More Efficent (Really slow query taking forever!)

What I am trying to do is to get all of the users with the right conditions so I'm building with a foreach statment a sub_queries to make it work.

the problem is that I got 100,000+ records in the Database. and this kind of query takes forever to run.

I know I'm not doing it in the best way but I also tried left joins, which was really slow too..

this is the function I'm using:

    public function get_affected_users_by_conditions($conditions, $mobile_type)
    {
        // Basic Query 
        // Selecting all of the users from `enswitch_mobile users` table
        // The total and the `users` with the conditions
        $sql = "SELECT COUNT(*) AS `users`,
                       (SELECT COUNT(*) FROM `enswitch_mobile_users`) AS `total`
                FROM 
                    `enswitch_mobile_users` AS `musers`
                WHERE 
                    `musers`.`phone_type` = :mobile_type";

        $value_counter = 0;
        $values = array();
        // This is the foreach loop I was talking about
        // I am looping all the conditons.
        // and when theres a value i'm adding it as a subquery.
        foreach($conditions as $cnd) {
            switch ($cnd['condition']) {

                // REALLY SLOW SUB-QUERY:
                case 'talked_atleast':
                    $value_counter++;
                    // Here I'm trying to CUT the query by users who talked atleast $value seconds
                    $sql .= " AND (SELECT SUM(TIME_TO_SEC(TIMEDIFF(`finished_call`,`start_call`))) FROM `enswitch_calls` WHERE `user_id` = `musers`.`id`) >= :value".$value_counter;
                    $values[$value_counter] = $cnd['value'];
                    break;

// REALLY SLOW SUB-QUERY:
                case 'purchase_atleast':
                    // Here I am trying to CUT the users by subquery who check if the users has bought at least $value times
                    $value_counter++;
                    $sql .= " AND (SELECT COUNT(*) FROM (SELECT user_id FROM enswitch_new_iphone_purchases
                                                    UNION
                                                  SELECT user_id FROM enswitch_new_android_purchases) AS p WHERE `status` > 0 AND user_id` = `musers`.`id`) >= :value".$value_counter;
                    $values[$value_counter] = $cnd['value'];
                    break;
// REALLY SLOW SUB-QUERY:
                case 'never_purchase':
                    // Here I am trying to CUT the users by subquery to get only the users who never made a puchase.
                    $sql .= ' AND (SELECT COUNT(*) FROM (SELECT user_id FROM enswitch_new_iphone_purchases
                                                    UNION
                                                  SELECT user_id FROM enswitch_new_android_purchases) AS p WHERE `status` = 0 AND `user_id` = `musers`.`id`) = 0';
                    break;
            }
        }


        $query = DB::query(Database::SELECT, $sql);
        $query->bind(':mobile_type', $mobile_type);
        // Looping the values and binding it into the SQL query!
        foreach ($values as $k => $v) {
            $query->bind(':value'.$k, $values[$k]);
        }
        // Executing query
        $result = $query->execute();
        return array('total_users' =>$result[0]['total'], 'affected_users'=>$result[0]['users']);
    }

EDIT: The Slowest Query as Requested: (MySQL)

SELECT COUNT(*) AS `users`,
       ( SELECT COUNT(*) 
           FROM `enswitch_mobile_users`
       ) AS `total`
  FROM `enswitch_mobile_users` AS `musers`
 WHERE `musers`.`phone_type` = 'iphone'
   AND ( SELECT COUNT(*) 
           FROM ( SELECT `status`,
                         `user_id` 
                    FROM `enswitch_new_iphone_purchases`
                  UNION
                  SELECT `status`,
                         `user_id` 
                    FROM `enswitch_new_android_purchases`
                ) AS `p` 
          WHERE `p`.`status` > 0 
            AND `p`.`user_id` = `musers`.`id`
       ) >= 2

Upvotes: 0

Views: 117

Answers (1)

Ed Gibbs
Ed Gibbs

Reputation: 26353

The subquery in the second SELECT column will execute for every m_users row that passes the WHERE condition:

SELECT
  COUNT(*) AS users,
  (SELECT COUNT(*) FROM enswitch_mobile_users) AS total <-- here's the problem
FROM enswitch_mobile_users AS musers
WHERE musers.phone_type = whatever

If I'm reading this correctly, you need a one-row result with the following columns:

  • users - number of enswitch_mobile_users rows with the specified phone_type
  • total - count of all enswitch_mobile_users rows

You can get the same result with this query:

SELECT
  COUNT(CASE WHEN musers.phone_type = whatever THEN 1 END) AS users,
  COUNT(*) AS total
FROM enswitch_mobile_users

The CASE checks for the phone type, and if it matches the one you're interested it it yields a 1, which is counted. If it doesn't match, it yields a NULL, which is not counted.

Upvotes: 1

Related Questions