MySQL MAX_JOIN_SIZE error

Question

I have two tables. One is a call history table which logs calls made (starttime, endtime, phone number, user, etc). The other is an orders table which logs order details (order number, customer info, orderdate, etc.). Orders are not always created when a call is created so there isnt a guaranteed ID to match them up. Right now, I'm interested in getting totals by day. When I try to run a a query to sum calls and join orders by day I get the following error:

The SELECT would examine more than MAX_JOIN_SIZE rows; check your WHERE and use SET SQL_BIG_SELECTS=1 or SET MAX_JOIN_SIZE=# if the SELECT is okay

This is the query I use:

SELECT
    DATE_FORMAT(c.date_call_start,'%Y-%m-%d') as date,
    COUNT(c.id) as calls,
    COUNT(o.id) as orders

FROM tbl_calls c
    LEFT OUTER JOIN tbl_orders o 
    ON DATE_FORMAT(c.date_call_start,'%Y-%m-%d')  =  DATE_FORMAT(o.created,'%Y-%m-%d') 

WHERE c.campaign_id = 1 
    AND DATE_FORMAT(c.date_call_start,'%Y-%m-%d') = '2013-12-09'

GROUP BY DATE_FORMAT(c.date_call_start,'%Y-%m-%d')

Even when there are only a few calls for a particular day, it still shows the same error. So I'm pretty sure it my query that needs work.

I have also tried a sub query, but that doesn't rollup the totals from the subquery.

SELECT

    DATE_FORMAT(c.date_call_start,'%Y-%m-%d') as date,
    count(c.id) as calls,
    (select count(DISTINCT o.id)
                    FROM tbl_orders o
                    WHERE DATE_FORMAT(o.created,'%Y-%m-%d') = DATE_FORMAT(c.date_call_start,'%Y-%m-%d')
                ) as orders

FROM tb_calls c

WHERE c.campaign_id = 1 
    AND DATE_FORMAT(c.date_call_start,'%Y-%m-%d') BETWEEN '2013-12-09' AND '2013-12-15'

GROUP BY DATE_FORMAT(c.date_call_start,'%Y-%m-%d')
WITH ROLLUP

Any thoughts on how I can get this query to work? Ultimately I'd like a result like below so I can do other calculations like % orders etc.

date         |  calls   |    orders
------------------------------------
2013-12-01   |       100|       10
2013-12-02   |       125|       20
NULL         |       225|       30

UPDATED: Based on the answer I did the following: created call_date field with a date field (no datetime) to tbl_calls created date_order field with a date format (not datetime) to tbl_orders Updated each table and set the new fields to = date_format(the_date_time_stamp,'%Y-%m-%d') from the same table. Also added an index to each of the new date fields.

That made the following query work:

SELECT
    c.call_date as date,
    COUNT(DISTINCT c.id) as calls,
    COUNT(DISTINCT o.id) as orders,
    ROUND((COUNT(DISTINCT o.id) / COUNT(DISTINCT c.id))*100,2) as conversion

FROM tbl_calls c
 JOIN tbl_orders o 
    ON c.call_date  =  o.date_order

WHERE c.campaign_id = 1 
    AND c.call_date BETWEEN '2013-12-09' AND '2013-12-15' 

GROUP BY c.call_date
WITH ROLLUP

Which gives me the following result and I can build off of this. Thanks to each of you who provided suggestions. I tried each. All make sense. However, since I ultimately had to create the additional date fields I chose the answer by

date         | calls  | orders| conversion
-------------------------------------------
2013-12-09   |  151   | 6     | 3.97
2013-12-10   |  164   | 2     | 1.22
2013-12-11   |  165   | 6     | 3.64
2013-12-12   |  189   | 1     | 0.53
2013-12-13   |  116   | 4     | 3.45
null         |  785   | 19    | 2.42

Answer 1

If @Barmar 's suggestion does not work, then you may need to split the fields into DATE and TIME.

A different direction is to make two temp tables (giving you three queries:

CREATE TEMPORARY TABLE `tbl_calls_temp`  SELECT *  FROM tbl_calls c WHERE DATE(c.date_call_start) = '2013-12-09' AND c.campaign_id = 1

Then do the same restricting for the tbl_orders TABLE

CREATE TEMPORARY TABLE `tbl_orders_temp` SELECT * FROM tbl_orders o WHERE DATE(o.created) = '2013-12-09'

Finally query against the two temporary tables. Depending on how much data you get, you may want to add indexes to the temporary tables... but in all likelihood you are facing a full-join

SELECT
    DATE_FORMAT(c.date_call_start,'%Y-%m-%d') as date,
    COUNT(c.id) as calls,
    COUNT(o.id) as orders

FROM tbl_calls_temp c
    LEFT OUTER JOIN tbl_orders_temp o 
    ON DATE_FORMAT(c.date_call_start,'%Y-%m-%d')  =  DATE_FORMAT(o.created,'%Y-%m-%d') 
GROUP BY DATE_FORMAT(c.date_call_start,'%Y-%m-%d')

And that should be much faster... assuming you have any indexes in your initial tables that can be queried.

Answer 2

Presumably you want to count the calls and orders for each date. However, that is not what your query does, because it creates a cartesian product for all orders on a given date.

Instead, summarize the data first by date and then combine the results. This may be what you want:

select c.date, calls, orders
from (select DATE_FORMAT(c.date_call_start, '%Y-%m-%d') as date, count(*) as calls
      from tbl_calls c
      WHERE c.campaign_id = 1 and
            DATE_FORMAT(c.date_call_start, '%Y-%m-%d') = '2013-12-09'
      group by DATE_FORMAT(c.date_call_start, '%Y-%m-%d')
     ) c left outer join
     (select DATE_FORMAT(o.created,'%Y-%m-%d') as date, count(*) as orders
      from tbl_orders o
      group by DATE_FORMAT(o.created, '%Y-%m-%d')
     ) o
     on c.date = o.date;

Answer 3

First - try the results of EXPLAIN SELECT.... where ... is the rest of your select query above.

Since you're performing the join on two fields which have a function applied to them - I'm take a guess and say MySQL is performing two full table scans and using type all for the join. See this for an explanation of the EXPLAIN output.

DATE_FORMAT(c.date_call_start,'%Y-%m-%d')  =  DATE_FORMAT(o.created,'%Y-%m-%d')

You'll most likely want to create a separate field in each table that contains just the result of the DATE_FORMAT call. Then create an index for each of these new fields. Then join on these new indexed fields. MySQL should like that much better.