Complex Sort Function using Join, SQL, Order Categories by Products' expiry, Opencart

Question

UPDATE

My end goal was to sort categories by the expiry dates of the products within. So the category with a product closest to expiring goes first, categories with all products expired go last. Thanks to Greg, see the complete answer at the bottom of the question.

/UPDATE

This works fine except for the second line (SELECT MIN(date_expires... Any ideas on how to rewrite this line?

 SELECT   c.category_id, 
          name, 
          c.image, 
          description,
          (SELECT MIN(date_expires) 
           FROM     product 
           WHERE date_expires >= NOW() AND 
                 p.product_id = p2c.product_id) AS expiring
FROM category c 
        LEFT JOIN category_description cd 
            ON (c.category_id = cd.category_id) 
        LEFT JOIN product_to_category p2c 
            ON (c.category_id = p2c.category_id) 
        LEFT JOIN product p 
            ON (p2c.product_id = p.product_id) 
WHERE c.parent_id = 0 AND c.status = '1'
GROUP BY c.category_id
ORDER BY expiring

Some sample data:

category:
category_id = 62
category_id = 64
category_id = 63

product_to_category:
product_id = 43 category_id = 62
product_id = 50 category_id = 63
product_id = 56 category_id = 63
product_id = 58 category_id = 62
product_id = 59 category_id = 63
product_id = 60 category_id = 63
product_id = 61 category_id = 63
product_id = 62 category_id = 63
product_id = 63 category_id = 64

product:
product_id = 43 date_expires = 2012-07-11 20:35:00
product_id = 50 date_expires = 2012-06-29 00:00:00
product_id = 56 date_expires = 2012-07-13 00:00:00
product_id = 58 date_expires = 2012-07-26 15:01:00
product_id = 59 date_expires = 2012-07-16 14:12:00
product_id = 60 date_expires = 2012-07-18 08:15:00
product_id = 61 date_expires = 2012-08-02 13:04:00
product_id = 62 date_expires = 2012-08-24 00:00:00
product_id = 63 date_expires = 2012-07-12 19:16:00

ANSWER My end goal was to get

SELECT c.category_id, minDateExpires.Expiring, 
CASE WHEN Expiring > 1 THEN 1 ELSE 2 END as available 
FROM category c
LEFT JOIN product_to_category p2c ON (c.category_id = p2c.category_id)   
LEFT JOIN product p ON (p2c.product_id = p.product_id)   
LEFT JOIN ( 
    SELECT MIN(date_expires) AS expiring, category_id FROM product p join product_to_category p2c on p2c.product_id = p.product_id WHERE date_expires >= GETDATE() GROUP BY category_id
) minDateExpires on p2c.category_id = minDateExpires.category_id
ORDER BY available, expiring  

If you are looking for this for OpenCart in particular and you are using the Categories module then go to catalog/model/catalog/category.php and edit function getCategoriesPag() to:

    public function getCategoriesPag($data) {
        $query = $this->db->query("SELECT c.category_id, name, c.image, description, minDateExpires.Expiring, CASE WHEN Expiring > 1 THEN 1 ELSE 2 END as available FROM " . DB_PREFIX . "category c LEFT JOIN " . DB_PREFIX . "category_description cd ON (c.category_id = cd.category_id) LEFT JOIN " . DB_PREFIX . "category_to_store c2s ON (c.category_id = c2s.category_id) LEFT JOIN " . DB_PREFIX . "product_to_category p2c ON (c.category_id = p2c.category_id) LEFT JOIN " . DB_PREFIX . "product p ON (p2c.product_id = p.product_id) LEFT JOIN (SELECT MIN(date_expires) AS expiring, category_id FROM product p JOIN product_to_category p2c on p2c.product_id = p.product_id WHERE date_expires >= NOW() GROUP BY category_id) minDateExpires on p2c.category_id = minDateExpires.category_id WHERE c.parent_id = 0 AND cd.language_id = '" . (int)$this->config->get('config_language_id') . "' AND c2s.store_id = '" . (int)$this->config->get('config_store_id') . "'  AND c.status = '1' GROUP BY c.category_id ORDER BY available, Expiring, c.sort_order");

        return $query->rows;
    }

You will need to add your own date_expires field to the database and to your back end admin panel.

Answer 1

I don't believe this statement is true:

The original problem lies in the fact that the sub query cannot see the tables you joined to in the main query. So p.product_id and p2c.product_id do not exist in the sub query.

Since that condition is part of an outer join in the main query, that condition is already known to be true or will evaluate to null and thereby cause a null result. Now your sample data suggests you don't need outer joins anyway and I can see that there are expiration dates into the future on many of the rows.

I believe you meant to write this as your subquery. It might be worth a try even though some of the other info we've got suggests it still won't quite be the final solution.

(
SELECT MIN(date_expires) 
FROM product as p2
WHERE date_expires >= NOW() AND 
      p2.product_id = p.product_id
) AS expiring

EDIT Make the change I just suggested, but also get rid of the GROUP BY clause.

EDIT 2

1) You seem to have attributes in multiple tables that might belong in a single product table. I'll assume that you have good reasons for that.

2) Mysql allows you to mix aggregate and non-aggregate columns, but that is a bad practice and if my theory proves true, that was a big part of the reason we were all stumped for a little while.

3) All of the left outer joins seem be doing what you'd normally use an inner join for. You won't regret doing this right the first time.

4) The expiration date can be looked up via a subquery the way you first wrote it or as another join as done by the other answer. If there is really a one-to-one relationship there, then MIN() and GROUP BY are not necessary and may confuse somebody. (In fact, see edit #3.)

5) Since your subquery evaluates as null for any product with an expiration falling in the past, you may need to ORDER BY a second column or just use the expiration date regardless of whether it's past or future.

EDIT 3

SELECT
    c.category_id,
    MIN(name) as name,
    c.image /* better as a subquery if you can't do an aggregate on a LOB */
    MIN(c.description) as description,

    /* don't even need the subquery because you've already joined the product table */
    CASE
        WHEN MIN(p.date_expires) >= NOW() THEN MIN(p.date_expires)
        ELSE NULL /* this may not be what you really wanted */
    END as expiring_my_first_guess,

    /* what was really intended */
    MIN( CASE WHEN p.date_expires >= NOW() THEN p.date_expires ELSE NULL END ) as expiring,

FROM category c 
    INNER JOIN category_description cd 
        ON (c.category_id = cd.category_id) 
    INNER JOIN product_to_category p2c 
        ON (c.category_id = p2c.category_id) 
    INNER JOIN JOIN product p 
        ON (p2c.product_id = p.product_id) 
WHERE c.parent_id = 0 AND c.status = '1'
GROUP BY c.category_id
ORDER BY expiring

Answer 2

UPDATE TO ANSWER NOW THAT DATA SAMPLES ARE AVAILABLE

Still assuming it's Microsoft SQL Server, this query produces the following result

SELECT c.category_id, minDateExpires.Expiring 
FROM category c   
LEFT JOIN product_to_category p2c ON (c.category_id = p2c.category_id)   
LEFT JOIN product p ON (p2c.product_id = p.product_id)   
LEFT JOIN ( 
    SELECT MIN(date_expires) AS expiring, Product_ID FROM product WHERE date_expires >= GETDATE() GROUP BY Product_ID 
) minDateExpires on p.product_id = minDateExpires.product_id 
ORDER BY expiring  

Results:

category_id Expiring
63      NULL
63      NULL
62      NULL
64      NULL
63      2012-07-16 14:12:00.000
63      2012-07-18 08:15:00.000
62      2012-07-26 15:01:00.000
63      2012-08-02 13:04:00.000
63      2012-08-24 00:00:00.000

The null columns are there because there is no min date for those product id's >= the current date/time. Are you actually after the minimum date time grouped by categoryid, not product id? In which case your query could be something like:

SELECT c.category_id, minDateExpires.Expiring
FROM @category c
LEFT JOIN @product_to_category p2c ON (c.category_id = p2c.category_id)   
LEFT JOIN @product p ON (p2c.product_id = p.product_id)   
LEFT JOIN ( 
    SELECT MIN(date_expires) AS expiring, Category_ID FROM @product p join @product_to_category p2c on p2c.product_id = p.product_id WHERE date_expires >= GETDATE() GROUP BY Category_id
) minDateExpires on p2c.category_id = minDateExpires.category_id
ORDER BY expiring  

which gives you

64  NULL
63  2012-07-16 14:12:00.000
63  2012-07-16 14:12:00.000
63  2012-07-16 14:12:00.000
63  2012-07-16 14:12:00.000
63  2012-07-16 14:12:00.000
63  2012-07-16 14:12:00.000
62  2012-07-26 15:01:00.000
62  2012-07-26 15:01:00.000