CODEWITHSUNDEEP
mysqlsqlruby-on-railspostgresqlactiverecord
goddamnyouryan
goddamnyouryan

Reputation: 6896

Rails Slow Postgres Query with Order using Index

I am doing a query on my database of a few million items that gets really slow when I add in an order. Here is the code I am calling:

Post.where(source_id: source_ids_array).page(1).per(100).order("position asc, external_created_at desc")

(I am using Kaminari to do pagination)

Which gives me the following sql:

Post Load (36537.8ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."source_id" IN (17805, 18768, 20717, 17803, 17804, 18329, 20705, 19075, 19110, 19082, 18328)  ORDER BY position asc, external_created_at desc LIMIT 100 OFFSET 0

However, when I modify the query to just be:

Post.where(source_id: source_ids_array).page(1).per(100).order("position asc")

I get the following sql:

Post Load (279.6ms)  SELECT  "posts".* FROM "posts"  WHERE "posts"."source_id" IN (17805, 18768, 20717, 17803, 17804, 18329, 20705, 19075, 19110, 19082, 18328)  ORDER BY position asc LIMIT 100 OFFSET 0

Which is insanely faster.

My indexes in my schema.db look like this:

add_index "posts", ["external_created_at"], name: "index_posts_on_external_created_at", using: :btree
add_index "posts", ["position", "external_created_at"], name: "index_posts_on_position_and_external_created_at", using: :btree
add_index "posts", ["position"], name: "index_posts_on_position", using: :btree

How can I go about speeding up this query?

Edit: here is my EXPLAIN ANALYZE:

Limit  (cost=633132.80..633133.05 rows=100 width=891) (actual time=31927.725..31927.751 rows=100 loops=1)
  ->  Sort  (cost=633132.80..635226.42 rows=837446 width=891) (actual time=31927.720..31927.729 rows=100 loops=1)
        Sort Key: "position", external_created_at
        Sort Method: top-N heapsort  Memory: 78kB
        ->  Bitmap Heap Scan on posts  (cost=19878.94..601126.22 rows=837446 width=891) (actual time=487.399..30855.211 rows=858629 loops=1)
              Recheck Cond: (source_id = ANY ('{17805,18768,20717,17803,17804,18329,20705,19075,19110,19082,18328}'::integer[]))
              Rows Removed by Index Recheck: 1050547
              ->  Bitmap Index Scan on index_posts_on_source_id  (cost=0.00..19669.58 rows=837446 width=0) (actual time=485.025..485.025 rows=927175 loops=1)
                    Index Cond: (source_id = ANY ('{17805,18768,20717,17803,17804,18329,20705,19075,19110,19082,18328}'::integer[]))
Total runtime: 31927.998 ms

Upvotes: 1

Views: 1061

Answers (1)

max
max

Reputation: 101901

Although its not very well documented can specify the sort order when creating an index:

add_index :posts, [:external_created_at, :position], 
    order: { position: :asc, external_created_at: :desc }

If we then run rake db:structure:dump we can see that it creates the following SQL:

CREATE INDEX "index_posts_on_external_created_at_and_position" 
 ON "posts" ("external_created_at" DESC, "position" ASC);

Note that we don't need to specify using: :btree since Postgres defaults to using B-tree or the name:.

Upvotes: 3

Related Questions