CODEWITHSUNDEEP
postgresql
nmakb
nmakb

Reputation: 1245

query taking nested loop instead of hash join

Below query is nested loop and runs for 21 mins, after disabling nested loop it works in < 1min. Table stats are up to date and vacuum is run on the tables, any way to figure out why postgres is taking nested loop instead of efficient hash join.

Also to disable a nested loop is it better to set enable_nestloop to off or increase the random_page_cost? I think setting nestloop to off would stop plans from using nested loop if its going to be efficient in some places. What would be a better alternative, please advise.

SELECT DISTINCT ON (three.quebec_delta)
    last_value(three.reviewed_by_nm) OVER wnd AS reviewed_by_nm,
    last_value(three.reviewer_specialty_nm) OVER wnd AS reviewer_specialty_nm,
    last_value(three.kilo) OVER wnd AS kilo,
    last_value(three.review_reason_dscr) OVER wnd AS review_reason_dscr,
    last_value(three.review_notes) OVER wnd AS review_notes,
    last_value(three.seven_uniform_charlie) OVER wnd AS seven_uniform_charlie,
    last_value(three.di_audit_source_system_cd) OVER wnd AS di_audit_source_system_cd,
    last_value(three.di_audit_update_dtm) OVER wnd AS di_audit_update_dtm,
    three.quebec_delta
FROM
    ods_authorization.quebec_foxtrot seven_uniform_foxtrot
    JOIN ods_authorization.golf echo ON seven_uniform_foxtrot.four = echo.oscar
    JOIN ods_authorization.papa three ON echo.five = three.quebec_delta
        AND three.xray = '0'::bpchar
WHERE
    seven_uniform_foxtrot.two_india >= (zulu () - '2 years'::interval)
    AND lima (three.kilo, 'ADVISOR'::character varying)::text = 'ADVISOR'::text
WINDOW wnd AS (PARTITION BY three.quebec_delta ORDER BY three.seven_uniform_charlie DESC ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)

Plan running for 21m and taking nested loop

Unique  (cost=550047.63..550257.15 rows=5238 width=281) (actual time=1295000.966..1296128.356 rows=319863 loops=1)
  ->  WindowAgg  (cost=550047.63..550244.06 rows=5238 width=281) (actual time=1295000.964..1296013.046 rows=461635 loops=1)
        ->  Sort  (cost=550047.63..550060.73 rows=5238 width=326) (actual time=1295000.929..1295089.796 rows=461635 loops=1)
                Sort Key: three.quebec_delta, three.seven_uniform_charlie DESC
                Sort Method: quicksort  Memory: 197021kB
              ->  Nested Loop  (cost=1001.12..549724.06 rows=5238 width=326) (actual time=8.274..1292470.826 rows=461635 loops=1)
                    ->  Gather  (cost=1000.56..527782.84 rows=24896 width=391) (actual time=4.287..12701.687 rows=3484699 loops=1)
                            Workers Planned: 2
                            Workers Launched: 2
                          ->  Nested Loop  (cost=0.56..524293.24 rows=10373 width=391) (actual time=3.492..400998.923 rows=1161566 loops=3)
                                ->  Parallel Seq Scan on papa three  (cost=0.00..436912.84 rows=10373 width=326) (actual time=1.554..2455.626 rows=1161566 loops=3)
                                        Filter: ((xray = 'november'::bpchar) AND ((lima_sierra(kilo, 'two_zulu'::character varying))::text = 'two_zulu'::text))
                                        Rows Removed by Filter: 501723
                                ->  Index Scan using five_tango on golf echo  (cost=0.56..8.42 rows=1 width=130) (actual time=0.342..0.342 rows=1 loops=3484699)
                                        Index Cond: (five_hotel = three.quebec_delta)
                    ->  Index Scan using lima_alpha on quebec_foxtrot seven_uniform_foxtrot  (cost=0.56..0.88 rows=1 width=65) (actual time=0.366..0.366 rows=0 loops=3484699)
                            Index Cond: (four = echo.oscar)
                            Filter: (two_india >= (zulu() - 'two_two'::interval))
                            Rows Removed by Filter: 1
Planning time: 0.777 ms
Execution time: 1296183.259 ms

Plan after setting enable_nestloop to off and work_mem to 8GB. I get the same plan when increasing random_page_cost to 1000.

Unique  (cost=5933437.24..5933646.68 rows=5236 width=281) (actual time=19898.050..20993.124 rows=319980 loops=1)
  ->  WindowAgg  (cost=5933437.24..5933633.59 rows=5236 width=281) (actual time=19898.049..20879.655 rows=461769 loops=1)
        ->  Sort  (cost=5933437.24..5933450.33 rows=5236 width=326) (actual time=19898.022..19978.839 rows=461769 loops=1)
                Sort Key: three.quebec_delta, three.seven_uniform_charlie DESC
                Sort Method: quicksort  Memory: 197056kB
              ->  Hash Join  (cost=1947451.87..5933113.80 rows=5236 width=326) (actual time=11616.323..17931.146 rows=461769 loops=1)
                      Hash Cond: (echo.oscar = seven_uniform_foxtrot.four)
                    ->  Gather  (cost=438059.74..4423656.32 rows=24897 width=391) (actual time=1909.685..7291.289 rows=3484833 loops=1)
                            Workers Planned: 2
                            Workers Launched: 2
                          ->  Parallel Hash Join  (cost=437059.74..4420166.62 rows=10374 width=391) (actual time=1904.546..7385.948 rows=1161611 loops=3)
                                  Hash Cond: (echo.five = three.quebec_delta)
                                ->  Parallel Seq Scan on golf echo  (cost=0.00..3921922.09 rows=8152209 width=130) (actual time=0.003..1756.576 rows=6531668 loops=3)
                                ->  Parallel Hash  (cost=436930.07..436930.07 rows=10374 width=326) (actual time=1904.354..1904.354 rows=1161611 loops=3)
                                        Buckets: 4194304 (originally 32768)  Batches: 1 (originally 1)  Memory Usage: 1135200kB
                                      ->  Parallel Seq Scan on papa three  (cost=0.00..436930.07 rows=10374 width=326) (actual time=0.009..963.728 rows=1161611 loops=3)
                                              Filter: ((xray = 'november'::bpchar) AND ((lima(kilo, 'two_zulu'::character varying))::text = 'two_zulu'::text))
                                              Rows Removed by Filter: 502246
                    ->  Hash  (cost=1476106.74..1476106.74 rows=2662831 width=65) (actual time=9692.517..9692.517 rows=2685656 loops=1)
                            Buckets: 4194304  Batches: 1  Memory Usage: 287171kB
                          ->  Seq Scan on quebec_foxtrot seven_uniform_foxtrot  (cost=0.00..1476106.74 rows=2662831 width=65) (actual time=0.026..8791.556 rows=2685656 loops=1)
                                  Filter: (two_india >= (zulu() - 'two_two'::interval))
                                  Rows Removed by Filter: 9984069
Planning time: 0.742 ms
Execution time: 21218.770 ms

Upvotes: 0

Views: 1707

Answers (2)

jjanes
jjanes

Reputation: 44373

If you just replace COALESCE(r_cd, 'ADVISOR') = 'ADVISOR' with

(r_cd = 'ADVISOR' or r_cd IS NULL)

That might use the current table statistics to improve the estimates enough to change the plan.

Upvotes: 1

Laurenz Albe
Laurenz Albe

Reputation: 247830

Try an index on papa(lima_sierra(kilo, 'two_zulu'::character varying)) and ANALYZE the table. With that index in place, PostgreSQL collects statistics on the expression, which should improve the estimate, so that you don't get a nested loop join.

Upvotes: 2

Related Questions