CODEWITHSUNDEEP
hadoopamazon-s3hivepresto
Lior Baber
Lior Baber

Reputation: 882

Presto and hive partition discovery

I'm using presto mainly with hive connector to connect to hive metastore.

All of my tables are external tables pointing to data stored in S3.

My main issue with this is that there is no way (at least on I'm aware of ) to do partition discovery in Presto ,so before I start query a table in presto I need to switch to hive and run msck repair table mytable

is there more reasonable way to do it in Presto?

Upvotes: 6

Views: 7523

Answers (3)

Lou Zell
Lou Zell

Reputation: 5617

There is now a way to do this:

CALL system.sync_partition_metadata(schema_name=>'<your-schema>', table_name=>'<your-table>', mode=>'FULL')

Credit to this post and this video

Upvotes: 0

Ricardo Pedrotti
Ricardo Pedrotti

Reputation: 83

I'm on version 0.227 and the following helps me:

select * from hive.yourschema."yourtable$partitions"

This select returns all the partitions mapped in your catalog. You can filter, order, etc. as a normal query would.

Upvotes: 5

Damien Carol
Damien Carol

Reputation: 1164

No.

If the HIVE metastore doesn't see the partitions, PrestoDB will not see it.

Maybe a cron can help you.

Upvotes: 3

Related Questions