CODEWITHSUNDEEP
presto
Sobhagya Mohanty
Sobhagya Mohanty

Reputation: 628

Exception while running query on hive bucketed table in presto

Get this Exception while running query on streaming hive table using presto.

HIVE_PARTITION_SCHEMA_MISMATCH (16777224)

com.facebook.presto.spi.PrestoException: Found sub-directory in bucket directory
at com.facebook.presto.hive.BackgroundHiveSplitLoader.listAndSortBucketFiles(BackgroundHiveSplitLoader.java:367)
at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:333)
at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:212)
at com.facebook.presto.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:67)
at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:168)
at com.facebook.presto.hive.util.ResumableTasks.safeProcessTask(ResumableTasks.java:45)
at com.facebook.presto.hive.util.ResumableTasks.lambda$submit$66(ResumableTasks.java:33)
at io.airlift.concurrent.BoundedExecutor.executeOrMerge(BoundedExecutor.java:69)
at io.airlift.concurrent.BoundedExecutor.access$000(BoundedExecutor.java:28)
at io.airlift.concurrent.BoundedExecutor$1.run(BoundedExecutor.java:40)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

PS : Table contains data and returns result using hql. And other tables works fine on presto.

Upvotes: 1

Views: 861

Answers (1)

Dain Sundstrom
Dain Sundstrom

Reputation: 2858

FWIU, this means that you have a table that is declared with bucketing in the Hive metastore but is not properly bucketed. Specifically, bucketed tables should not contain directories. Unlike Hive, Presto always takes advantage of bucketing properties, so an incorrectly bucketed Hive table will fail the query.

The only work arounds I know of are to either remove the bucketing flag from the table metadata using Hive, or rewrite the table using Hive to be properly bucketed.

Upvotes: 1

Related Questions