CODEWITHSUNDEEP
parquetpresto
Anas Mosaad
Anas Mosaad

Reputation: 131

Query local parquet using presto

Using spark and drill, I am able to query local parquet files.

Does presto provide the same capability? In other words, is it possible to query local parquet files using presto - without going through HDFS or hive?

Upvotes: 4

Views: 4356

Answers (2)

Anas Mosaad
Anas Mosaad

Reputation: 131

I did not find a straightforward way to do this. This has been long time now and I am not sure if there are other options available at the moment.

What I did was; create a custom hive meta store that would return the schemas, tables with paths of my parquet files. In presto, configured it using that meta store and that worked pretty fine.

Upvotes: 1

Will
Will

Reputation: 11500

From my understanding, Presto's localfile is only for http_request_logs (which is why they have settings for: presto-logs.http-request-log.location). I wasn't able to query local parquet data with Presto.

I was able to query data using Apache Drill. Out of the box, you can switch out the below directory with your local file system and run regular SQL on it:

# Start with /bin/drill-embedded
0: jdbc:drill:zk=local> select * from dfs.`/somedir/withparquetfiles/`

Upvotes: 0

Related Questions