CODEWITHSUNDEEP
ramazon-s3parquet
ATU
ATU

Reputation: 111

How to read parquet file from AWS S3 bucket using R without downloading it locally?

I'm able to download the parquet file from AWS S3 bucket on local and then read from it (see the below code). But is there any way that I can directly read parquet file from S3 and read, without storing in local?

save_object("Financial_Sample.parquet", file = "Financial_Sample.parquet", bucket = 'my-bucket')
df <- read_parquet("Financial_Sample.parquet")```

Upvotes: 3

Views: 1119

Answers (1)

jsch
jsch

Reputation: 126

Take a look at the arrow package: https://arrow.apache.org/docs/r/index.html

It can directly read from s3 and even filter before reading using some dplyr verbs.

Upvotes: 2

Related Questions