How limit number of lines read from a parquet file in sparklyr

Question

I have a huge parquet file that dont fits in memory nor in disk when read, theres a way to use spark_read_parquet to only read the first n lines?

Answer 1

This might be a hacky way, but

spark_read_parquet(..., memory=FALSE) %>% head(n)

seems to do the job for me.