Ani Menon
Ani Menon

Reputation: 28277

How to define partitions to Dataframe in pyspark?

Suppose I read a parquet file as a Dataframe in pyspark, how can I specify how many partitions it must be?

I read the parquet file like this -

df = sqlContext.read.format('parquet').load('/path/to/file')

How may I specify the number of partitions to be used?

Upvotes: 0

Views: 267

Answers (0)

Related Questions