CODEWITHSUNDEEP
scalaapache-sparkcompression
codewin
codewin

Reputation: 41

How to extract a bz2 file in spark

I have a csv file zipped in bz2 format, like unix/linux do we have any single line command to extrac/decompress the file file.csv.bz2 to file.csv in spark-scala?

Upvotes: 1

Views: 3512

Answers (1)

Aadhil Rushdy
Aadhil Rushdy

Reputation: 74

You can use built in function in SparkContext(sc), this worked for me

sc.textFile("file.csv.bz2").saveAsTextFile("file.csv")

Upvotes: 3

Related Questions