CODEWITHSUNDEEP
scalaapache-spark
Jelly
Jelly

Reputation: 1296

Spark Scala: write some sequence files into one folder

I have a task to write some sequence files, for example: sequence1, sequence2, sequence3 into one folder. If I'm trying like that

  sequence1.saveAsSequenceFile("home/sample1")
  sequence2.saveAsSequenceFile("home/sample1")
  sequence3.saveAsSequenceFile("home/sample1")

I receive the error on second line: "directory home/sample1 already exists.

Anybody know, Is there any way to do this?

Upvotes: 0

Views: 35

Answers (1)

SCouto
SCouto

Reputation: 7928

Just union them

val rddUnion = sequence1.union(sequence2).union(sequence3)

And then write them all together

rddUnion.saveAsSequenceFile("home/sample1")

Upvotes: 1

Related Questions