Use Spark in Scala to read sequence files containing multiple schemas and write to avro

Question

I need to read sequence files that contains messages with different schemas:

• File 1
  • message A
  • message B
• File 2
  • message B ...

Split them by message type then write the result to avro:

• message_a/data.avro
• message_b/data.avro

Using Spark, I'm able to read and parse the data:

spark.sparkContext.sequenceFile(path, classOf[LongWritable], classOf[BytesWritable]).map { (key, value) =>
  val record: GenericRecord = parseValue(value)
  val messageType = extractType(record)
  messageType -> record
}

But from there I'm not sure how to pursue, I've tried sequence file writer but I'm not able to write a proper avro file.