Reputation: 399
I have 2 RDD's that I joined them together using left join. As a result, the fields of the right RDD are now defined as Option as they might be None (null). when writing the result to a file it looks something like this: Some(value) for example: Some('value1'), Some('Value2'). How can I remove the 'Some' / remove the Option from the field definition?
Upvotes: 0
Views: 360
Reputation: 16085
If you have an Option[String]
and turn this into a String
, you still need to handle the case where your value is None
.
For example you can turn None
's into empty strings:
val myInput: Option[String] = ...
val myOutput: String = myInput.getOrElse("")
Or into null
's:
val myInput: Option[String] = ...
val myOutput: String = myInput.orNull
Or not write them at all:
val myInput: Option[String] = ...
// Does nothing if myInput is None
myInput.foreach(writeToFile)
Upvotes: 3