CODEWITHSUNDEEP
pythonapache-sparkpyspark
Mario Rodríguez
Mario Rodríguez

Reputation: 149

How to convert list of lists into list of strings in Spark

Imagine that after reading a file you have a list with this format:

>>> data.take(2)
[['Hello ', 'how ', 'are ', 'you'], ['fine ', 'thank ', 'you']]

However, you just want to get a list of strings such that:

['Hello how are you', 'fine thank you']

Upvotes: 0

Views: 1360

Answers (2)

koiralo
koiralo

Reputation: 23099

In Scala we can use mkString to make a list to string 

val data = List(List("Hello ", "how ", "are ", "you "), List("fine ", "thank ", "you"))
 data.map(_.mkString)

Output:

List[String] = List("Hello how are you" , "fine thank you")

Upvotes: 1

Mario Rodríguez
Mario Rodríguez

Reputation: 149

Simply perform a map with lambda on the list in question and operate withjoin on each inner list.

>>> data = data.map(lambda x : ''.join(x))
>>> data.take(2)
['Hello how are you', 'fine thank you']

Upvotes: 3

Related Questions