CODEWITHSUNDEEP
scalaapache-sparkdata-structuresapache-spark-sql
Riccardo
Riccardo

Reputation: 413

How to convert Seq[Column] into a Map[String,String] and change value?

I'm new to Scala and Spark and I'm struggling to do the following, I'm trying to convert a Seq into a Map and at the same time, modify the value when I'm converting the sequence by adding a "_suffix" like so:

val columns = Seq($"col2",$"col3")
val map = columns.map(t => t.toString() -> t.toString()+"_suffix").toMap

However, I'm getting the following error:

scala> val map = columns.map(t => t.toString() -> t.toString()+"_suffix").toMap
<console>:25: error: Cannot prove that String <:< (T, U).
       val map = columns.map(t => t.toString() -> t.toString()+"_suffix").toMap

Furthermore, the .map result returns the following:

scala> val map = columns.map(t => t.toString() -> t.toString()+"_suffix")
map: Seq[String] = List((col2,col2)_suffix, (col3,col3)_suffix)

And this is what I'm trying to produce:

map: Seq[(String, String)] = List((col2,col2_suffix), (col3,col3_suffix)

So I can ultimately convert it into a Map:

map: scala.collection.immutable.Map[String,String] = Map(col2 -> col2_suffix, col3 -> col3_suffix)

I'm pretty stuck trying to achieve this, any advice?

Upvotes: 2

Views: 3282

Answers (2)

Shaido
Shaido

Reputation: 28322

What you have done is correct, just add some parenthesis and it will work:

columns.map(t => t.toString() -> (t.toString() + "_suffix")).toMap

As you can see from your results, the "_suffix" is added to the string of the tuple, i.e.

(col2,col2)_suffix

What you want to do is add the string to the second element only, hence the parenthesis is necessary.

Upvotes: 2

RAGHHURAAMM
RAGHHURAAMM

Reputation: 1099

Here is your supposed input($ to be removed before quotes as shown in question):

scala> val columns = Seq("col2","col3")
columns: Seq[String] = List(col2, col3)

In your code there is no need to use toString() method on each string as they are already strings, and use map as below:

val map = columns.map(t => (t,t+"_suffix")).toMap

In Scala REPL: 

scala> val map = columns.map(t => (t,t+"_suffix"))
map: Seq[(String, String)] = List((col2,col2_suffix), (col3,col3_suffix))

scala> val map = columns.map(t => (t,t+"_suffix")).toMap
map: scala.collection.immutable.Map[String,String] = Map(col2 -> col2_suffix, col3 -> col3_suffix)

Upvotes: 2

Related Questions