CODEWITHSUNDEEP
scalaapache-spark
mop
mop

Reputation: 433

the first column as "key" then add the rest every column's value

the first column as "key" then add the rest every column's value

in fact the source data file more that 22 columns

as following only an example: source file(column delimiter is a space):

a 1 2 3
b 1 2 3
a 2 3 4
b 3 4 5

desired output:

a 3 5 7
b 4 6 8

val data = scala.io.Source.fromFile("/root/1.txt").getLines

data.toList

how to do next step? thx

Upvotes: 1

Views: 534

Answers (1)

Evgeny Veretennikov
Evgeny Veretennikov

Reputation: 4249

General algorithm for solving this task:

  1. Split each line by separator
  2. Group lines by first column
  3. Remove first column from each line
  4. Transform all strings to numbers
  5. Sum lines
  6. Print result

With plain Scala:

val data = List("a 1 2 3", "b 1 2 3", "a 2 3 4", "b 3 4 5")
data.map(_.split(" ")) // 1
  .groupBy(_.head) // 2
  .mapValues(
    _.map(
      _.tail // 3
        .map(_.toInt)) // 4
      .reduce((a1, a2) => a1.zip(a2).map(tuple => tuple._1 + tuple._2))) // 5
  .foreach(pair => println(s"${pair._1} ${pair._2.mkString(" ")}")) // 6

Upvotes: 1

Related Questions