CODEWITHSUNDEEP
apache-spark
Anaadih.pradeep
Anaadih.pradeep

Reputation: 2573

Why is "error: illegal start of simple expression" for reduceByKey?

I am new to scala and spark and while implementing reduceByKey i got the below error.

var redRdd = filterRdd.reduceByKey((acc , val) => if (acc > val ) acc else val)
<console>:1: error: illegal start of simple expression
var redRdd = filterRdd.reduceByKey((acc , val) => if (acc > val ) acc else val)

filterRdd is just (order_id, Amount) tuple.

scala> filterRdd.first
res10: (Int, Double) = (1,299.98)

I want to first get the max value per partition and then reduce output from all partitions. can someone help me in understanding why i am getting this error?

Upvotes: 1

Views: 2451

Answers (1)

Knight71
Knight71

Reputation: 2949

val is a key word in scala and you cannot have a variable named as val. Change the val to v it will work.

 var redRdd = filterRdd.reduceByKey((acc , v) => if (acc > v ) acc else v)

It will work.

And you could use math.max(acc,v) instead of the if else.

Upvotes: 3

Related Questions