CODEWITHSUNDEEP
scalaapache-flink
www.data-blogger.com
www.data-blogger.com

Reputation: 4164

Apache Flink - groupBy

I am trying to follow the first exercise on http://dataartisans.github.io/flink-training/exercises/.

Now I come to the following problem. What does the groupBy function give me back? And how does the foldLeft method transform that - for me unknown - result?

The code is the following:

mails.map {
          m => (m._1.substring(0, 7), m._2.substring(m._2.lastIndexOf("<") + 1, m._2.length - 1))
        }
        .groupBy(0, 1)
        .reduceGroup( ms => ms.foldLeft("", "", 0)( (c, m) => (m._1, m._2, c._3 + 1) ) )

Regards, Kevin

Upvotes: 1

Views: 2774

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62285

groupBy returns grouped data set: https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/batch/dataset_transformations.html#groupreduce-on-grouped-dataset

foldLeft defines the folding (or reducing) order. See here: https://en.wikipedia.org/wiki/Fold_%28higher-order_function%29

Upvotes: 2

Related Questions