CODEWITHSUNDEEP
apache-flink
JanOels
JanOels

Reputation: 141

State Schema Evolution with POJOs

I'm using flink 1.11 with Scala and i have a question regarding the schema evolution using a POJO.

In the documentation is written, that POJOs are supported for state schema evolution (with some limitations).

Are Scala case clases also considered as POJO and therefore supported?

case class WordCount(word: String, count: Int)

Or have i to write something like this:

class WordCount(var word: String, var count: Int) {
    def this() {
      this(null, -1)
    }
}

Upvotes: 0

Views: 385

Answers (1)

Arvid Heise
Arvid Heise

Reputation: 3634

Case classes are not POJOs. In particular, they do not satisfy:

  • The class has a public no-argument constructor
  • All non-static, non-transient fields in the class (and all superclasses) are either public (and non-final) or have a public getter- and a setter- method that follows the Java beans naming conventions for getters and setters. (afaik case classes have final fields with getters in the generated JVM class)

You can implement all required things in a normal scala class but your IDE might not support you well. An option is to create your class in Java, let your IDE beanify it and convert it to scala (or use it directly).

There is also the option to create evolution support for case classes with a custom serializer. That will eventually be available by Flink. (You could also go ahead and contribute it).

Upvotes: 2

Related Questions