CODEWITHSUNDEEP
javascalastateapache-flink
Manos Ntoulias
Manos Ntoulias

Reputation: 543

Flink Raw and Managed Stated. When to use them?

Assume that I have the following code. Engine gets initilized by implementations of this abstract class and it is a complex class with hashtables inside. What are the disadvantages of declaring it this way? From my understanding this is neither raw nor managed state. Should I use one of them and why?

Managed state is described here and uses classes existing in the flink library (e.g., ValueState).

Raw state on the other hand is described here and from what I see can only be used when extending the AbstractStreamOperator class which is not the case here (RichMapFunction is extended). (A code example would be aprreciated)

abstract class EmbeddedEngineMap(fsmList: List[FSMInterface],
                             predList: List[PredictorInterface],
                             predictorEnabled: Boolean,
                             expirationTime: Long,
                             collectStats: Boolean,
                             finalsEnabled: Boolean,
                             distance: (Double,Double)) extends RichMapFunction[GenericEvent, Unit] {

       protected var engine: ERFEngine = _
       ....
}

Upvotes: 0

Views: 123

Answers (1)

Dominik Wosiński
Dominik Wosiński

Reputation: 3874

Well, basically if You don't use the Flink state, this basically means that Your ERFEngine will be instantiated as a new object each time the Job is started or restarted. This basically means that if during the execution of the job Your engine keeps some state it will be lost when the job fails, restarts or is stopped.

I won't describe the difference between the raw and managed state since the post You are citing does this pretty well. Basically Raw state is a rather low-level API that allows You to implement Your own operators, so generally, the managed state is preferred as it leverages some of the Flink functionalities (for example allows redistribution on parallelism change) and is generally easier to use.

Upvotes: 2

Related Questions