Sridhar Viswanathan
Reputation: 1
Does Spark Continuous Processing support Stateful Operations like FlatMapGroupswithState
Does Spark Continuous Processing support Stateful Operations like FlatMapGroupswithState getting error like Continuous processing does not support AppendColumns operations.
Upvotes: 0
Views: 204
Answers (1)
Paulo Castro
Reputation: 11
No. According to this part of the documentation "aggregations are not yet supported" in continuous mode, this includes group aggregations.
Upvotes: 1
Related Questions
- Spark Streaming with mapGroupsWithState
- Where is the state saved for arbitrary state processing using mapGroupsWithState?
- Can intermediate state be dropped/controlled in Spark structured streaming in Complete Output mode? (Spark 2.4.0)
- Spark arbitrary stateful stream aggregation, flatMapGroupsWithState API
- How to do stateful aggregation using flatMapGroupsWithState?
- Can I use mapWithState() like updateStateByKey() in Spark Streaming?
- Spark structured streaming state management
- How to do stateless aggregations in spark using Structured Streaming 2.3.0 without using flatMapsGroupWithState?
- Spark structured streaming stateless mode
- Stateful streaming Spark processing