CODEWITHSUNDEEP
scalaapache-sparkspark-streaming
scalacode
scalacode

Reputation: 1106

Dstream Spark Streaming

Hello I wrote the following code 

val receivedStream: DStream[PubsubSparkMessage] =
      PubsubSource.createStream(ssc, pubsubConfig, pubsubSubscriber)
    @volatile var receivedMessages: List[PubsubSparkMessage] = List() //scalastyle:ignore
    receivedStream.foreachRDD { rdd =>
      if (!rdd.isEmpty()) {
        receivedMessages = List(rdd.collect)
        receivedMessages
      }
    }

I am getting an error because type mismatch; My question is how to modify List[Array[PubsubSparkMessage]] to List[PubsubSparkMessage]

Upvotes: 0

Views: 45

Answers (1)

Emiliano Martinez
Emiliano Martinez

Reputation: 4133

Something like:

val list : List[Array[PubsubSparkMessage]] = List()

val result = list.flatMap(x => x)

Upvotes: 0

Related Questions