CODEWITHSUNDEEP
javaapache-sparkspark-streamingsparkcore
taismile
taismile

Reputation: 101

How To Convert List Object to JavaDStream Spark?

i have List object bellow:

List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> rddQueue =
    Arrays.asList(tupleList);

how can convert its to JavaDStream Spark ? thank advance !

Upvotes: 5

Views: 1643

Answers (1)

Yuval Itzchakov
Yuval Itzchakov

Reputation: 149636

You can do that using the JavaStreamingContext and a Queue<T>:

List<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> 
  list = Arrays.asList();

// Random batch time, for the sake of the example.
JavaStreamingContext jssc = 
  new JavaStreamingContext("local[*]", "TestApp", new Duration(4000));

Queue<JavaRDD<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>>> rddQueue = 
  new LinkedList<>();

rddQueue.add(jssc.sparkContext().parallelize(list));

JavaDStream<Tuple2<Tuple3<String, Long, Object>, Tuple2<Long, Object[]>>> dStream = 
  jssc.queueStream(rddQueue);

Upvotes: 5

Related Questions