CODEWITHSUNDEEP
javaapache-spark
Harisyam
Harisyam

Reputation: 107

Convert an JavaRDD<ArrayList<T>> to JavaRDD<T>

Is there a direct way of converting JavaRDD<ArrayList<T>> to JavaRDD<T> using the Apache-Spark's Java API?

JavaPairRDD<NullWritable, ArrayList<Record>> baseRDD = sc.newAPIHadoopFile(args[2], InputFormat2.class, NullWritable.class,ArrayList.class, conf);  
JavaRDD<ArrayList<Record>> mapLines1 = baseRDD.values();

I want to convert the JavaRDD<ArrayList<Record>> to JavaRDD<Record>.

Upvotes: 0

Views: 726

Answers (1)

zero323
zero323

Reputation: 330423

You can simply flatMap:

rdd.flatMap(new FlatMapFunction<ArrayList<Record>, Record>() {
  @Override
  public Iterable<Record> call(ArrayList<Record> records) {
    return records;
  }
});

Upvotes: 2

Related Questions