Reputation: 107
Is there a direct way of converting JavaRDD<ArrayList<T>>
to JavaRDD<T>
using the Apache-Spark's Java API?
JavaPairRDD<NullWritable, ArrayList<Record>> baseRDD = sc.newAPIHadoopFile(args[2], InputFormat2.class, NullWritable.class,ArrayList.class, conf);
JavaRDD<ArrayList<Record>> mapLines1 = baseRDD.values();
I want to convert the JavaRDD<ArrayList<Record>>
to JavaRDD<Record>
.
Upvotes: 0
Views: 726
Reputation: 330423
You can simply flatMap
:
rdd.flatMap(new FlatMapFunction<ArrayList<Record>, Record>() {
@Override
public Iterable<Record> call(ArrayList<Record> records) {
return records;
}
});
Upvotes: 2