CODEWITHSUNDEEP
javaapache-spark
zpwpal
zpwpal

Reputation: 183

Use shared variable in Spark

Hi I am using BLAS to do some math computation in Spark.I got 2 JavaPairRDDs that both has a Double[] field, I want to caculate dot product as follow:

userPairRDD.cartesian(itemPairRDD).mapToPair(
                    new PairFunction<Tuple2<Tuple2<String, Double[]>, Tuple2<String, Double[]>>, String, ItemAndWeight>() {
                        @Override
                        public Tuple2<String, ItemAndWeight> call(Tuple2<Tuple2<String, Double[]>, Tuple2<String, Double[]>> tuple2Tuple2Tuple2) throws Exception {
                            BLAS.getInstance().ddot("......");
                            .......
                        }
                    }
            )

My question is, in my call(),I called BLAS.getInstance() every time it might be inefficient, Can I create only one BLAS object outside call() and just the very object to do ddot()? Is there any point to take care as this is a distributed program? Thanks in advance.

Upvotes: 1

Views: 101

Answers (1)

petertc
petertc

Reputation: 3911

You don't need shared variable in this case. BLAS.getInstance() just return a static/singleton instance, so no inefficient thing here.

Upvotes: 1

Related Questions