CODEWITHSUNDEEP
javamongodbapache-spark
Asish Tripathy
Asish Tripathy

Reputation: 11

MongoDB Spark Connector : mongo-spark cannot connect to DB

I'm trying MongoDB's Spark connector to read data from MongoDB. I supplied the DB and collection details to Spark conf object while starting the application. And then use the following piece of code to read.

// create configuration
SparkSession spark = SparkSession.builder()
  .master("local")
  .appName("MongoSparkConnectorIntro")
  .config("spark.mongodb.input.uri", "mongodb://localhost:27017/Employee.zipcodes")
  .config("spark.mongodb.output.uri", "mongodb://localhost:27017/Employee.test")
  .getOrCreate(); 
// Create a JavaSparkContext using the SparkSession's SparkContext object
JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());

/*Start Example: Read data from MongoDB************************/

JavaMongoRDD<Document> rdd = MongoSpark.load(jsc);

/*End Example**************************************************/

// Analyze data from MongoDB
System.out.println(rdd.count());
System.out.println(rdd.first().toJson());

But this not able to connect the localhost DB. This is showing the following error.

Exception in thread "main" java.lang.NoSuchMethodError: com.mongodb.spark.config.ReadConfig$.apply(Lorg/apache/spark/SparkConf;Lscala/collection/Map;)Ljava/lang/Object;
 at com.mongodb.spark.MongoSpark$Builder.build(MongoSpark.scala:259)
 at com.mongodb.spark.MongoSpark$.load(MongoSpark.scala:375)
 at com.mongodb.spark.MongoSpark.load(MongoSpark.scala)
 at com.mycompany.app.App2.main(App2.java:35)

I'm using the following maven dependencies.

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.11</artifactId>
  <version>2.1.0</version>      
</dependency>
<dependency>
    <groupId>org.mongodb.spark</groupId>
    <artifactId>mongo-spark-connector_2.11</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>bson</artifactId>
    <version>3.2.2</version>
</dependency>   
<dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-library</artifactId>
    <version>2.11.7</version>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>2.2.0</version>
</dependency>

Upvotes: 1

Views: 1650

Answers (1)

user1808151
user1808151

Reputation: 11

Can you try improving your mongo spark connector version to latest

<dependency>
    <groupId>org.mongodb.spark</groupId>
    <artifactId>mongo-spark-connector_2.11</artifactId>
    <version>2.2.1</version>
</dependency>

Upvotes: 1

Related Questions