CODEWITHSUNDEEP
mongodbapache-sparkhadooppymongo
ce123
ce123

Reputation: 1

MongoDB & Spark: difference between mongo-hadoop and mongo-spark

what is the difference between mongo-hadoop and mongo-spark connector and does pymongo work only with mango-hadoop?

Is pymongo used only with mongo-hadoop?

Upvotes: 0

Views: 687

Answers (1)

Snail Pacer
Snail Pacer

Reputation: 91

The MongoDB Connector for Hadoop is a library which allows MongoDB (or backup files in its data format, BSON) to be used as an input source, or output destination, for Hadoop MapReduce tasks. It is designed to allow greater flexibility and performance and make it easy to integrate data in MongoDB with other parts of the Hadoop ecosystem including the following:

  1. Pig
  2. Spark
  3. MapReduce
  4. Hadoop Streaming
  5. Hive
  6. Flume

The MongoDB Connector for Spark provides integration between MongoDB and Apache Spark.

With the connector, you have access to all Spark libraries for use with MongoDB datasets: Datasets for analysis with SQL (benefiting from automatic schema inference), streaming, machine learning, and graph APIs. You can also use the connector with the Spark Shell.

PyMongo doesn't provide any support for PySpark, but the Spark connector does:

https://docs.mongodb.com/spark-connector/master/python-api/

Upvotes: 2

Related Questions