CODEWITHSUNDEEP
pythonapache-sparkpysparkpycharm
kashyap
kashyap

Reputation: 508

Facing Py4JJavaError on executing PySpark Code

enter image description hereI'm trying to execute my first PySpark code in PyCharm IDE ,and facing the following exception.

from pyspark import SparkContext


def example():
    sc = SparkContext('local')
    words = sc.parallelize(["scala", "java", "hadoop", "spark", "akka"])
    print(sc.getConf().getAll())
    return words.count()


print(example())

and Printed the following data.

[('spark.master', 'local'), ('spark.rdd.compress', 'True'), ('spark.serializer.objectStreamReset', '100'), ('spark.driver.port', '59627'), ('spark.executor.id', 'driver'), ('spark.submit.deployMode', 'client'), ('spark.app.id', 'local-1526547201037'), ('spark.driver.host', 'LAPTOP-DDRRK6SB'), ('spark.ui.showConsoleProgress', 'true'), ('spark.app.name', 'pyspark-shell')]

and the following exception.

py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: java.lang.IllegalArgumentException

Sorry for my English, Expecting what's wrong with the code.

Upvotes: 1

Views: 763

Answers (1)

kashyap
kashyap

Reputation: 508

I don't know exact problem, On rolling back to 1.8.0_171 to this java version, It is working fine. Thanks Rumoku for your suggestion.

Upvotes: 1

Related Questions