CODEWITHSUNDEEP
scalaapache-sparkcassandraspark-cassandra-connector
AndreaM16
AndreaM16

Reputation: 3975

Unable to connect to remote Spark Master - ROR TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message

I posted this question some time ago but it came out that I was using my local resources instead of remote's ones.

I have a remote machine configured with spark : 2.1.1, cassandra : 3.0.9 and scala : 2.11.6.

Cassandra is configured at localhost:9032 and spark master at localhost:7077.

Spark master is set to 127.0.0.1 and its port to 7077.

I'm able to connect to cassandra remotely but unable to do the same thing with spark.

When connecting to the remote spark master, I get the following error:

ERROR TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message.

Here are my settings via code

val configuration = new SparkConf(true)
                        .setAppName("myApp")
                        .setMaster("spark://xx.xxx.xxx.xxx:7077")
                        .set("spark.cassandra.connection.host", "xx.xxx.xxx.xxx")
                        .set("spark.cassandra.connection.port", 9042)
                        .set("spark.cassandra.input.consistency.level","ONE")
                        .set("spark.driver.allowMultipleContexts", "true")
val sparkSession = SparkSession
                   .builder()
                   .appName("myAppEx")
                   .config(configuration)
                   .enableHiveSupport()
                   .getOrCreate()

I don't understand why cassandra works just fine and spark does not.

What's causing this? How can I solve?

Upvotes: 1

Views: 20114

Answers (2)

AndreaM16
AndreaM16

Reputation: 3975

I answer to this question in order to help other people who are struggling with this problem.

It came out that it was caused by a mismatch between Intellij Idea's scala version and server's one.

Server had scala ~ 2.11.6 while the IDE was using scala ~ 2.11.8.

In order to make sure of using the very same version, it was necessary to change IDE's scala version by doing the following steps:

File > Project Structure > + > Scala SDK > Find and select server's scala Version > Download it if you haven't it already installed > Ok > Apply > Ok

Upvotes: 5

Frischling
Frischling

Reputation: 2268

Could this be a typo? The errormessage reports 8990, in your connect config you have port 8090 for spark.

Upvotes: 1

Related Questions