CODEWITHSUNDEEP
mongodbscalaapache-spark
Trinh Thang
Trinh Thang

Reputation: 23

Can't connect Spark and Scala 2.11 to MongoDB with SSL with .pem configuration

I have a problem when connecting from scala & spark application to Mongodb although I did on C# successful.

            My C# code:
            var url = "mongodb://user:pass@servername:27017/admin?ssl=true";
            var clientSettings = MongoClientSettings.FromUrl(new MongoUrl(url));
            var bytes = Convert.FromBase64String("MIID5TCCAs2gAwIBAgIJANoGBQADggEBAHaIdU9ri.....");
            var cert = new X509Certificate2(bytes);
            clientSettings.SslSettings = new SslSettings
            {
                ClientCertificates = new[] { cert },
                CheckCertificateRevocation = false
            };

            clientSettings.UseSsl = true;
            clientSettings.VerifySslCertificate = false;
            var client = new MongoClient(clientSettings);
            database = client.GetDatabase("databasename");

I searched the office mongodb drive document but it's quite different for each version and all of them is not right. Could you please advice with connection with Scala 2.11 and Spark 2.3? My mongodb prod is configured using .pem file.

Thank you very much! Trinh

Upvotes: 2

Views: 1163

Answers (1)

Tom Lous
Tom Lous

Reputation: 2899

The SSL config for Spark <-> MongoDB is not very well documented. In essence you need to add truststore & keystore config to your driver & executor config, see https://jira.mongodb.org/browse/SPARK-115

So you need to add that config to your spark-submit command eg: --conf "spark.executor.extraJavaOptions=-Djavax.net.ssl.trustStore=path_to_trustStore -Djavax.net.ssl.trustStorePassword=trustPassword -Djavax.net.ssl.keyStore=path_to_keyStore -Djavax.net.ssl.keyStorePassword=keyPassword"

And you need to add the crt (first part of your .pem probably) to the keystore, like:

keytool -alias mongodb -importcert -trustcacerts -file mongodb.crt -keystore path_to_keyStore -storepass keyPassword

If you find a better solution, please let me know

Upvotes: 1

Related Questions