CODEWITHSUNDEEP
javassl
Ciro Pinto-Coelho
Ciro Pinto-Coelho

Reputation: 51

I'm struggling with an SSL Handshake Fatal Error in a Simple Web Scraper

I’m just learning Java; please bear with my current lack of knowledge. I’m trying to write a program to scrape publicly available data from various web sites. I’m using a simple Java class to start the development of this web scraper (see SimpleWebScraper class below). This class works well for some sites (e.g., https://www.codetriage.com/), but others (e.g., https://forum.mrmoneymustache.com/ask-a-mustachian/) return an SSL handshake fatal alert (see below).

I’m running on macOS V10.14.6, Java SE Runtime Environment (build 1.7.0_51-b13), Java HotSpot 64-Bit Server VM (build 24.51-b03, mixed mode), and using IntelliJ IDEA 2019.2 Community Edition (runtime version 11.0.3+12-b304.10 x86_64).

Can you please suggest changes I should make to this code or my operating environment to address this fatal alert message?

Thanks in advance for your help.

Simple Web Scraper:

import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;


public class SimpleWebScraper {
    public static void main(String[] args)
    {
        // Determine the number of command line arguments passed to the program...
        int numberOfCommandLineArguments = args.length;

        if (numberOfCommandLineArguments < 1) {
            System.out.println("\nYou failed to include the URL for the web site you want scraped in the command line.\n");
        } else {
            // Initialize the string that holds the URL for the web site we want to scrape
            String webSiteURL = args[0];

            System.out.printf("\nTrying to retrieve the title from the URL: %s\n", webSiteURL);

            try {
                // Create a document object and use JSoup to fetch the website
                Document webSiteDocument = Jsoup.connect(webSiteURL).get();

                // Use JSoup's title() method to fetch the title
                System.out.printf("\tTitle: %s\n\n", webSiteDocument.title());

            // In case of any IO errors, we want the messages written to the console
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

Fatal Alert Message:

Trying to retrieve the title from the URL: https://forum.mrmoneymustache.com/ask-a-mustachian/
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1959)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1077)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:746)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:722)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:306)
    at org.jsoup.helper.HttpConnection.get(HttpConnection.java:295)
    at SimpleWebScraper.main(SimpleWebScraper.java:22)

Process finished with exit code 0

Upvotes: 0

Views: 72

Answers (1)

Ciro Pinto-Coelho
Ciro Pinto-Coelho

Reputation: 51

I updated the JDK to v1.8, changed the required macOS environment variables to point to the V1.8 of the JDK, and the set the parameters for the IntelliJ IDEA IDE to use the V1.8 JDK. Together, these changes eliminated the error message and the code I included in my questions now functions as designed. :-)

Upvotes: 1

Related Questions