CODEWITHSUNDEEP
javahtmlunit
bharathi
bharathi

Reputation: 6271

WebClient Proxy Authorization error with HTMLUnit

I am new to HTMLUnit. I have tried some sample example to get better understanding of the HTMLUnit.

I am using HTMLUnit 2.15 version.

Code:

    final WebClient webClient = new WebClient(BrowserVersion.getDefault(), "***.***.com", portNo);

    // set proxy username and password
    final DefaultCredentialsProvider credentialsProvider = (DefaultCredentialsProvider) webClient.getCredentialsProvider();
    credentialsProvider.addCredentials(username, password);
    //credentialsProvider.addNTLMCredentials(username, password, null, -1, "localhost", "domain");
    HtmlPage page = null;
    try {
        page = webClient.getPage("http://htmlunit.sourceforge.net");
    } catch (FailingHttpStatusCodeException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (MalformedURLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    System.out.println("Google" + page.getTitleText());

When I run this code I am getting the below error message.

   com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 407 Proxy Authorization Required for http://htmlunit.sourceforge.net/
at com.gargoylesoftware.htmlunit.WebClient.throwFailingHttpStatusCodeExceptionIfNecessary(WebClient.java:527)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:352)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:407)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:392)
at Main.main(Main.java:46)
Exception in thread "main" java.lang.NullPointerException
at Main.main(Main.java:58)

I do know why I am getting proxy exception after seeing the credential.

Upvotes: 3

Views: 3716

Answers (3)

Linus
Linus

Reputation: 950

I recently did the same.For me the below worked. This uses the DefaultCredentialsProvider on the WebClient.

DefaultCredentialsProvider credentialsProvider = new DefaultCredentialsProvider();
        ProxyConfig proxyConfig = new ProxyConfig(proxy.getAddress(), proxy.getPort());
            try (WebClient client = new WebClient()) {
            if (!proxy.getUserName().isEmpty()) {
                credentialsProvider.addCredentials(proxy.getUserName(), proxy.getPassword());
                client.setCredentialsProvider(credentialsProvider);
            }
                client.getOptions().setThrowExceptionOnFailingStatusCode(false);
                client.getOptions().setThrowExceptionOnScriptError(false);
                client.getOptions().setUseInsecureSSL(true);

                /* Clearing Cache and Cookies */
                client.getCookieManager().clearCookies();
                client.getCache().clear();

                /* Setting proxy to be used */
                client.getOptions().setProxyConfig(proxyConfig);

                /* Actual Web Page LOAD */
                Page startPage = client.getPage(url);

                            /* Create ProxyStatus record */
                WebResponse response = startPage.getWebResponse();
                }catch (Exception e) {
                    e.printStackTrace();
                }

Upvotes: 0

jacobcs
jacobcs

Reputation: 569

Try setting the credentials directly on WebClient's credential provider. This works for me in version 2.18:

import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;

import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class App {

    private static final String PROXY_HOST= "***.***.com";
    private static final int PROXY_PORT=8080;
    private static final String USERNAME = "username";
    private static final String PASSWORD = "password";

    public static void main(String...strings) throws FailingHttpStatusCodeException, MalformedURLException, IOException {

        WebClient client =  new WebClient(BrowserVersion.FIREFOX_38, PROXY_HOST, PROXY_PORT);
        client.getCredentialsProvider().setCredentials(AuthScope.ANY, new NTCredentials(USERNAME, PASSWORD, "", ""));
        HtmlPage page = client.getPage("http://google.com");
        page.save(new File("google.html"));
        client.close();
    }

}

Upvotes: 1

RendezAWS
RendezAWS

Reputation: 103

You can try to use :

credentialsProvider.addNTLMCredentials(userName, password, proxyHostName, Integer.parseInt(proxyPort), workstation, domain);

Please provide ip for proxyHostName and you can get your workstation by :

workstation = System.getenv("COMPUTERNAME");

And anyways you have to use correct domain name where your NT Credentials falls in.

Upvotes: 0

Related Questions