CODEWITHSUNDEEP
javaweb-scrapinghtmlunit
Henrique Fernandes Cipriano
Henrique Fernandes Cipriano

Reputation: 585

How to get Output (Image or PDF) from a Web Page using Htmlunit (Including external images and css)

WebClient webClient = new WebClient();
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
        webClient.getOptions().setJavaScriptEnabled(false);
        webClient.getOptions().setCssEnabled(true);
        webClient.getOptions().setDownloadImages(true);


        Page page = webClient.getPage("http://www.example.com");
        WebResponse response = page.getWebResponse();

how should I use the response to render an image or pdf?

I have found several questions and several libraries that "do this".

But I could not in any of them, an effective way of through an http request, transform the return into an image or pdf, downloading and including all css and images external links automatically.

I don't care about the format (PNG or PDF), as long as the output is similar to what the browser renders.

Upvotes: 1

Views: 1036

Answers (2)

ksahin
ksahin

Reputation: 226

You have two options :

1) Keep using htmlUnit, get the response's HTML using 

page.asXml();

And then using a third party like iTextRenderer : https://stackoverflow.com/a/17826418/3650731

2) Or you could use Headless Chrome and take a screenshot with it. The ouput should be better in most cases than with htmlUnit + iText because htmlUnit does not render Javascript / modern Html5 /css3 as good as Headless chrome.

Here is a blog post I wrote about Headless Chrome with Java : https://ksah.in/introduction-to-chrome-headless/

And here is a SO answer about how to take a screenshot with Headless chrome : https://stackoverflow.com/a/43388989/3650731

Upvotes: 1

RBRi
RBRi

Reputation: 2889

HtmlUnit is a headless browser, in the end this means there is no page rendering/layouting at all.

Depending on your use case you can write an XHtml dump of your page to the disk and take care to dump the required dependencies also. That is what i'm doing for WETATOR (www.wetator.org). The source code is freely available if you like you can have a look (maybe you can start at https://wetator.repositoryhosting.com/trac/wetator_wetator/browser/trunk/wetator/src/org/wetator/backend/htmlunit/XHtmlOutputter.java)

Upvotes: 0

Related Questions