CODEWITHSUNDEEP
javapythonhtml-parsingjsoup
Spart
Spart

Reputation: 113

How to extract source html from webpage?

I am trying to extract the html source of this page, http://www.fxstreet.com/rates-charts/currency-rates/

I want what I see when I save the page from chrome as a .html file.

I tried to do this in java, using bufferedreader, and then using jsoup. I also tried to do it in python, however I keep getting the following message:

"This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser."

The end goal is to extract the values in the main table.

Upvotes: 1

Views: 2268

Answers (2)

JohnnyQ
JohnnyQ

Reputation: 543

Just to extract the main table can be easily done using Jsoup

here's a method that will take all the content from the main table on the page

public void parse(){
        try{

        Document doc = Jsoup.connect("http://www.fxstreet.com/rates-charts/currency-rates/").get();
        Element content = doc.getElementById("ddlPairsChoose");
        Elements table = doc.getElementsByClass("applet-content");      

        System.out.print(table);

        }

        catch(Exception e){

            System.out.print("error --> " + e);
        }       
    }

It prints out the table on the page

Upvotes: 1

Xeon
Xeon

Reputation: 5989

Try using HtmlUnit and setting setJavascriptEnabled(true)

Look also at: this and this

JSoup isn't headless browser to execute Javascript so you must choose other library to get the page and then you can use JSoup to parse it.

Upvotes: 4

Related Questions