CODEWITHSUNDEEP
javaandroidhtmljsouphtml-parsing
Anand Kumar
Anand Kumar

Reputation: 169

HTML of webpage and Jsoup response are not the same

Connection.Response loginResponse = Jsoup.connect("https://users.premierleague.com/accounts/login/")
                    .userAgent("Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)")
                    .data("csrfmiddlewaretoken", csrfmiddlewaretokenValue)
                    .data("login", "[email protected]")
                    .data("password", "****")
                    .data("app", "plfpl-web")
                    .data("redirect_uri", "https://fantasy.premierleague.com/a/team/my")
                    .timeout(600000)
                    .cookie("auth", "token")
                    .method(Connection.Method.POST)
                    .execute();

I'm using the above code to login to a website using Jsoup in my Android app. But the actual HTML of the webpage after login and the response from Jsoup are different. The main body of the webpage is not found in the Jsoup response. I guess it's some security feature. I've tried different user agents and large timeout values but without luck. What shall I do to solve the problem?

Upvotes: 0

Views: 311

Answers (1)

StanislavL
StanislavL

Reputation: 57421

In case of SPA (Single Page Application) DOM is not loaded but generated on fly. So page contains just a DOM root - e.g. a div and all the rest is built by JavaScript.

E.g. React based web application.

JSoup won't help in this case.

To solve check what is sent by browser and try to imitate the requests parsing responses.

Upvotes: 1

Related Questions