CODEWITHSUNDEEP
javascriptjqueryajaxperl
AndyMac
AndyMac

Reputation: 830

Perl get with delayed javascript response

I'm trying to parse my professor's web page at school to get an automated update when she changes homework problems and ran into something I haven't had to do before: A delayed response because of ajax / jquery.

So in perl, I'm just using LWP::Simple and the get function to grab the html of the page. The problem is, she's using jquery to dynamically update the page. With my "get" I get the page before the ajax / jquery has finished loading.

Is there a way to request the page, but get the final output? Even if I have to just use a fixed delay (say, 10s), that'd be fine. I just want this to run once a day from my school account so it can e-mail me to tell me the assignments have been updated. School is behind a VPN, and checking in every day is a hassle I thought I could remedy with some quick scripting :).

Upvotes: 1

Views: 409

Answers (1)

Mark Stosberg
Mark Stosberg

Reputation: 13411

Some have recommended WWW::Mechanize::Firefox, which might not be an option if you are running the script from a headless server. Another option is read her JavaScript/jQuery code to see what it's doing. Your Perl code could likely make a direct HTTP request for the for the same resource that the AJAX code is doing. By understanding how the jQuery AJAX request modifies the page, you may be able to sufficiently update your Perl to reflect it.

The mirror method from LWP::Simple may be useful for the AJAX request if not the primary request. It does a "conditional GET", which means it only downloads the body if something has changed, which speeds things up a bit.

Upvotes: 2

Related Questions