CODEWITHSUNDEEP
phphtmlcurl
user2730570
user2730570

Reputation: 11

Bypassing loading screen when getting HTML content with curl

We are using curl to get a response from a third-party webserver. there's a code snippet:

 $url = "https://book.some-site.com/cgi-bin/booking-form.cgi";

 $uagent = "Opera/9.80 (Windows NT 6.1; WOW64) Presto/2.12.388    Version/12.14";

 $ch = curl_init( $url );

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);        
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");     
curl_setopt($ch, CURLOPT_USERAGENT, $uagent);  
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0); 
curl_setopt($ch, CURLOPT_TIMEOUT, 0);          
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);

$content = curl_exec( $ch );
$err     = curl_errno( $ch );
$errmsg  = curl_error( $ch );
$header  = curl_getinfo( $ch );
curl_close( $ch );

Everything is working fine till we hit a loading screen on one of the pages. We get the following response from the webserver "...

We are processing your request...
Your search results will display shortly.

" which is a loading/waiting screen. after that we get nothing. When working in a browser after the loading screen the actual response is displayed. Any ideas how to get the actual response nad bypass the loading screen?

Thanks in advance.

Upvotes: 1

Views: 922

Answers (1)

Hayden Schiff
Hayden Schiff

Reputation: 3330

Usually, when a website has a loading screen, then shows the results without redirecting you to a new page, it means they loaded the results via Ajax. So the HTML page loads with nothing but a "hey, it's loading" message, and then some JavaScript runs that downloads the actual content from a different page. You'll need to investigate their JS code and then load the page that they load via Ajax.

You might look into enabling "logging XMLHttpRequests" in your web browser's developer tools to make it easier to figure out what page they're loading via Ajax.

Upvotes: 1

Related Questions