CODEWITHSUNDEEP
phpweb-scrapingfile-get-contents
Vineet Sharma
Vineet Sharma

Reputation: 221

Efficient way to access multiple URLs using file_get_contents() if they are from same domain?

The question title basically explains it all. I have to collect data from multiple pages of a single domain periodically.

I can use file_get_contents() in a loop but that will take a long time. Is there anything that I can do to speed up the process if all the pages are from a single domain?

Update: I am thinking something like not looking up the domain again and again etc.

Thanks.

Upvotes: 1

Views: 418

Answers (1)

GullDe
GullDe

Reputation: 390

You can use curl_multi_init or use threads assuming you don't need the results of the previous request to process the next one. That should speed up processing quite a bit.

Upvotes: 1

Related Questions