Reputation: 2813
I am trying to write a crawler in a bash to download images from bing. Everything works fine except the fact that bing loads only 30 images in the beginning (uses infinite scrolling) and hence I am not able to download more than 30 images using wget.
Any workaround to download more than 30 images from bing images at once.?
-- Thanks for any help. :)
Upvotes: 0
Views: 2884
Reputation: 218847
Bing has an API, perhaps that would work better than screen scraping? It would certainly be more allowed/supported by Bing than screen scraping, and less prone to breaking changes as they change their offerings over time.
If you're really set on the screen scraping for whatever reason, you might try loading the page with debugging (such as with Firebug) and watch what happens when the infinite scroll is triggered. Then try to manually trigger it with code.
Upvotes: 1