CODEWITHSUNDEEP
httpcookiesscreen-scraping
user967007
user967007

Reputation:

how to handle cookies while scraping

I'm in the middle of making a small scraping utility which is designed to run as quickly as possible using multiple http connections to the server.

How does one handle cookies in this situation..

For example if the first connection scrapes the page for links, and the server sets the cookie to something,, wouldn't firing off additional connections cause the cookies to be out of sync ?

Upvotes: 1

Views: 166

Answers (1)

huntsfromshadow
huntsfromshadow

Reputation: 1085

The answer is it really depends on the server.

If the server changes the cookie with each and every request yes it will throw off the cookie. What exactly this does again depend on the server.

If say the cookie is just set once at login, then it wouldn't really matter.

Upvotes: 1

Related Questions