CODEWITHSUNDEEP
pythonweb-scraping
Notgivinit
Notgivinit

Reputation: 67

Website error after scraping

I made a simple scraper that accesses an album, and scrapes lyrics for each song from azlyrics.com.

After about an hour of working, the website crashed, with an error:

Chrome:

www.azlyrics.com didn’t send any data. ERR_EMPTY_RESPONSE

Tor, firefox, waterfox:

The connection was reset The connection to the server was reset while the page was loading.

It's the same for all devices on my home network. If I use mobile data to access it via my phone it works fine.

I tried fixing it with ipconfig /release /renew, but it didn't work. I'm at a loss for what else I could do or why it even happened. Any help is greatly appreciated.

Upvotes: 2

Views: 384

Answers (1)

Marat
Marat

Reputation: 15738

Apparently your IP was banned by the website for suspicious activity. There are couple ways around that:

  • talk to website owners. This is the most straightforward and nicest way
  • change your IP, e.g. by connecting though a pool of public proxies or Tor. This is a little bit dirty and it is not so robust, e.g. you can be banned by user-agent or some other properties of your scraper.

Upvotes: 4

Related Questions