CODEWITHSUNDEEP
python-2.7exceptionweb-scrapingbeautifulsouppython-requests
pb_ng
pb_ng

Reputation: 361

Connection Aborted Error 10060

Using Python 2.7

I am trying to run the following program but it is returning a Connection Aborted 10060 error

import requests
from bs4 import BeautifulSoup

def re_crawler(pages):
    page = 1
    while page <= pages:
        url = 'http://www.digitalmarketplace.service.gov.uk/g-cloud/search?page=' + str(page) + '&lot=saas'
        code = requests.get(url)
        text = code.text
        soup = BeautifulSoup(text)
        for link in soup.select('div.search-result h2.search-result-title a'):
             href = "http://www.digitalmarketplace.service.gov.uk" + link.get('href')
             print (href)
        page += 1

re_crawler(2)

Upon running this program, I am getting the following output

Traceback (most recent call last):
  File "C:/Users/PB/PycharmProjects/crawler/SaaS UK.py", line 18, in <module>
    re_crawler(2)
  File "C:/Users/PB/PycharmProjects/crawler/SaaS UK.py", line 10, in re_crawler
    code = requests.get(url)
  File "C:\Python27\lib\requests\api.py", line 68, in get
    return request('get', url, **kwargs)
  File "C:\Python27\lib\requests\api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\requests\sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\requests\adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

Process finished with exit code 1

Kindly advise what this error is about and how to resolve it

Many thanks

Upvotes: 1

Views: 3720

Answers (1)

Arthur Burkhardt
Arthur Burkhardt

Reputation: 700

Try https:// instead of http://. Seems to solve the problem on my side.

try:
    r = requests.get('http://www.digitalmarketplace.service.gov.uk', timeout=1)
except:
    print("http connection failed, trying https now")
    r = requests.get('https://www.digitalmarketplace.service.gov.uk', timeout=1)
print('{} ({})'.format(r.url, r.status_code))

http connection failed, trying https now

https://www.digitalmarketplace.service.gov.uk/ (200)

Upvotes: 1

Related Questions