CODEWITHSUNDEEP
pythonpython-2.7urlwebweb-crawler
RIshu
RIshu

Reputation: 217

Check Response Time, Length and URL status(get code 200,404 etc ) using Python

I have multiple URL's for that I want to check the status like URL Status(return code like 200,404 etc). Below is the code that will getcode. I want to check response time, length from the same code. Can anyone help me to modify below code to find response time,length and getcode for multiple urls.

import socket
from urllib2 import urlopen, URLError, HTTPError

import urllib
socket.setdefaulttimeout( 23 )  # timeout in seconds
#print "---------URL----------", " ---Status Code---"
url='https://www.google.com'

    try :
      response = urlopen( url )
    except HTTPError, e:
        print 'The server couldn\'t fulfill the request. Reason:', str(e.code)
        #Want to get code for that but its not showing

    except URLError, e:
        print 'We failed to reach a server. Reason:', str(e.reasonse)
        #Want to get code for that but its not showing


    else :

        code=urllib.urlopen(url).getcode()
        **#here getcode is working
        print url,"-------->", code
        #print 'got response!'

Upvotes: 2

Views: 3707

Answers (2)

niemmi
niemmi

Reputation: 17263

Following code will print the response status and headers for all successful requests followed by the time it took to complete it:

import socket
import time
from urllib2 import urlopen

URLS = ['http://www.google.com', 'http://www.yahoo.com']
socket.setdefaulttimeout( 23 )  # timeout in seconds

for url in URLS:
    print 'Checking URL {0}'.format(url)
    start = time.clock()
    try:
        response = urlopen(url)
        print response.getcode()
        print response.info()
    except StandardError, e:
        print 'Check failed: {0}'.format(e)

    print 'took {0}s\n'.format(time.clock() - start)

Output:

Checking URL http://www.google.com
200
Date: Wed, 23 Mar 2016 05:12:30 GMT
Expires: -1
...

took 0.21013614795s

Checking URL http://www.yahoo.com
200
Date: Wed, 23 Mar 2016 05:12:31 GMT
...

took 0.983750700381s

As @contracode suggested you might want to take a look at requests which is the most popular HTTP library for Python.

Upvotes: 1

contracode
contracode

Reputation: 405

I think your use case is better-suited to the Requests library. Install that with pip install requests and check out the documentation. You should be up and running in no time!

Upvotes: 1

Related Questions