CODEWITHSUNDEEP
pythongeventurlopen
yossi
yossi

Reputation: 13315

why python gevent is slower than serial ?

i am using gevent to preform concurrent download.
based on this example this is the code:

import gevent
from gevent import monkey

urls = ['https://www.djangoproject.com/','http://www.nytimes.com/','http://www.microsoft.com']
monkey.patch_all()

import urllib2
from datetime import datetime

def print_head(url):
    print ('Starting %s' % url)
    data = urllib2.urlopen(url).read()   
    print ('%s: %s bytes: %r' % (url, len(data), data[:50]))

startTime = datetime.now()
jobs = [gevent.spawn(print_head, url) for url in urls]
gevent.joinall(jobs)
totalTime = datetime.now() - startTime
print "Total time: %s" % totalTime

my problem is that the above code takes much longer than the serial version and in most cases it is timed out. here is the serial version which is much faster:

import urllib2
from datetime import datetime

urls = ['https://www.djangoproject.com/','http://www.nytimes.com/','http://www.microsoft.com']

def print_head(url):
    print ('Starting %s' % url)
    data = urllib2.urlopen(url).read()
    print ('%s: %s bytes: %r' % (url, len(data), data[:50]))

startTime = datetime.now()    
for url in urls:
    try:
        print_head(url)
    except:
        print 'ops EXCEPTION :('

totalTime = datetime.now() - startTime
print "Total time: %s" % totalTime

Upvotes: 4

Views: 1942

Answers (1)

yossi
yossi

Reputation: 13315

ok
the problem was an old gevent package.
i just uninstalled the old one and installed the new one from here as @reclosedev pointed out.
and it is now working fine.

Upvotes: 1

Related Questions