CODEWITHSUNDEEP
pythonurlopen
user1371662
user1371662

Reputation: 103

python urllib2.urlopen(url) process block

I am using urllib2.urlopen() and my process is getting blocked

I am aware that urllib2.urlopen() has default timeout.

How to make the call unblockable?

The backtrace is

(gdb) bt 
#0 0x0000003c6200dc35 in recv () from /lib64/libpthread.so.0 
#1 0x00002b88add08137 in ?? () from /usr/lib64/python2.6/lib-dynload/_socketmodule.so 
#2 0x00002b88add0830e in ?? () from /usr/lib64/python2.6/lib-dynload/_socketmodule.so 
#3 0x000000310b2d8e19 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.6.so.1.0

Upvotes: 0

Views: 4340

Answers (2)

Mikko Ohtamaa
Mikko Ohtamaa

Reputation: 83676

If your problem is that you need to urllib to finish reading

read() operation is blocking operation in Python.

If you want to create asynchronous requests

If your problem is need to set timeout

Again, use requests library as mentioned above.

Upvotes: 3

Kimvais
Kimvais

Reputation: 39618

You can try using strace (or similar) tool to figure out what the actual system call is that is blocking your python script, e.g on linux: $ strace python yourscript.py

yourscript.py:

from urllib2 import urlopen
urlopen("http://somesite.local/foobar.html")

$ strace python yourscript.py

... lots of system call stripped ...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("127.0.0.1")}, 16

Upvotes: 2

Related Questions