CODEWITHSUNDEEP
pythonpython-requestshtmlurllib
Felipe De Carli
Felipe De Carli

Reputation: 37

What exactly requests function does?

So I`m trying to send a request to a webpage and read its response. I did a code that compares the request and the page, and I cant get the same page text. Am I using "requests" correctly? I really think that I misunderstand how requests function works and what it does. Can someone help me please?

 import requests
 import urllib

 def search():
         pr = {'q':'pink'}
         r = requests.get('http://stackoverflow.com/search',params=pr)
         returntext = r.text
         urllibtest(returntext)

 def urllibtest(returntext):
         connection = urllib.urlopen("http://stackoverflow.com/search?q=pink")
         output = connection.read()
         connection.close()
         if output == returntext:
                 print("ITS THE SAME PAGE")
         else:
                 print("ITS NOT THE SAME PAGE")

 search()

Upvotes: 2

Views: 61

Answers (1)

wim
wim

Reputation: 362945

First of all, there is no good reason to expect two different stack overflow searches to return the exact same response anyway.

There is one logical difference here too, requests automatically decodes the output for you:

>>> type(output)
str
>>> type(r.text)
unicode

You can use the content instead if you don't want it decoded, and use a more predictable source to see the same content returned - for example:

>>> r1 = urllib.urlopen('http://httpbin.org').read()
>>> r2 = requests.get('http://httpbin.org').content
>>> r1 == r2
True

Upvotes: 1

Related Questions