CODEWITHSUNDEEP
pythonpython-requests
gapi31
gapi31

Reputation: 27

Python requests - using twitter search

I am trying to use requests to get data from twitter but when i run my code i get this error: simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This is my code so far:

import requests

url = 'https://twitter.com/search?q=memes&src=typed_query'
results = requests.get(url)
better_results = results.json()
better_results['results'][1]['text'].encode('utf-8')
print(better_results)

Upvotes: 1

Views: 550

Answers (1)

alexzander
alexzander

Reputation: 1875

because you are making a request to a dynamic website.

when we are making a request to a dynamic website we must render the html first in order to receive all the content that we were expecting to receive.

just making the request is not enough.

other libraries such as requests_html render the html and javascript in background using a lite browser.

you can try this code:

# pip install requests_html
from requests_html import HTMLSession

url = 'https://twitter.com/search?q=memes&src=typed_query'

session = HTMLSession()
response = session.get(url)

# rendering part
response.html.render(timeout=20)

better_results = response.json()
better_results['results'][1]['text'].encode('utf-8')
print(better_results)

Upvotes: 1

Related Questions