CODEWITHSUNDEEP
pythonjsondictionaryweb-scraping
Sameh Weangy
Sameh Weangy

Reputation: 55

Json , scrape into web page - python

i am scraping into certain webpage using requests and beautifulsoup libs in python

so i got the element that i want in this simple code 

<script>
data = {'user':{'id':1,'name':'joe','age':18,'email':'[email protected]'}}
</script>

so i want to get the email value in variable but the whole element comes back into list and when i specify the text of that tag i can't get it into json it gives me errors in the columns so any idea ? i'll appreciate any help

Upvotes: 2

Views: 271

Answers (1)

Druta Ruslan
Druta Ruslan

Reputation: 7412

Something simple, maybe will help you.

import json
from bs4 import BeautifulSoup

html = """
<script>
data = {'user':{'id':1,'name':'joe','age':18,'email':'[email protected]'}}
</script>
"""

soup = BeautifulSoup(html, 'html.parser')
# slices [7:] mean that we ignore the `data = `
# and replace the single quotes to double quotes for json.loads()
json_data = json.loads(soup.find('script').text.strip()[7:].replace("'", '"'))
print(json_data)
print(type(json_data))

Output

{'user': {'id': 1, 'name': 'joe', 'age': 18, 'email': '[email protected]'}}
<class 'dict'>

Upvotes: 1

Related Questions