CODEWITHSUNDEEP
pythonbeautifulsouppython-requests
ankush reddy
ankush reddy

Reputation: 501

How to get a specific value using python requests?

I am trying to parse this site https://www.5dimes.eu/ and extract a specific value from it. as shown in the figure but it is not at all the getting populated with the get request. But I am able to see it when I open the browser. enter image description here

I am just getting the output as shown below.

import requests
import json
from bs4 import BeautifulSoup
import time
t = requests.get("https://www.5dimes.eu/")
soup = BeautifulSoup(t.content)
print(soup.find_all('td'))


<td align="center" colspan="3"><input class="login" id="customerID" maxlength="50" name="customerID" type="text"/></td>
</tr>
<tr valign="middle">
<td align="center" colspan="3" height="20" valign="bottom">Password</td>
</tr>
<tr>
<td colspan="3" height="1"><img alt="5Dimes Sportsbook - Casino - Racebook - Lottery - Poker" border="0" height="1" src="images/trans.gif" title="5Dimes Sportsbook - Casino - Racebook - Lottery - Poker" width="1"/></td>
</tr>
<tr valign="middle">
<td align="center" colspan="3"><input autocomplete="off" class="login" maxlength="50" name="password" type="password"/>
<input id="ioBB" name="ioBB" type="hidden"/>
</td>

Upvotes: 1

Views: 848

Answers (2)

ankush reddy
ankush reddy

Reputation: 501

Found a solution using requests_html.

Here is the code.

import requests
import json
from bs4 import BeautifulSoup
import time


from requests_html import HTMLSession

session = HTMLSession()

r = session.get("https://www.5dimes.eu/", headers={'User-Agent': 'Mozilla/5.0'})
r.html.render()
print(r.html.html)

Upvotes: 0

αԋɱҽԃ αмєяιcαη
αԋɱҽԃ αмєяιcαη

Reputation: 11515

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from bs4 import BeautifulSoup

options = Options()
options.add_argument('--headless')

driver = webdriver.Firefox(options=options)
driver.get("https://www.5dimes.eu/")

soup = BeautifulSoup(driver.page_source, 'html.parser')

val = soup.find("input", {'id': 'ioBB'}).get("value")

print(val)
driver.quit()

enter image description here

Upvotes: 2

Related Questions