CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
mad
mad

Reputation: 37

extracting attribute value from html

import requests
from bs4 import BeautifulSoup

r = requests.get("http://www.nepalstock.com.np/marketdepthofcompany/238")
soup = BeautifulSoup(r.content,"lxml")
value_select = soup.select_one("select.form-control")
for val in value_select.find("option")[1:]:
    n = val['value']
    print (n)

Why does the above code prints out text from the html not the attribute value?

Upvotes: 1

Views: 60

Answers (1)

Padraic Cunningham
Padraic Cunningham

Reputation: 180391

Because find returns a single tag, you need find_all:

for val in value_select.find_all("option")[1:]:
    n = val['value']
    print (n)

Or use a css selector to skip the first option:

for val in value_select.select("option + option"):

Both work the same:

In [1]: import requests

In [2]: from bs4 import BeautifulSoup

In [3]: r = requests.get("http://www.nepalstock.com.np/marketdepthofcompany/238")

In [4]: soup = BeautifulSoup(r.content,"lxml")

In [5]: value_select = soup.select_one("select.form-control")

In [6]: for val in value_select.find_all("option")[1:]:
   ...:         n = val['value']
   ...:         print (n)
   ...:     
ACEDBL
ACEDPO
ADBL
AHPC
ALDBL
ALDBLP
ALICL
ALICLP
APEX
APEXPO
API
ARDBL
ARDBLP
ARUN
ARUNPO
AVU
BARUN
BBBLNP
...................................

In [7]: for val in value_select.select("option + option"):
   ...:         n = val['value']
   ...:         print (n)
   ...:     
ACEDBL
ACEDPO
ADBL
AHPC
ALDBL
ALDBLP
ALICL
ALICLP
APEX
APEXPO
API
ARDBL
ARDBLP
ARUN
ARUNPO
AVU
BARUN
BBBLNP
..........................

To get the options where the values are ints, select the id StockSymbol_Select2, there are more than one select.form-control so you need to specify which one exactly:

import requests
from bs4 import BeautifulSoup

r = requests.get("http://www.nepalstock.com.np/marketdepthofcompany/238")
soup = BeautifulSoup(r.content,"lxml")
value_select = soup.select_one("#StockSymbol_Select2")
for val in value_select.select("option + option"):
    print (val["value"])

That will give you what you want:

In [13]: value_select = soup.select_one("#StockSymbol_Select2")

In [14]: for val in value_select.select("option + option"):
   ....:         print (val["value"])
   ....:     
216
294
397
360
406
660
385
599
262
666
697
..............................

Upvotes: 2

Related Questions