CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Anuj Masand
Anuj Masand

Reputation: 359

Web Scraping, How to extract data from two same tags using bs4 in python

I am using bs4 with python and trying to fetch data from a web page. Link I used inspect element over the info i want, but both have same tag,class. 

             <a class="cell__value" data-tracker-action="click" data-tracker-label="information_technology.01" href="/markets/sectors/information-technology">
             Information Technology
            </a>
           </div>
           <div class="cell__return">
            <div class="cell__label">
             % Price Change
            </div>
            <div class="cell__value" data-type="better">
             +0.05%
            </div>
           </div>
          </div>
          <div class="cell">
           <div class="cell__name">
            <div class="cell__label">
             Industry
            </div>
            <a class="cell__value" data-tracker-action="click" data-tracker-label="information_technology.02" href="/markets/sectors/information-technology">
             Software &amp; Services
            </a>
           </div>
           <div class="cell__return">
            <div class="cell__label">
             % Price Change
            </div>
            <div class="cell__value" data-type="worse">
             -0.04%
            </div>
           </div>
          </div>
         </div>

I am doing it this way:

sect= soup.find("a",{"data-tracker-label":"information_technology.01"})
print sect.text
sect_per= soup.find("div",{"data-type":"worse"or"better"})
print sect_per.text
ind=soup.find("a",{"data-tracker-label":"information_technology.02"})
print ind.text
ind_per=soup.find("div",{"div",{"data-type":"worse"or"better"})
print ind_per

both print ind_per and print ind_per are giving me same result because of same class and tag

i need to extract +0.05% and -0.04% respectively.

Please suggest me way to do it.

Upvotes: 2

Views: 1949

Answers (3)

KARAN KAJROLKAR
KARAN KAJROLKAR

Reputation: 1

<p class="sort-num_votes-visible">
<span class="text-muted">Votes:</span>
**<span data-value="2333089" name="nv">2,333,089</span>**
<span class="ghost">|</span> <span class="text-muted">Gross:</span>
**<span data-value="28,341,469" name="nv">$28.34M</span>**
</p>

"I want to get a voting count and gross for movies but both have the same name is "nv" so we use indexing for these"

 vote_mov=container.findAll("span",{"name":"nv"})
    vote=vote_mov[0].text
    
    gross_mov=container.findAll("span",{"name":"nv"})
    gross=gross_mov[1].text

"Here it 1st get voting and then grossenter image description here

Upvotes: 0

Zroq
Zroq

Reputation: 8392

soup = BeautifulSoup(example, "html.parser")

for cell in soup.find_all("div", class_="cell"): 
    name = ""
    namecell = cell.find("a", class_="cell__value", text=True)
    if namecell is not None:
         name = namecell.get_text(strip=True)
    price_chage = cell.find("div", class_="cell__value").get_text(strip=True)
    print ( "%s: Price Change:  %s" % (name, price_chage))

Which outputs:

Information Technology: Price Change: +0.05%

Software & Services: Price Change: -0.04%

You can save that values for further processing.

Upvotes: 2

falsetru
falsetru

Reputation: 369054

or returns the left operand if the left operand is truth value (non-empty string for string):

>>> "worse" or "better"
'worse'

So, the following line:

ind_per = soup.find("div",{"div",{"data-type":"worse" or "better"})

is basically doing same with:

ind_per = soup.find("div",{"div",{"data-type":"worse"})

You need to query them separately:

ind_per = soup.find("div",{"div",{"data-type": "worse"})
print ind_per
ind_per = soup.find("div",{"div",{"data-type": "better"})
print ind_per

or using for loop:

for data_type in ('worse', 'better'):
    ind_per = soup.find("div",{"div",{"data-type": data_type})
    print ind_per

Upvotes: 0

Related Questions