Python scrape data from "div: class

Question

I'm trying to scrape some financial data (the Key data) that is between <div> </div> tags. I want to then put it into an excel file that already has other scraped data in it. The HTML looks like

<div class="tabElemNoBor resp  overfH"><div class="clearfix">
    <div class="tabTitleWhite"><div class="tabTitleLeftWhite"><b>Key data</b></div></div>
</div><!-- inner td --><div class="std_txt th_inner " style="padding-top:10px">        <style>
        .fCCR { clear: both; line-height: 22px;}
        .fCCR:nth-of-type(even) {background-color: #F6F6F6}
        .fCCR:nth-of-type(odd) {background-color: #FFF}
        .fCCT {display: inline-block;padding-left: 7px;}
        .fCCV {display: inline-block;width:120px;float: right;text-align: right;padding-right: 7px;}  
        </style>
        <div class="fCC">
        <div class="fCCR"><div class="fCCT">Capitalization (USD)</div><div class="fCCV">45 016 163 291</div></div><div class="fCCR"><div class="fCCT">Net sales (USD)</div><div class="fCCV">27 753 973 000</div></div><div class="fCCR"><div class="fCCT">Number of employees</div><div class="fCCV">143 000</div></div><div class="fCCR"><div class="fCCT">Sales / Employee (USD)</div><div class="fCCV">194 084</div></div><div class="fCCR"><div class="fCCT">Free-Float</div><div class="fCCV">99,8%</div></div><div class="fCCR"><div class="fCCT">Free-Float capitalization (USD)</div><div class="fCCV">44 932 273 933</div></div><div class="fCCR"><div class="fCCT">Avg. Exchange 20 sessions (USD)</div><div class="fCCV">380 055 475</div></div><div class="fCCR"><div class="fCCT">Average Daily Capital Traded</div><div class="fCCV">0,84%</div></div></div>
</div></div>

I was thinking I could maybe use BeautifulSoup but I'm not really sure what I would need to do.

I have tried the following:

for value in elem:
    print (value.text)

to try and get the top div tag and then just print everything that is between it, but it doesn't seem to work.

EDIT: URL https://www.marketscreener.com/DOLLAR-GENERAL-CORPORATIO-5699818/financials/

Any help appreciated.

Thanks

Answer 1

Print the table to screen and save it to csv:

import csv
import requests
from bs4 import BeautifulSoup

url = 'https://www.marketscreener.com/DOLLAR-GENERAL-CORPORATIO-5699818/financials/'

soup = BeautifulSoup(requests.get(url).content, 'html.parser')

all_data = []
for div in soup.select('div.fCCR'):
    all_data.append( div.get_text(strip=True, separator='|').split('|') )

all_data.insert(0, [div.find_previous('b').text])

# pretty print all data:
print(*all_data[0])
print('-' * 80)
for row in all_data[1:]:
    print(('{:<45}'*2).format(*row))

# save it to csv:
with open('data.csv', 'w', newline='') as csvfile:
    spamwriter = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
    for row in all_data:
        spamwriter.writerow(row)

Prints:

Key data
--------------------------------------------------------------------------------
Capitalization (USD)                         45 016 163 291                               
Net sales (USD)                              27 753 973 000                               
Number of employees                          143 000                                      
Sales / Employee (USD)                       194 084                                      
Free-Float                                   99,8%                                        
Free-Float capitalization (USD)              44 932 273 933                               
Avg. Exchange 20 sessions (USD)              380 055 475                                  
Average Daily Capital Traded                 0,84%                                        

and the data.csv file in LibreOffice`: