CODEWITHSUNDEEP
pythonhtmlweb-scrapingbeautifulsoup
The_Bandit
The_Bandit

Reputation: 73

BeautifulSoup Web Scrape Running but Not Printing

Mega new coder here as I learned Web scraping yesterday. I'm attempting to scrape a site with the following html code:

<div id="db_detail_colorways">
    <a class="db_colorway_line" href="database_detail_colorway.php? 
    ID=11240&amp;table_name=glasses">
        <div class="db_colorway_line_image"><img 
    src="database/Sport/small/BallisticNewMFrameStrike_MatteBlack_Clear.jpg"/>. 
        </div>. 
        <div class="grid_4" style="overflow:hidden;">Matte Black</div><div 
        class="grid_3">Clear</div><div class="grid_1">$133</div><div 
        class="grid_2">OO9060-01</div><div class="clear"></div></a><a

There are 4 total items being scraped. The goal is to print the attribute stored in <div class="grid_4" the code should loop over the 4 items being scraped, so for the html code provided, the first being displayed is "Matte Black" Here is my code:

for frame_colors in soup.find_all('a', class_ = 'db_colorway_line'):
    all_frame_colors = frame_colors.find_all('div', class_ = 'grid_4').text
    print(all_frame_colors)

Basically the code runs correctly and everything else thus far has run correctly in this jupyter notebook, but this runs and does not print out anything. I'm thinking it's a syntax error, but I could be wrong. Hopefully this makes sense. Can anyone help? Thanks!

Upvotes: 0

Views: 93

Answers (1)

imxitiz
imxitiz

Reputation: 3987

You are treating a list of elements as a single element

frame_colors.find_all('div', class_ = 'grid_4').text

You can run loop of all_frame_colors and get the text from there like this:

for frame_colors in soup.find_all('a', class_ = 'db_colorway_line'):
    all_frame_colors = frame_colors.find_all('div', class_ = 'grid_4')
    for af in all_frame_colors:
        print(af.text)

If it solves you problem then don't forget to mark this as an answer!

Upvotes: 2

Related Questions