CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
timotom
timotom

Reputation: 75

How to extract text FAST 'N FREE with BeautifulSoup

I'd to extract the text between the strong tags below:

<div class="u-flL sh-col">
<span id="shSummary">
<div class=" vi-fnf-ship  fnfvar0">
<img alt="Estimated by eBay FAST 'N FREE " src="https://ir.ebaystatic.com/rs/v/xmyxg1ubry1npie2zlpan5za3yu.png" class="vi-fnf-ship-img">
<span class="vi-fnf-ship-txt "><strong class="sh_gr_bld">FAST 'N FREE</strong></span>

I took a hit-or-miss approach on the following but no luck:

    # shippingCost = soup.find('strong', {'class':"sh_gr_bld"}).text.strip()
    #shippingCost = soup.find('div', {'class': ' vi-fnf-ship  fnfvar0'}).find('span', {'class': 'vi-fnf-ship-txt'})\
    #   .find('strong', {'class': 'sh_gr_bld'}).text
    # shippingCost = soup.find_all('strong[class="sh_gr_bld"]').text
    # shippingCost = soup.find('span', {'class': 'vi-fnf-ship-txt'}).text#.find('b', {'class': 'sh_gr_bld'}).text
    # shippingCost = soup.find('div')
    # shippingCost = soup.find_all('span', {'class': 'vi-fnf-ship-txt'}).text#.find('span').next_sibling
    # shippingCost = soup.select('.vi-fnf-ship-txt'):nth-of-type(1).text
    shippingCost = soup.select('img.vi-fnf-ship-img span.vi-fnf-ship-txt strong.sh_gr_bld').text

Thanks.

==== UPDATE

What I did was to prettify the soup but to no avail:

soup = BeautifulSoup(response.text, 'html.parser')
print(soup.prettify())

==== NEW UPDATE

I redirected the prettify output to a file.txt and saw the code. So it is in there for sure with the same tags as above.

The URL of the item is:

https://www.ebay.com/itm/Mens-Legacy-Air-Bubble-90-Running-Trainers-Shoes/323806767262?_trkparms=ispr%3D1&hash=item4b6463289e:m:m_fYF4CZiE5Q9q08V38EY5w&enc=AQAEAAACQBPxNw%2BVj6nta7CKEs3N0qWNuu8y9VA2HEw0wmPsL5MTRFTJmnuraG452Pk3WQNpsgmrIf6ePIv561MkEiJV0pbFv9zmD1JW8JOdsIntwNXTFqw1McvYYqbaOR4YjsvuadL81czU45zEDv4c6pnAr%2FxMKDDYWViq81G9CPiJps3CAXKI8YcKTdUooXwBzpWHe0mCqp9WtgKcdyEUl85CxBxnYT7lC9lE%2BuZeNSfmbUfYMdiOxpjW8bZGX39SM8wagpyNHh79ILbJzX49%2BBpK0I11nzUm8xxnTPF53XqIKksC20%2BA0LHzrHYhV%2FwiuVk0Pb6t%2BUbTHPnUPbe%2B2OX4Pq8o8WvpergM0K2HXjzK2YOkP0M69O%2FjtCEpv22Gd0tP5MMLmsk4fuNxzQIADa2P199CYxynr76eLUr2u63alCb3heTTvPncuJzk02EGEdi38Nm%2BPcq2PTwjY1S%2F4mZ1ZolPl4lPxmfVr4gXrCaXfMExPYokV4FOmo46FJovcncwt4oHFjpSDCufOrbH4xcqrjfTRQ%2BigsxPaH5hWpzILfWTPNXbcIcaJRceFBFZrg8Ysa3oFuHEBgaBZKHRnZmWuFqPB%2B68WmqbZ1tunmg%2BXBKzGqLLfqnBWWw3qDXYr0V2AbALr73VLCeWzQIJzm0E0D%2FdB0KTn2YTHZzfD%2FrXYEUz2i19CwLG7SA8S9no0IFA16%2BpqE4G3s%2FE%2BAKFz3aQJZVpxSTc7Imy0CTF%2FjsA92yilzyIlsIeTc2AjaKy%2BTM%2Fjg%3D%3D&checksum=323806767262b0325dcc5d12405d9773312793615829&enc=AQAEAAACQBPxNw%2BVj6nta7CKEs3N0qWNuu8y9VA2HEw0wmPsL5MTRFTJmnuraG452Pk3WQNpsgmrIf6ePIv561MkEiJV0pbFv9zmD1JW8JOdsIntwNXTFqw1McvYYqbaOR4YjsvuadL81czU45zEDv4c6pnAr%2FxMKDDYWViq81G9CPiJps3CAXKI8YcKTdUooXwBzpWHe0mCqp9WtgKcdyEUl85CxBxnYT7lC9lE%2BuZeNSfmbUfYMdiOxpjW8bZGX39SM8wagpyNHh79ILbJzX49%2BBpK0I11nzUm8xxnTPF53XqIKksC20%2BA0LHzrHYhV%2FwiuVk0Pb6t%2BUbTHPnUPbe%2B2OX4Pq8o8WvpergM0K2HXjzK2YOkP0M69O%2FjtCEpv22Gd0tP5MMLmsk4fuNxzQIADa2P199CYxynr76eLUr2u63alCb3heTTvPncuJzk02EGEdi38Nm%2BPcq2PTwjY1S%2F4mZ1ZolPl4lPxmfVr4gXrCaXfMExPYokV4FOmo46FJovcncwt4oHFjpSDCufOrbH4xcqrjfTRQ%2BigsxPaH5hWpzILfWTPNXbcIcaJRceFBFZrg8Ysa3oFuHEBgaBZKHRnZmWuFqPB%2B68WmqbZ1tunmg%2BXBKzGqLLfqnBWWw3qDXYr0V2AbALr73VLCeWzQIJzm0E0D%2FdB0KTn2YTHZzfD%2FrXYEUz2i19CwLG7SA8S9no0IFA16%2BpqE4G3s%2FE%2BAKFz3aQJZVpxSTc7Imy0CTF%2FjsA92yilzyIlsIeTc2AjaKy%2BTM%2Fjg%3D%3D&checksum=323806767262b0325dcc5d12405d9773312793615829

The FREE 'N FAST text in the shipping section.

Upvotes: 0

Views: 92

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195573

The "trick" is to set shipping preferences to United Kingdom before getting the main page:

import requests
from bs4 import BeautifulSoup

url = 'https://www.ebay.com/itm/Mens-Legacy-Air-Bubble-90-Running-Trainers-Shoes/323806767262?_trkparms=ispr%3D1&hash=item4b6463289e:m:m_fYF4CZiE5Q9q08V38EY5w&enc=AQAEAAACQBPxNw%2BVj6nta7CKEs3N0qWNuu8y9VA2HEw0wmPsL5MTRFTJmnuraG452Pk3WQNpsgmrIf6ePIv561MkEiJV0pbFv9zmD1JW8JOdsIntwNXTFqw1McvYYqbaOR4YjsvuadL81czU45zEDv4c6pnAr%2FxMKDDYWViq81G9CPiJps3CAXKI8YcKTdUooXwBzpWHe0mCqp9WtgKcdyEUl85CxBxnYT7lC9lE%2BuZeNSfmbUfYMdiOxpjW8bZGX39SM8wagpyNHh79ILbJzX49%2BBpK0I11nzUm8xxnTPF53XqIKksC20%2BA0LHzrHYhV%2FwiuVk0Pb6t%2BUbTHPnUPbe%2B2OX4Pq8o8WvpergM0K2HXjzK2YOkP0M69O%2FjtCEpv22Gd0tP5MMLmsk4fuNxzQIADa2P199CYxynr76eLUr2u63alCb3heTTvPncuJzk02EGEdi38Nm%2BPcq2PTwjY1S%2F4mZ1ZolPl4lPxmfVr4gXrCaXfMExPYokV4FOmo46FJovcncwt4oHFjpSDCufOrbH4xcqrjfTRQ%2BigsxPaH5hWpzILfWTPNXbcIcaJRceFBFZrg8Ysa3oFuHEBgaBZKHRnZmWuFqPB%2B68WmqbZ1tunmg%2BXBKzGqLLfqnBWWw3qDXYr0V2AbALr73VLCeWzQIJzm0E0D%2FdB0KTn2YTHZzfD%2FrXYEUz2i19CwLG7SA8S9no0IFA16%2BpqE4G3s%2FE%2BAKFz3aQJZVpxSTc7Imy0CTF%2FjsA92yilzyIlsIeTc2AjaKy%2BTM%2Fjg%3D%3D&checksum=323806767262b0325dcc5d12405d9773312793615829&enc=AQAEAAACQBPxNw%2BVj6nta7CKEs3N0qWNuu8y9VA2HEw0wmPsL5MTRFTJmnuraG452Pk3WQNpsgmrIf6ePIv561MkEiJV0pbFv9zmD1JW8JOdsIntwNXTFqw1McvYYqbaOR4YjsvuadL81czU45zEDv4c6pnAr%2FxMKDDYWViq81G9CPiJps3CAXKI8YcKTdUooXwBzpWHe0mCqp9WtgKcdyEUl85CxBxnYT7lC9lE%2BuZeNSfmbUfYMdiOxpjW8bZGX39SM8wagpyNHh79ILbJzX49%2BBpK0I11nzUm8xxnTPF53XqIKksC20%2BA0LHzrHYhV%2FwiuVk0Pb6t%2BUbTHPnUPbe%2B2OX4Pq8o8WvpergM0K2HXjzK2YOkP0M69O%2FjtCEpv22Gd0tP5MMLmsk4fuNxzQIADa2P199CYxynr76eLUr2u63alCb3heTTvPncuJzk02EGEdi38Nm%2BPcq2PTwjY1S%2F4mZ1ZolPl4lPxmfVr4gXrCaXfMExPYokV4FOmo46FJovcncwt4oHFjpSDCufOrbH4xcqrjfTRQ%2BigsxPaH5hWpzILfWTPNXbcIcaJRceFBFZrg8Ysa3oFuHEBgaBZKHRnZmWuFqPB%2B68WmqbZ1tunmg%2BXBKzGqLLfqnBWWw3qDXYr0V2AbALr73VLCeWzQIJzm0E0D%2FdB0KTn2YTHZzfD%2FrXYEUz2i19CwLG7SA8S9no0IFA16%2BpqE4G3s%2FE%2BAKFz3aQJZVpxSTc7Imy0CTF%2FjsA92yilzyIlsIeTc2AjaKy%2BTM%2Fjg%3D%3D&checksum=323806767262b0325dcc5d12405d9773312793615829'
set_ship_to_url = 'https://www.ebay.com/gh/setuserpreference'

with requests.session() as s:
    r = s.post(set_ship_to_url, json={"userPreferedCountry":"GBR"}) # <-- set Ship To preference to United Kingdom
    soup = BeautifulSoup(s.get(url).content, 'html.parser')

print(soup.select_one('strong.sh_gr_bld').text)

Prints:

FAST 'N FREE

Upvotes: 1

Related Questions