CODEWITHSUNDEEP
pythonhtmlbeautifulsouphtml-parsingpython-requests-html
Leng1
Leng1

Reputation: 21

Unable to find tag when data scraping

I am new to Python and I've been working on a program that alerts you when a new item is uploaded to jp.mercari.com (a shopping site). I have the alert part of the program working, but it operates based on the number of items that come up on the search results. When I scrape the website I am unable to find what I am looking for despite being able to locate it when I inspect element on the page. The scraping program looks like this:

from bs4 import BeautifulSoup
import requests

url = "https://jp.mercari.com/search?keyword=pachinko"

result = requests.get(url)
doc = BeautifulSoup(result.text, "html.parser")

tag = doc.find_all("mer-text")
print(tag)

For more context, this is the website and some of the HTML. I've circled the parts I am trying to find in red: screenshot of website and html

Does anyone know why I am unable to find what I'm looking for?

Here is another example of the same problem but from a website that is in English:

import requests


url = "https://www.vinted.co.uk/vetements?search_text=pachinko"

result = requests.get(url)
doc = BeautifulSoup(result.text, "html.parser")

tag = doc.find_all("span")
print(tag)

Again, I can see the part of HTML I want to find when I inspect element but I can't find it when I scrape the website: screenshot of website and html

Upvotes: 2

Views: 616

Answers (1)

eccentricOrange
eccentricOrange

Reputation: 1222

Here's what's happening with me: the element you seek (<mer-text>) is being found. However, the output is in Japanese, and Python doesn't know what to do with that. In my browser, it's being translated to English automatically by Google, so that's easier to deal with.

Upvotes: 1

Related Questions