CODEWITHSUNDEEP
python-3.xwebweb-scrapingscrapyweb-crawler
amal
amal

Reputation: 3592

How to remove HTML Tags and '\n' from Scrapy Python Output

I'm new to Python and Web Scraping. I wrote below 2 lines to extract title and price from website. However it gives output with html tags and '\n' characters. How can I remove them and get only text output?

product_name = response.css('#productTitle::text')[0].extract().strip('\n')
product_price = response.css('#priceblock_ourprice')[0].extract().strip()

Output

[
    "                \n                    \n                    \n                \n\n                \n                    \n                    \n                        Stainless Steel Food Grinder Attachment fit KitchenAid Stand Mixers Including Sausage Stuffer, Dishwasher Safe,Durable Mixer Accessories as Meat Processor\n                    \n                \n\n                \n                    \n                    \n                \n            ",
    "<span id=\"priceblock_ourprice\" class=\"a-size-medium a-color-price priceBlockBuyingPriceString\">$87.99</span>"
]

Upvotes: 0

Views: 884

Answers (1)

Georgiy
Georgiy

Reputation: 3561

Removing extra spaces an \n:

for text in str_list:
    text = text.replace("\n","") #remove all '\n' from text
    while "  " in text : # if 2 space symbols in sting
        r_str = text .replace("  ", " ") # replace 2 spaces with 1 space and repeat until no more 2 consecutive spaces in text

Second selector also should have ::text in selector:
product_price = response.css('#priceblock_ourprice::text').extract_first()

Upvotes: 1

Related Questions