CODEWITHSUNDEEP
javascriptpythonweb-scrapingscrapy-splash
Amar
Amar

Reputation: 666

Scrapy splash response won't return complete html

I am trying to scrape this link https://www.myntra.com/women-kurtas-kurtis-suits . But when in try to render it through splash http API. I get this below partially rendered result enter image description here

Am I missing something here ?

This is the actual page.

enter image description here

Upvotes: 1

Views: 708

Answers (1)

Samy
Samy

Reputation: 649

If you're trying to scrape products, why don't you use the non JS rendered HTML returned by default? You will find JSON objects containing products details inside it. Here's an example from the web page you posted:

    <script type="application/ld+json"> {"@context":"https://schema.org","@type":"Product","name":"AKS Women Blue & Grey Printed Kurta with Palazzos","image":"http://assets.myntassets.com/assets/images/8076903/2018/12/8/fb0cf882-a473-4aae-86c2-edf912b70b6e1544251004970-AKS-Women-Kurta-Sets-2261544251003921-1.jpg","description":"Women Printed Kurta with Palazzos","brand":{"@type":"Thing"},"offers":{"@type":"Offer","priceCurrency":"INR","price":989},"AggregateRating":{"@type":"AggregateRating","itemReviewed":"AKS Women Blue & Grey Printed Kurta with Palazzos","ratingCount":0,"reviewCount":""}}</script>

Using JSON Python library, you can extract the data and use it as you want.

Upvotes: 1

Related Questions