CODEWITHSUNDEEP
javascriptpython-3.xluascrapyscrapy-splash
Tobi
Tobi

Reputation: 301

Lua, Scrapy/Splash: Clicking button with no href

I am trying to implement a click event to get the details of every entry from this page: https://www.mrlodge.de/wohnungen/

The Html markup of the button which links to the Details looks like this:

<li class="action mrl-list__item details-bt">
   <button>
      <span class="icon icon-arrow-right">
        ::before
      </span>
      "Details"
   </button>
</li>

I have some experience with LUA and Splash but have no idea how to attack this problem since there is no actual href link given in the html markup. I have read about the Splash method mouseclick(), which needs pixel directions. However I am looking for a more generic solution with Splash.

Please help

Upvotes: 0

Views: 318

Answers (1)

Casper
Casper

Reputation: 1435

This page doesn't use javascript. Try disabling javascript and the page still works. The page works with forms instead.

>>> fetch('https://www.mrlodge.de/wohnungen/')
2019-07-10 14:56:41 [scrapy.core.engine] INFO: Spider opened
>>> response.xpath('//form/input[@name="name_url"]/@value').extract() 
[u'/wohnen-auf-zeit/2-zimmer-wohnung-muenchen-maxvorstadt-11609/', u'/wohnen-auf-zeit/4-zimmer-haus-muenchen-fuerstenried-10756/', u'/wohnen-auf-zeit/3-zimmer-wohnung-muenchen-lerchenau-11653/', u'/wohnen-auf-zeit/2-zimmer-wohnung-muenchen-glockenbachviertel-4180/', u'/wohnen-auf-zeit/2-zimmer-wohnung-muenchen-berg-am-laim-11625/']

Upvotes: 1

Related Questions