CODEWITHSUNDEEP
pythonhtmlxpathscrapy
Godz MadGOD
Godz MadGOD

Reputation: 69

How do i use XPATH properly in scrapy?

http://mnregaweb4.nic.in/netnrega/asset_report_dtl.aspx?lflag=eng&state_name=WEST%20BENGAL&state_code=32&district_name=NADIA&district_code=3201&block_name=KRISHNAGAR-I&block_code=&panchayat_name=DOGACHI&panchayat_code=3201009009&fin_year=2020-2021&source=national&Digest=8+kWKUdwzDQA1IJ5qhD8Fw

Above is the link of the page

: https://i.sstatic.net/8bhzV.png

The red marked box number is what I am trying to get through xpath

: https://i.sstatic.net/mca05.png

The red marked box is the inspect line of that same item. my code is below


**scrapy shell**

**fetch("http://mnregaweb4.nic.in/netnrega/asset_report_dtl.aspx?lflag=eng&state_name=WEST%20BENGAL&state_code=32&district_name=NADIA&district_code=3201&block_name=KRISHNAGAR-I&block_code=&panchayat_name=DOGACHI&panchayat_code=3201009009&fin_year=2020-2021&source=national&Digest=8+kWKUdwzDQA1IJ5qhD8Fw")**

**assetid = response.xpath("//div[3]/center/table[2]/tbody/tr[4]/td[2]")**

**assetid**

**[]**(This is what it returns.)

**assetid = response.xpath("//div[3]/center/table[2]/tbody/tr[4]/td[2]/text()")**(I tried this also)

**assetid**

**[]**(This is what it returns.)

when is use view(response) it says true & opens the same page in browser.

My code is below

: https://i.sstatic.net/YAf38.png

: https://i.sstatic.net/fTWwH.png

Upvotes: 0

Views: 71

Answers (1)

Murat Demir
Murat Demir

Reputation: 716

When you say get end of the xpath you will print what you expect also I update your xpath too:

instead this:

assetid = response.xpath("//div[3]/center/table[2]/tbody/tr[4]/td[2]")

use this:

assetid = response.xpath('//table[2]//tr[4]/td[2]/text()').get()

I hope it will works.

Upvotes: 1

Related Questions