CODEWITHSUNDEEP
web-scrapingscrapyscrapy-shell
BabyCoder
BabyCoder

Reputation: 3

need help Web extracting <ul> and <li> using scrapy

Currently learning scrapy and with only a small knowledge of extracting a and tags but in need of help when it comes to extracting ul and li for example:

response.css('a.example')

response.css('div.example')

but what is the command for 'ul' and 'li"?

Upvotes: 0

Views: 1438

Answers (1)

Upendra
Upendra

Reputation: 726

What you are missing here is an understanding of CSS selectors. For example,

  • a.example would select <a> elements with class set to exmaple
  • div.example would select <div> elements with class set to exmaple
  • .example would select every element where class is example

Once you learn this, you would know that selecting ul and li doesn't need a separate command. you just need to use the correct selector. Here is an example:

<ul>
<li> one</li>
<li> two</li>
</ul

Now if you want to select the text inside the first li element, you would use

response.css('li::text).get()

If you want to extract both the element's text, you would use

response.css('li::text).getall()

I would suggest to go through sites like W3 Schools and understand CSS selectors

Upvotes: 1

Related Questions