CODEWITHSUNDEEP
pythondjangoxpathscrapydjango-dynamic-scraper
Vasile
Vasile

Reputation: 841

Trying gather data with django-dynamic-scraper (django, scrapy)

I am using django-dynamic-scraper in one of my applications, I have gone through the docs and following is my setup: enter image description here enter image description here 1)Unfortunately, the scraper works only with first page and gathers well only url_proiect and question, on the other side, other attributes remain empty. I tried to paste both relative and absolute XPath for these objects, but nothing worked.

2)I don't understand well how should I set Detail Pages for SCRAPER ELEMENTS and REQUEST PAGE TYPES.

What is the best way to debug a scraper in such a case? How can I find the correct XPath with minimal number of tryings?

Upvotes: 0

Views: 573

Answers (0)

Related Questions