how to keep a reference for the requests in scrapy

Question

I'm scraping a page from links created out from numbers,

www.some_page.com/some_number

when the pages exist the url changes, when I try:

response.request.url

I don't get the:

www.some_page.com/some_number

That I'm using to make the search, so I lost the number I'm using to get the page, I need to keep this number to match the data again.

How can I pass some value, the number in this as an argument when I make the request and get it back in the response?

Answer 1

This is exactly what .meta is for:

def parse(self, response):
     return scrapy.Request(url, 
                           meta={"number": number},
                           callback=self.parse_page)

def parse_page(self, response):
    print(response.meta["number"])