How to use scrapy-splash with rotating​ proxy?

Question

I was only successful with scraping js content by using the following as a request

def start_requests(self): 
    for url in self.start_urls: 
        yield scrapy.Request(
            url, 
            self.parse, 
            meta={ 'splash': { 'endpoint': 'render.html', 'args': {'wait': 0.5} } }
        )

How do i apply both excute and render.html endpoints in the scrapy request to use a rotating proxy service while scraping dynamic content?

Answer 1

You don't have to use both execute and render.html endpoints, execute is more general. Regarding proxy, execute endpoint supports proxy parameter (see the documentation) which is either a proxy URL, or a Proxy Profile name.