CODEWITHSUNDEEP
pythonnode.jsscrapy
user11072240
user11072240

Reputation:

How to execute scrapy python in a javascript file?

I have a python script that I launched with the command scrapy crawl mynamefile.

I would like to run the python cli command scrapy crawl from a js file in my node js application, then display it in the console.

The problem is that I don't how to import the scrapy python library in node.

Thanks for your help!

Upvotes: 0

Views: 409

Answers (1)

Tomáš Linhart
Tomáš Linhart

Reputation: 10210

I don't think it is a good idea to run Python code from NodeJS. If you are for some reason forced to do that, you probably need to call Python script in a child process (see for example this article).

Alternatively, you might use something like Scrapyrt, which exposes HTTP API for your Scrapy project and provides on-demand scraping ability.

Yet the best option seems to be using some native JavaScript scraping solution, like Apify.

Upvotes: 2

Related Questions