CODEWITHSUNDEEP
javaapacheweb-crawlernutch
Hafiz Muhammad Shafiq
Hafiz Muhammad Shafiq

Reputation: 8670

How to run apache nutch different jobs in parallel manner

I am using nutch 2.3. All jobs run one after other i.e. first generator, fetch, parse, index etc. I want to run some jobs simultaneously. I know some jobs cannot run in parallel but other can e.g parse job, dbupdate, indexjob should be run with fetch.

Is it possible ? My basic objective is to run fetcher job all the time. I suppose that we can do it with different timestamp. Can anyone guide me the proper way ?

Upvotes: 10

Views: 417

Answers (1)

Mubin Shrestha
Mubin Shrestha

Reputation: 398

If you check out the nutch web app server, you will find out that it can execute multiple crawl job in parallel.You should check out the source code of the Nutch 2.3 for webapp[NutchUiServer]. Hope this helps.

Upvotes: 5

Related Questions