CODEWITHSUNDEEP
batch-fileprocesswait
Sudhakar Samak
Sudhakar Samak

Reputation: 399

Make command prompt wait till all processes finish execution

I am running a batch file through java code. My batch file consists of several commands. i want this process to terminate only after the execution of all the processes. my java code is

String command = "cmd /c start /wait C:\\python27\\tutorial\\check.bat "+key+" "+formattedDate;
Process p=Runtime.getRuntime().exec(command);
p.waitFor();

There are some function after this which i want to execute after all the processes in my batch file are completed. My batch file has

cd\
cd python27
cd tutorial
start scrapy crawl flipkart -a key="%1" -o %2flipkart.xml
start scrapy crawl myntra -a key="%1" -o %2myntra.xml
start scrapy crawl jabong -a key="%1" -o %2jabong.xml
start scrapy crawl hs18 -a key="%1" -o %2hs18.xml
start scrapy crawl indiatimes -a key="%1" -o %2indiatimes.xml
start scrapy crawl shopclues -a key="%1" -o %2shopclues.xml
start scrapy crawl croma -a key="%1" -o %2croma.xml
start scrapy crawl amazon -a key="%1" -o %2amazon.xml
exit

Due to this exit command the batch file is returning back and the main thread continues. I want this batch file to exit only after all the processes are completed. Any help is appreciated.

Upvotes: 0

Views: 2961

Answers (2)

dbenham
dbenham

Reputation: 130819

I execute each command via cmd.exe so that I can redirect an unused file handle to a lock file. The process holds an exclusive lock on the file until it completes. The file cannot be deleted until the lock is released (the process ends)

I have a loop that attempts to delete the lock files. If a deletion fails, then the process must still be alive so it loops back to try again. I use TIMEOUT to introduce a delay so that the deletion loop does not swamp the CPU.

@echo off
setlocal
cd \python27\tutorial

set "jobs=flipkart myntra jabong hs18 indiatimes shopclues croma amazon"

for %%A in (%jobs%) do start cmd /c scrapy crawl %%A -a key="%1" -o "%~2%%A.xml" 9^>%%A.lock

:waitForCompletion
timeout /t 1 nobreak >nul
for %%A in (%jobs%) do (
  del %%A.lock >nul 2>nul
  if exist %%A.lock goto :waitForCompletion
)

exit

There is probably no need to see any output in your console windows, so you could use START /B to run each process within the same console, and redirect stdout to nul to hide the output. I left stderr as is just in case an error pops up.

@echo off
setlocal
cd \python27\tutorial

set "jobs=flipkart myntra jabong hs18 indiatimes shopclues croma amazon"

for %%A in (%jobs%) do start /b cmd /c scrapy crawl %%A -a key="%1" -o "%~2%%A.xml" ^>nul 9^>%%A.lock

:waitForCompletion
timeout /t 1 nobreak >nul
for %%A in (%jobs%) do (
  del %%A.lock >nul 2>nul
  if exist %%A.lock goto :waitForCompletion
)

exit

Upvotes: 0

rojo
rojo

Reputation: 24466

Did you intend to set all the scrapy threads to launch asynchronously? If not, simply remove the start from each line to accomplish your goal.

If the non-blocking threads were intentional, then I'm guessing the threads will probably complete after unpredictable durations, and scrapy crawl amazon might not necessarily be the last thread to finish.

You could add a loop to your batch script just above exit to check whether scrapy exists in the process list -- something like this:

cd \python27\tutorial
start scrapy crawl flipkart -a key="%1" -o %2flipkart.xml
start scrapy crawl myntra -a key="%1" -o %2myntra.xml
start scrapy crawl jabong -a key="%1" -o %2jabong.xml
start scrapy crawl hs18 -a key="%1" -o %2hs18.xml
start scrapy crawl indiatimes -a key="%1" -o %2indiatimes.xml
start scrapy crawl shopclues -a key="%1" -o %2shopclues.xml
start scrapy crawl croma -a key="%1" -o %2croma.xml
scrapy crawl amazon -a key="%1" -o %2amazon.xml

echo Waiting for other threads to complete...
:wait
>NUL 2>NUL (
    timeout /t 1 /nobreak || ping -n 2 localhost
    tasklist /fi "imagename eq scrapy.exe" | find /i "scrapy" && goto wait
)
exit

Upvotes: 1

Related Questions