CODEWITHSUNDEEP
bashipythonjupyter-notebookipython-parallel
retsreg
retsreg

Reputation: 13

Run bash commands in parallel using iPython

I'm using iPython notebook on a 16 core machine (so there are 15 "engines"). If I run this cell ...

%%px
%%bash
echo 'hi' > file1.txt

... then the result is one file called file1.txt that is written to disk 15 times. What I really want is to write to 15 different files, file1.txt through file15.txt.

I'm new to all this, so I imagine there's a simple solution!

Thanks,

Retsreg

Upvotes: 1

Views: 433

Answers (1)

John Zwinck
John Zwinck

Reputation: 249404

The Bash special variable $$ is the PID of the shell, so you could do this:

echo hi > file$$.txt

That will write files like file3392.txt etc., where the number is the PID of each session. You can later rename the files if you need to.

Upvotes: 1

Related Questions