CODEWITHSUNDEEP
pythonbashshellsubprocesspopen
sh kim
sh kim

Reputation: 183

convert from bash shell to python3 using subprocess.Popen()

I'm struggling with converting from bash shell to python3.

Here's shell command that I want to convert to python:

cat $outDir/aDir/* | cut -f2 | sort -u > $outDir/outFile.txt

I already use subprocess.call()and it worked but I want to know how make it with Popen().

Here's my code which didn't work :

import subprocess
import glob

filePath = outDir + 'aDir/*'
outFilePath = outDir + '/outFile.txt'

fileList = []
for files in glob.glob(filePath):
    fileList.append(files)
with open(files, 'r') as inFile, open(outFilePath, 'w') as outFile : 
  p = subprocess.Popen(['cat'], stdin=inFile, stdout=subprocess.PIPE)   
  p2 = subprocess.Popen(['cut', '-f2'], stdin = p1.stdout, stdout=subprocess.PIPE)
  p3 = subprocess.Popen(['sort', '-u'], stdin = p2.stdout, stdout = outFile)

and could you explain why shell=True is harmful? I saw it in many answers but don't know why...

Thank you.

Upvotes: 2

Views: 426

Answers (2)

Will
Will

Reputation: 24739

What about just using shell=True and keeping the pipes?

with open(files, 'r') as inFile, open(outFilePath, 'w') as outFile : 
  p = subprocess.Popen('cut -f2 | sort -u', shell=True, stdin=filePath, stdout=subprocess.PIPE)
  p.communicate()

Or even, more simply:

p = subprocess.Popen("cat {} | cut -f2 | sort -u > '{}'".format(filePath, outFilePath), shell=True)
p.communicate()

Or, even more simply (thanks @tripleee!):

subprocess.call("cat {} | cut -f2 | sort -u > '{}'".format(filePath, outFilePath), shell=True)

As for shell=True, the only danger is really if your input is not safe. I'd recommend quoting all inputs with single quotes, and escaping and sanitizing all inputs.

Upvotes: 0

iruvar
iruvar

Reputation: 23394

You need to pass a list of files to cat So

subprocess.Popen(['cat'], stdin=inFile, stdout=subprocess.PIPE)

should become

subprocess.Popen(['cat'] + [fileList], stdout=subprocess.PIPE)

And consequently inFile should no longer be needed

So, all in all

import subprocess
import glob

filePath = outDir + '/aDir/*'
outFilePath = outDir + '/outFile.txt'

fileList = glob.glob(filePath)
with open(outFilePath, 'w') as outFile: 
  subprocess.Popen(['cat'] + [fileList], stdout=subprocess.PIPE)
  p2 = subprocess.Popen(['cut', '-f2'], stdin = p1.stdout, stdout=subprocess.PIPE)
  p3 = subprocess.Popen(['sort', '-u'], stdin = p2.stdout, stdout = outFile)

Upvotes: 2

Related Questions