CODEWITHSUNDEEP
mysqlpython-3.xsubprocesspipepv
Rohit Thaply
Rohit Thaply

Reputation: 31

Why is Python subprocess not working as expected with named_pipe?

I am using python subprocess to run a mysqlimport command. For tracking the uploading progress, I am using a named pipe and pipe_viewer. Here is another similar problem.

The subprocess Pipes are running without any error, but mysqlimport command is not uploading data to the required table.

import subprocess, os, sys
os.mkfifo('named_pipe')
pv = subprocess.Popen(
    ["pv", "-f", "file.csv", ">", "named_pipe"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)
pro = subprocess.Popen(
["mysqlimport", "--ignore-lines=1", "--fields-terminated-by=\',\'", 
"--local", "-u", "root", "-pRoot@123", "database_name", named_pipe],
stdin=pv.stdout,
)

pv.stdout.close()

This is the simple mysqlimport command which is running fine and uploading data but without pipe_viewer, the progress can not be tracked:

pro = subprocess.Popen(
["mysqlimport", "--ignore-lines=1", "--fields-terminated-by=\',\'",
 "--local", "-u", "root", "-pRoot@123", "database_name", "file.csv"], 
stdout=subprocess.PIPE
)

Here is the terminal command using pipe_viewer and mysqlimport which is working fine:

$ mkfifo named_pipe

$ pv -f file.csv > named_pipe | mysqlimport --ignore-lines=1 --fields-terminated-by=',' --local -u root -pRoot@123 database_name named_pipe

Upvotes: 2

Views: 737

Answers (1)

Daid Braam
Daid Braam

Reputation: 193

To get the ">" redirection to work correctly, you most likely need to set "shell=True" to the Popen function.

Upvotes: 1

Related Questions