CODEWITHSUNDEEP
pythonshellsorting
user2960593
user2960593

Reputation: 87

sort tab delimited file

I am trying to call unix sort command from python to sort a tab delimited file but I couldnt. When I googled about it I found that adding -t$'\t' to the command works and I worked for me too when I tried to sort from the shell but it doesnt work when I try from the python.

This is what I tried in my script

tabdel="$'\t'"
sort_file_cmd="sort -t {1} -k2,2 -k6,6n {0}".format(file_to_be_sorted.name,tabdel)
print sort_file_cmd,shlex.split(sort_file_cmd)
subprocess.call(sort_file_cmd,stdout=sort_bt,shell=True)

The print sort_file_cmd prints this

sort -t $'\t' -k2,2 -k6,6n human_vs_mouse.tab

which when run from shell works perfectly but produces sort: multi-character tab ‘$\t’ error when run from python script

How do I overcome this?

Sample data

gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|10980 gi|58801268|ref|NP_001011737.1| olfactory receptor 1357 [Mus musculus]  3071921 1   307 1.90237e-150    1108.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|460 gi|22129025|ref|NP_667153.1| olfactory receptor 351 [Mus musculus]    302 10  915 2   303 5.70073e-105    806.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|4490 gi|33238878|ref|NP_666817.1| olfactory receptor 24 [Mus musculus]    308 1   921 1   307 9.58658e-105    805.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|458 gi|22129031|ref|NP_667152.1| olfactory receptor 353 [Mus musculus]    302 10  915 2   303 1.01585e-103    798.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|13639 gi|268837230|ref|NP_667200.2| olfactory receptor 1496 [Mus musculus]    3071921 3   309 1.50986e-99 771.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|13345 gi|283837936|ref|NP_666450.2| olfactory receptor 374 [Mus musculus] 310 1   930 1   310 4.18033e-99 768.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|455 gi|22129035|ref|NP_667150.1| olfactory receptor 354 [Mus musculus]    302 13  918 8   309 1.85488e-98 764.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|410 gi|22129071|ref|NP_667122.1| olfactory receptor 1377 [Mus musculus]   305 1   915 1   304 3.06622e-97 755.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|312 gi|53933206|ref|NP_001005569.1| olfactory receptor 366 [Mus musculus] 307 1   921 1   307 2.14345e-96 749.0  
gi|52317161|ref|NM_001004713.1| Homo sapiens olfactory receptor, family 1, subfamily I, member 1 (OR1I1), mRNA  gnl|BL_ORD_ID|4458 gi|58801284|ref|NP_001011748.1| olfactory receptor 867 [Mus musculus]    3091927 1   309 7.36974e-96 748.0

Upvotes: 0

Views: 1419

Answers (2)

Bakuriu
Bakuriu

Reputation: 101999

You can avoid using shell=True, simply use a list instead of a string as command:

>>> subprocess.call(['sort', '-t', '\t', 'testing.txt'])
a       b
c       d
0

Instead of building a single stream use a list of strings, each one representing a token in the original command. In your case the complete "command line" would be:

subprocess.call(['sort', '-t', '\t', '-k2,2', '-k6,6n', file_to_be_sorted.name])

Note that this method is safer and you should try to avoid shell=True as much as possible. You must use shell=True only when you want to use some shell feature, like built-in commands ifs, loops, pipes (although these can be build use subprocess.Popen...) etc. In all cases where you simply want to execute a command passing some arguments, shell=True can be avoided.

You can use shlex.split in order to obtain the list of strings from the command string:

>>> cmdline = shlex.split("sort -t '\t' -k2,2 -k6,6n")
>>> cmdline
['sort', '-t', '\t', '-k2,2', '-k6,6n']
>>> cmdline.append(file_to_be_sorted.name)   # insert the last argument.

Note that in this case you have to surround the \t in single-quotes in the original string.

If you still want to use shell=True then simply do not use the $'\t' escape:

>>> subprocess.call("sort -t '\t' testing.txt", shell=True)
a       b
c       d
0

Works fine for me.

Upvotes: 1

devnull
devnull

Reputation: 123608

The $' ' quoting syntax is interpreted by the shell.

You need to say:

tabdel=r'\t'

Upvotes: 0

Related Questions