CODEWITHSUNDEEP
pythonlinuxstringbash
Matt
Matt

Reputation: 211

Taking Linux Command as Raw String in Python

)I have confirmed my Linux command works in the terminal, however when I try to call it from python it breaks.

The command is a bit long and has lots of single quotes, so I wrapped it around three double quotes (""") so python can interpret as a raw string (or so I thought). However, when I run it I am getting

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file

but I have double and tripple checked my single and double quotes and I have no idea where to go from here.

See the test script below

import os

os.system("""awk -F ' *[[:alnum:]_]*: *' 'BEGIN         {h="insert_job;box_name;command;owner;permission;condition;description;std_out_file;std_err_file;alarm_if_fail"; print h; n=split(h,F,/;/)} function pr() {if(F[1] in A) {for(i=1;i<=n;i++)printf "%s%s",A[F[i]],(i<n)?";":RS}} /insert_job/  {pr(); delete A} {for(i in F){if($0~"^"F[i])A[F[i]]=$2}} END           {pr()}' ./output/JILS/B7443_dev_jil_20140306104313.csv > /trvapps/autosys/admin/EPS/output/JILS/testout.txt""")

FYI I am using Python 2.4.3, hence why I am using os instead of subprocess.

Upvotes: 0

Views: 856

Answers (2)

Charles Duffy
Charles Duffy

Reputation: 295687

Cramming the awk script into one huge line is awful, and makes it nearly impossible to read and maintain. Don't do that -- if you really must use awk (a dubious claim), write it out on multiple lines, with proper indentation, like you would any other script.

To fix the bug with sh -c interpreting things wrong, use the subprocess module (passing an argument array and not setting shell=True) instead of os.system().

import subprocess

awk_script = r'''
*[[:alnum:]_]*: *

BEGIN {
   h="insert_job;box_name;command;owner;permission;condition;description;std_out_file;std_err_file;alarm_if_fail";
   print h;
   n=split(h,F,/;/)
}

function pr() {
  if(F[1] in A) {
    for(i=1;i<=n;i++)
      printf "%s%s", A[F[i]], (i<n) ? ";" : RS
  }
}

/insert_job/ {
  pr();
  delete A;
}

{
  for(i in F) {
    if($0~"^"F[i])
      A[F[i]]=$2
  }
}

END {pr()}
'''

exit_status = subprocess.call(['awk', '-F', awk_script],
  stdin=open('./output/JILS/B7443_dev_jil_20140306104313.csv', 'r'),
  stdout=open('/trvapps/autosys/admin/EPS/output/JILS/testout.txt', 'w'))
if exit_status != 0:
  raise RuntimeException('awk failed')

Upvotes: 1

l0b0
l0b0

Reputation: 58928

For your own sanity, try using pipes.quote (or something similar if that doesn't exist in 2.4), ' '.join(words) and '\n'.join(lines) to be able to build up the command rather than using a single complex string if you have to put it in Python. A better solution would be to call a script like @kojiro suggested.

It looks like you are doing some simple CSV munging. How about checking SO for tips on doing that in Python?

In any case, 400+ characters of awk on a single line is enough to make anyone squirm, and doing it in Python, which already has excellent string handling features, is just passing the pain to the next developer. Which will be very angry.

Upvotes: 2

Related Questions