CODEWITHSUNDEEP
pythonpython-3.x
Goku
Goku

Reputation: 534

Python - alternative for multiple os.system command

Below are the shell commands, that does few variable and file manipulation and uploads the final file to a repository.

Using repetitive os.system() is one option. Is there a simpler and elegant way to address this ?

Even with the os.system(), I'm facing issues with quotes and the variable replacement is throwing error.

I don't want to club these commands as a script and execute it :) Any input would be really helpful, thanks.

target_folder = tmp_folder+'/tmp'+str(count)
FILE_TYPE = "bundle"
FILE_NAME = "solution"

    os.system('''
    set +x
    file_name_wo_ext="$(cat 'target_folder'/*.yaml| grep -i name | awk '{print $2}' | awk '{print $1}')"
    output_folder="/home/'FILE_NAME'/'FILE_TYPE'/${file_name_wo_ext}/"
    tar_file_name=$(ls $output_folder | grep tar.gz)
    file_path="/home/"+FILE_NAME"/"+FILE_TYPE+"/${file_name_wo_ext}/${tar_file_name}"
    mv $file_path "`basename $file_path .tar.gz`.tgz"
    wget command to upload the file
    ''')

Error:

ls: cannot access '/home/+FILE_NAME/+FILE_TYPE+/cat': No such file or directory

ls: cannot access '+target_folder/*.yaml|': No such file or directory

OS: Ubuntu 18.04.

Upvotes: 0

Views: 831

Answers (2)

Serge Ballesta
Serge Ballesta

Reputation: 148900

Multiple os.system commands are generally a bad idea, because you start a brand new shell for each command, which prevents to use shell level variables, or change the environment in one command and use it in the next one.

I would use subprocess to start one single shell, and then send all the commands to its standard input:

  • one single shell execution
  • you can freely use set commands to change the behaviour of that shell or set variables
  • you have full control(*) on what is written to the standard output and standard error streams

(*) but you cannot expect to read something from the output streams before the end of the shell...

Upvotes: 0

AKX
AKX

Reputation: 168967

An approximation of your shell script in Python might be something like this.

Do note that your original script will behave in undefined ways if there are e.g. multiple name lines in the YAML, or multiple .tar.gzes...

import glob
import os


def find_filename_wo_ext(target_folder):
    for yaml_filename in glob.glob(os.path.join(target_folder, "*.yaml")):
        with open(yaml_filename, "r") as yaml_file:
            for line in yaml_file:
                if "name" in line.lower():
                    return line.partition(" ")[-1]  # might be wrong :)


def main():
    tmp_folder = "/tmp"
    count = 1

    target_folder = os.path.join(tmp_folder, f"tmp{count}")
    FILE_TYPE = "bundle"
    FILE_NAME = "solution"

    file_name_wo_ext = find_filename_wo_ext(target_folder)
    output_folder = os.path.join("/home", FILE_TYPE, FILE_NAME, file_name_wo_ext)
    tar_file_name = list(
        glob.glob(os.path.join(output_folder, "*.tar.gz"))
    )[0]
    file_path = os.path.join(output_folder, tar_file_name)
    os.rename(file_path, os.path.splitext(file_path)[0] + ".tgz")
    os.system("wget...")

Upvotes: 2

Related Questions