subprocess.CalledProcessError: ... returned non-zero exit status 255

Question

I'm trying to run someone else's code, which I'm imagining has never been run on Windows, and I cannot get past 'git log | head -n 1 | awk '{print $2}'' returned non-zero exit status 255:

Logging to ./logs/log_j0_t0.txt
Traceback (most recent call last):
  File "C:\Users\mcram\Documents\Github\low-memory-fnn-training\napoleon\utils\log_utils.py", line 55, in log_git_commit_info
    stderr=subprocess.PIPE)
  File "C:\Users\mcram\Anaconda3\envs\pytorch\lib\subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'git log | head -n 1 | awk '{print $2}'' returned non-zero exit status 255.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 86, in <module>
    main()
  File "main.py", line 23, in main
    init_log(args)
  File "C:\Users\mcram\Documents\Github\low-memory-fnn-training\napoleon\utils\log_utils.py", line 36, in init_log
    log_git_commit_info()
  File "C:\Users\mcram\Documents\Github\low-memory-fnn-training\napoleon\utils\log_utils.py", line 62, in log_git_commit_info
    returncode, err_type=subprocess.CalledProcessError, cmd=cmd)
  File "C:\Users\mcram\Documents\Github\low-memory-fnn-training\napoleon\utils\log_utils.py", line 109, in log_error
    raise err_type(msg, **kwargs)
subprocess.CalledProcessError: Command 'git log | head -n 1 | awk '{print $2}'' returned non-zero exit status 255.

I'm running Windows 10 and have git version 2.14.2.windows.1 installed. Below is the relevant function from log_utils.py. Line 55 is stderr=subprocess.PIPE).

def log_git_commit_info():
    get_commit_hash = "git log | head -n 1 | awk '{print $2}'"
    check_unstaged = 'git diff --exit-code'
    check_staged = 'git diff --cached --exit-code'
    status = 'git status'
    cmds = [get_commit_hash, check_unstaged, check_staged, status]
    do_checks = [True, False, False, True]
    saved_infos = []
    for cmd, do_check in zip(cmds, do_checks):
        try:
            process_result = subprocess.run(
                cmd,
                shell=True,
                check=do_check,
                universal_newlines=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE)
            saved_infos.append((process_result.returncode,
                                process_result.stdout.strip()))
        except subprocess.CalledProcessError as e:
            err_msg = str(e)
            returncode = int(err_msg.split()[-1][:-1])
            log_error(
                returncode, err_type=subprocess.CalledProcessError, cmd=cmd)
    commit_hash = saved_infos[0][1]
    log_info('Current commit: ' + commit_hash)
    if saved_infos[1][0] or saved_infos[2][0]:
        log_warn('Uncommitted changes present!')
        log_warn("Output of 'git status':\n" + saved_infos[3][1])
        log_info("Output of 'git diff HEAD':")
        diff_output = subprocess.run(
            'git diff HEAD',
            shell=True,
            check=do_check,
            universal_newlines=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        log_info(diff_output.stdout)

My only guess here is that git log | head -n 1 | awk '{print $2}' is producing the error, specifically error 255. I've looked everywhere for some explanation of the error codes, but haven't found anything, which seems odd. Shouldn't the docs explain the error codes?

Anyway, I tried just running that command from the prompt and I get this:

(pytorch) C:\Users\mcram\Documents\Github\low-memory-fnn-training\apps\cifar10>git log | head -n 1 | awk '{print $2}'
'head' is not recognized as an internal or external command,
operable program or batch file.

Could the fact that 'head' is not recognized as an internal or external command be the error 255? I'm wondering if this is just because I'm running from the command line. From https://github.com/sr320/LabDocs/issues/664#issuecomment-318851496 I'm also wondering if the problem is that Git Bash is not in my PATH. Opening Git Bash and running cat ~/.bash_profile, however, gives me cat: /c/Users/mcram/.bash_profile: No such file or directory. I do, however, get this, which doesn't seem to include a path to Anaconda:

$ echo $PATH
/c/Users/mcram/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/mcram/bin:/c
/ProgramData/DockerDesktop/version-bin:/c/Program Files/Docker/Docker/Resources/bin:/c/Program 
Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin:/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA
/v10.1/libnvvp:/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin:/c/Program Files/NVIDIA GPU 
Computing Toolkit/CUDA/v8.0/libnvvp:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows
/System32/WindowsPowerShell/v1.0:/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/c/Program 
Files/PuTTY:/cmd:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32
/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Program Files/NVIDIA Corporation/Nsight Compute 
2019.3.0:/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/c/Program Files/Intel
/Intel(R) Management Engine Components/DAL:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:
/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Users/mcram/AppData/Local
/Microsoft/WindowsApps:/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/extras/CUPTI/libx64:
/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64:/c/Users/mcram/AppData/Local
/GitHubDesktop/bin:/c/Users/mcram/AppData/Local/Microsoft/WindowsApps:/usr/bin/vendor_perl:/usr/bin
/core_perl

If I can't find .bash_profile I don't know how to update the PATH, and even if I was able to find it, I wouldn't know what to put in there since I'm not running the script in Jupyter, but rather at the Python command line. I have run out of ideas and could use some help. Thank you.

Answer 1

The Python documentation says the following:

On Windows with shell=True, the COMSPEC environment variable specifies the default shell.

That means that you're likely trying invoke cmd instead of sh or bash. Your best bet here is going to be avoiding shell=True and instead setting your cmd variable to something like the following:

cmd = ["sh", "-c", cmd]

Note that on Linux, we would typically write /bin/sh, not sh, but since you're using Windows that won't work. This solution will also be portable to Linux and most other Unix systems, in case that matters to you.

Once you're using sh (I assume from Git for Windows), it should have the PATH set appropriately for all of the other Unix commands.