CODEWITHSUNDEEP
pythonsubprocessgoogle-bigquery
user3294904
user3294904

Reputation: 454

Bigquery command line tool on Windows

This may be simple but for quite some time I am getting this error!

On windows and python 2.7

import subprocess

p = subprocess.Popen(["C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\bq"])

Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\subprocess.py", line 710, in init errread, errwrite) File "C:\Python27\lib\subprocess.py", line 958, in _execute_child startupinfo) WindowsError: [Error 2] The system cannot find the file specified

When I run normally from command prompt with bq it runs perfectly. I am missing something with subprocess and bq.

Thanks

EDIT: After trying with several solutions provided below I found that when I use "shell=True" most of the commands work on windows shell!

e.g: p = subprocess.Popen('dir', shell=True)

Upvotes: 2

Views: 1763

Answers (3)

jfs
jfs

Reputation: 414675

You need to provide the full name including file extensions e.g.,

from subprocess import Popen

p = Popen(r"C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\bq.exe")

Notice: .exe at the end.

r'' -- a raw string literal is used to avoid escaping backslashes in the Windows path.

Note: if r"C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin is in %PATH% then you could use p = Popen("bq"). See Popen with conflicting executable/path

Note: if the actual bq filename ends with .cmd as @Antonio suggests then (as said in the link) CreateProcess() Windows function that is used by Popen() won't find it unless you specify the file extension explicitly. If you use the shell (cmd.exe); it uses different rules (e.g., enumerates %PATHEXT% as said in the link) and therefore it might find bq.cmd even if you give only bq (assuming .cmd is in %PATHEXT%).

After trying with several solutions provided below I found that when I use "shell=True" most of the commands work on windows shell!

dir is an internal shell command. Unless there is some other dir program; it won't work without shell=True that starts cmd.exe (%COMSPEC%) on Windows. The reason some commands work with shell=True and appear to fail without it is due to the difference in rules that are used to find the executable compared to shell=False case (the default). These rules are enumerated in the link (follow it and read it, I'll wait. It is 4th time, the link is mentioned in the answer).

Upvotes: 2

Matt
Matt

Reputation: 17639

The error messages says, that it cannot find the executable you specified. This is most likely caused by the use of backslashes (\). Backslashes escape characters in a Python string. You can either replace \ with \\ or with /. Furthermore you should add the file extension .cmd. Try this:

p = subprocess.Popen(["C:/Program Files/Google/Cloud SDK/google-cloud-sdk/bin/bq.cmd"])

Upvotes: 2

Antxon
Antxon

Reputation: 1943

Use one of these:

from subprocess import Popen
p = Popen(["C:/Program Files/Google/Cloud SDK/google-cloud-sdk/bin/bq.cmd"])
p = Popen([r"C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\bq.cmd"])

bq executable in Google Cloud SDK for windows is called bq.cmd. When calling it from command line, cmd will look the .cmd extension automatically, while python interpreter does not.

Upvotes: 2

Related Questions