Get process ID of command executed inside terminal in Python subprocess

Question

I run vim inside gnome-terminal in a Python subprocess:

>>> import subprocess
>>> cmd=['gnome-terminal', '--', 'vim']
>>> p = subprocess.Popen(cmd)

It is possible to get the process ID for gnome-terminal with p.pid, but how can I get the process ID for vim from within the Python script?

Even though pstree in Bash shows vim as a child process of gnome-terminal, psutils does not list it.

>>> import psutil
>>> terminal_process = psutil.Process(p.pid)
>>> terminal_process.children()
[]

Answer 1

Here is a workaround. Name the vims by symbolic links, and find their pids:

import subprocess as sub,time,os,signal

N=5
vims= [ sub.Popen(f'ln -fs $(which vim) /dev/shm/vim{vn} && gnome-terminal -- /dev/shm/vim{vn} -c "s/$/Welcome to vim{vn}/"', shell=True) for vn in range(N) ]

time.sleep(1)

for vn in range(N):
    # Get the pids of vims. Vim pid is not equal to proc.pid!
    phelper= sub.Popen(f'ps -o pid= -C vim{vn}',shell=True, stdout=sub.PIPE, stderr=sub.PIPE) 
    try:
        out,err= phelper.communicate(timeout=1)
        vims[vn]= (vims[vn],int(out.decode(encoding="utf8")))  # proc_object --> (proc_object,vim pid)
    except TimeoutExpired:
        pass
    phelper.kill()

# do something:
time.sleep(10)

for proc,vimpid in vims:
    os.kill(vimpid,signal.SIGTERM)

Answer 2

Here is my own Python only take, which works well so far. Any issues with this code?

import psutil, subprocess

cmd=['gnome-terminal', '--', 'vim']
editor_cmd=cmd[-1]  # vim

proc = subprocess.Popen(cmd)       
proc.wait()

# find the latest editor process in the list of all running processes
editor_processes = []
for p in psutil.process_iter():
    try:
        process_name = p.name()
        if editor_cmd in process_name:
            editor_processes.append((process_name, p.pid))
    except:
        pass
editor_proc = psutil.Process(editor_processes[-1][1])

print(editor_proc)

Answer 3

I think this works fine

import time
import subprocess

cmd=['gnome-terminal','--', 'vim']
p = subprocess.Popen(cmd)
time.sleep(10)

a = subprocess.Popen(['ps', '-eo', 'pid,ppid,command'], stdout = subprocess.PIPE)
b = subprocess.Popen(['grep', 'vim'], stdin = a.stdout, stdout = subprocess.PIPE)

output, error  = b.communicate()
output = output.decode("utf-8").split('\n')
print(output)

The reason I used time.sleep(10) is because for some reason vim was not getting forked that fast so, I delayed it for 10 seconds.
Here we create 2 process for getting the ID of vim editor, we give the output of process a to b using stdout and stdin.

Then we use .communicate() to get stdout of process b into output.
Now our output is in form of bytes so we decode it to UTF-8 using .decode("utf-8") and then split on every new line.
It produces the output:

rahul@RNA-HP:~$ python3 so.py
# _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
# _g_io_module_get_default: Found default implementation dconf (DConfSettingsBackend) for ‘gsettings-backend’
# watch_fast: "/org/gnome/terminal/legacy/" (establishing: 0, active: 0)
# unwatch_fast: "/org/gnome/terminal/legacy/" (active: 0, establishing: 1)
# watch_established: "/org/gnome/terminal/legacy/" (establishing: 0)
['21325 21093 vim', '21330 21318 grep vim', '']
rahul@RNA-HP:~$ 

To verify this:

rahul@RNA-HP:~$ ps aux | grep gnome-terminal
rahul    21093  1.7  2.4 978172 45096 ?        Ssl  19:55   0:02 /usr/lib/gnome-terminal/gnome-terminal-server
rahul    21374  0.0  0.0   8988   840 pts/0    S+   19:57   0:00 grep --color=auto gnome-terminal
rahul@RNA-HP:~$ ps -eo pid,ppid,command | grep vim
21325 21093 vim
21376 21104 grep --color=auto vim
rahul@RNA-HP:~$ 

Here we can see that vim is forked from gnome-terminal 21093 is the id of gnome-terminal which is the ppid of vim.

Now, this happened if I didn't use time.sleep(10)

rahul@RNA-HP:~$ python3 so.py
['21407 21406 /usr/bin/python3 /usr/bin/gnome-terminal -- vim', '21409 21406 grep vim', '']

If we try to verify if those PID exist:

rahul@RNA-HP:~$ kill 21407
bash: kill: (21407) - No such process
rahul@RNA-HP:~$ 

Those ID dont exist for some reason.
If there are multiple instances of vim: It produces:

 ['21416 21093 vim', '21736 21093 vim', '21738 21728 grep vim', '']

To get the latest instantiated vim's pid:

output = output[len(output) - 3]

Our output is sorted in ascending order of pid's and our last and second last values are and grep vim so we need the third last argument for getting the pid of vim.
Comment if something can be improved.

Answer 4

This behavior is caused by gnome-terminal.

If you type the command ps | grep <pid> inside your shell, you will see something similar to <pid> pts/0 00:00:00 gnome-terminal <defunct>.

A process being defunct means it has finished its task and is waiting to get killed (or is misbehaving which isn't the case here). This means the process you launched from python has completed its job and is waiting for python to kill it.

Now if you look at pstree, you will see that another gnome-terminal process has been spawn at the root level. This means that the gnome-terminal process you launched in Python simply launched the "real terminal process" at the root level and exited. If you further investigate and look for processes starting with gnome-terminal using ps aux | grep gnome-terminal, you will see output like :

root      5047  0.0  0.0      0     0 pts/0    Z    10:07   0:00 [gnome-terminal] <defunct>
root      5053  0.0  0.3 595276 30468 ?        Sl   10:07   0:00 /usr/lib/gnome-terminal/gnome-terminal-server
root      7147  0.0  0.0  12780   972 pts/0    S+   10:17   0:00 grep gnome-terminal

There is your now defunct process, and a new gnome-terminal-server process. gnome-terminal-server is the process you are looking for.

Long story short pgrep -f gnome-terminal-server will return the pid you want.