CODEWITHSUNDEEP
python
nowox
nowox

Reputation: 29096

How to get a persistent python session from command line?

I would like to get this behavior from the shell: 

$ python --new-session -c ''
$ python --use-session -c 'x = 42'
$ python --use-session -c 'print x'
42

Why?

I am using a lot of helpers from the command line such as mako-render and jinja stuff to generate files for my (C/C++) project. At each invocation of Python, all the used modules are to be imported into the workspace which takes time. By using a persistent workspace across my Python calls I can save a lot of processing time.

My current solution is to use shelve to store everything I can between my sessions, but this is not convenient and I am looking for a less complicated solution.

I know something is possible using the Jupyter kernel. Unfortunately starting a new Python session connected to an existing kernel takes about 5 to 10 seconds on my system.

Upvotes: 3

Views: 216

Answers (1)

Leon
Leon

Reputation: 32474

The following bash script is a draft approximation of what you need:

python-session:

#!/usr/bin/env bash

if [[ $# -ne 1 ]]
then
    echo 1>&2 "Usage: python-session <session-name>"
    exit 1
fi

sessionname="$1"
sessiondir=/tmp/python-session."$sessionname"
stdin="$sessiondir/stdin"
stdout="$sessiondir/stdout"

endsession()
{
    echo Exiting python session "$sessionname"
    rm -rf "$sessiondir"
}

newsession()
(
    echo "Starting new session $sessionname"
    mkdir "$sessiondir"
    trap "endsession" EXIT
    touch "$stdin"
    touch "$stdout"
    tail -f "$stdin"|python -i -c "import sys; sys.ps1=sys.ps2=''">"$stdout"
)

if [ ! -d "$sessiondir" ]
then
    newsession & disown
    while [ ! -e "$stdout" ]; do sleep 0.01; done
fi

echo "Connected to python session $1 (leave the session by typing CTRL-D)"
tail -f -n 0 "$stdout"&
tailpid=$!
trap "kill $tailpid" EXIT
cat >> "$stdin"

Demonstration:

$ ./python-session 1
Starting new session 1
Connected to python session 1 (leave the session by typing CTRL-D)
x=123
$ ./python-session 1
Connected to python session 1 (leave the session by typing CTRL-D)
print x
123
$ ./python-session 2
Starting new session 2
Connected to python session 2 (leave the session by typing CTRL-D)
print x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
x='abc'
$ ./python-session 1
Connected to python session 1 (leave the session by typing CTRL-D)
print x
123
$ ./python-session 2
Connected to python session 2 (leave the session by typing CTRL-D)
print x
abc
exit()

Exiting python session 2
$ ./python-session 1
Connected to python session 1 (leave the session by typing CTRL-D)
exit()

Exiting python session 1
$

Limitations of the draft implementation

  • you must stop the session by entering the exit() command followed by an additional (even empty) line. After that you still have to hit CTRL-D to get disconnected from the now non-existent session

  • standard error stream of the python session is not captured

Upvotes: 2

Related Questions