Getting pyreadline3.Readline's history working with Python 3.9

Question

I am following this Q&A's procedure for accessing Python's REPL command history -- specifically at the Conda prompt, not in an IDE.

I am using Python 3.9 installed under Anaconda on Windows 10. Version 3.9 needed for backward compatibility with colleagues' past work.

The history is apparently handled by pyreadline3.Readline, which is the implementation of Gnu's readline for Python 3. I am relying on Anaconda to manage the package depedencies. Unfortunately, the REPL commands aren't being captured.

Annex A shows a session transcript in the Conda environment py39, which contains Python 3.9. It contains three chunks:

1. Installation of pyreadline3

2. Test-driving the history capture in Python, which fails

3. Using Cygwin's Bash to confirm that the written history file is empty.

What else can I do to enable history capture?

Tangentially relevant: %USERPROFILE%\.python_history captures REPL commands, but does not update until one exits the Python REPL session.

Annex A: Session transcript

REM Insteall pyreadline3
REM --------------------
(py39) C:\Users\User.Name> conda install pyreadline3
<...snip...>
Collecting package metadata (current_repodata.json): \ DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
<...snip...>
done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
  current version: 23.7.2
  latest version: 23.7.3
Please update conda by running
    $ conda update -n base -c defaults conda
Or to minimize the number of packages updated during conda update use
     conda install conda=23.7.3
## Package Plan ##
  environment location: C:\Users\User.Name\anaconda3\envs\py39
  added / updated specs:
    - pyreadline3
The following packages will be downloaded:
    package                    |            build
    ---------------------------|-----------------
    pyreadline3-3.4.1          |   py39haa95532_0         148 KB
    ------------------------------------------------------------
The following NEW packages will be INSTALLED:
  pyreadline3        pkgs/main/win-64::pyreadline3-3.4.1-py39haa95532_0
Proceed ([y]/n)? y <RETURN>
Downloading and Extracting Packages
pyreadline3-3.4.1    | 148 KB    |                                                                              |   0% DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/win-64/pyreadline3-3.4.1-py39haa95532_0.conda HTTP/1.1" 200 151256
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

REM Illustrate in Python that commands aren't capture
REM -------------------------------------------------
(py39) C:\Users\User.Name> python
Python 3.9.17 (main, Jul  5 2023, 21:22:06) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyreadline3 import Readline
>>> readline = Readline()
>>> readline.get_current_history_length() # Commands captured
0
>>> readline.get_history_length() # Length of buffer
100
>>> import os # Write history to file
>>> os.getcwd()
   'C:\\Users\\User.Name'
>>> readline.write_history_file("./pyHistory.txt")

# Use Cygwin's Bash to show that target history file is empty
#------------------------------------------------------------
User.Name@Laptop-Name /c/Users/User.Name
$ new #  This is "ls -ltA", cleaned up with "sed"
              0 2023-08-30 13:10 pyHistory.txt*
        2359296 2023-08-29 01:39 NTUSER.DAT*
        <...snip...>

Answer 1

Reading the documentation from on the built-in readline, and also about the how pyreadline3 has implemented history, we can summarize the situation as follows.

The default files for python history are stored in the following files (under $HOME).

.history            # Default pyreadline / pyreadline3
.python_history     # Default readline (built-in) when interactive

Furthermore it states that if you want to use the .history file, it has to already exist!

The example code from above link, shows that you can load/save the file at any time.

"The following example achieves the same goal but supports concurrent interactive sessions, by only appending the new history."

import atexit
import os
import readline
histfile = os.path.join(os.path.expanduser("~"), ".python_history")

try:
    readline.read_history_file(histfile)
    h_len = readline.get_current_history_length()
except FileNotFoundError:
    open(histfile, 'wb').close()
    h_len = 0

def save(prev_h_len, histfile):
    new_h_len = readline.get_current_history_length()
    readline.set_history_length(1000)
    readline.append_history_file(new_h_len - prev_h_len, histfile)
atexit.register(save, h_len, histfile)

This can be further manipulated in the rlcompleter config.

---

Note:
There is an unmerged PR in the original pyreadline3 repo. That PR fixes tab and ANSI color usage. It is therefore suggested to use the clone from here instead. In addition, it would make more sense that both readline implementations use the same file, preferably .python_history...