Paging output from python

Question

I'm trying to implement something similar to git log, which will only page the output if the log is of a certain length. If you're not familiar with git, I'm essentially trying to achieve this:

python some_script.py | less

With some help from the paging implementation in python2.6/pydoc.py, I was able to come up with this:

import os
text = '...some text...'
pipe = os.popen('less', 'w')
pipe.write(text)
pipe.close()

which works great, but os.popen() is deprecated. I've considered writing to a temp file and calling less with its path, but that doesn't seem ideal. Is this possible with subprocess? Any other ideas?

EDIT:

So I've gotten subprocess working. I was able to give it the text variable with Popen.communicate(text), but since I really want to redirect print statements, I've settled on this:

  import os, sys, subprocess, tempfile

  page = True
  if page:
      path = tempfile.mkstemp()[1]
      tmp_file = open(path, 'a')
      sys.stdout = tmp_file
  print '...some text...'
  if page:
      tmp_file.flush()
      tmp_file.close()
      p = subprocess.Popen(['less', path], stdin=subprocess.PIPE)
      p.communicate()
      sys.stdout = sys.__stdout__     

Of course, I'd end up wrapping it into functions. Does anyone see a problem with that?

Answer 1

In py2.7 I was able to do just

    sys.stdout = os.popen(pager, 'w')

but this stopped working in py3. Some of the above solutions didn't seem to work for me, but I wrote this decorator that does:

  def _get_pager_name():                                
      for p in (os.environ.get('PAGER', ''), '/usr/bin/less', '/bin/more', 'cat'):                                
          if Path(p).is_file():                                
              return p                                
      return None                                
                                  
                                  
  def paged(f):                                
      @wraps(f)                                
      def wrapper(*a, **kw):                                
          try:                                
              with subprocess.Popen([_get_pager_name()], stdin=subprocess.PIPE, text=True) as p:                                
                  sys.stdout = p.stdin                                
                  return f(*a, **kw)                                
          finally:                                
              sys.stdout = sys.__stdout__                                
                                  
      pager = _get_pager_name()                                
      return wrapper if sys.stdout.isatty() and pager else f        

Answer 2

With one caveat, this works for me as a way to redirect process output to a pager on Linux (I don't have Windows handy to test there) when, for whatever reason, it's not feasible to write it all to a file or StringIO and then feed it to the pager all at once:

import os, sys

less = None
if os.isatty(sys.stdout.fileno()):
    less = subprocess.Popen(
        ['less', '-R', '--quit-if-one-screen'],
        stdin=subprocess.PIPE)
    os.dup2(less.stdin.fileno(), sys.stdout.fileno())

Now for the caveat:

less behaves as if -E has been passed and terminates when I reach the bottom of the output. I assume that's because, as the child of the script, it's dying when the script does, but adding the following to the end of the script just causes less to hang when it would otherwise have exited and I haven't had time to figure out why:

if less:
    less.stdin.close()  # no different with or without
    less.communicate()  # no different with less.wait()

(It wasn't an issue when I first came up with this hack because I originated it for making a PyGTK 2.x program pipe its output to grep to work around PyGTK not exposing the function from the GTK+ C API needed to silence some spurious log messages.)

Answer 3

It is a good idea to be explicit in your code, so that it shows that you use a special print function printc() instead of the standard one. Using subprocess.call() is also sufficient (you don't need the pipe machinery). Furthermore, you can save a variable by not storing the name of the temporary file:

from __future__ import print_function

import subprocess, tempfile

page = True  # For tests

# Definition of a printc() function that prints to the correct output
if page:
    tmp_file = open(tempfile.mkstemp()[1], 'w')  # No need to store the name in a specific variable
    def printc(*largs, **kwargs):
        if 'file' not in kwargs:  # The code can still use the usual file argument of print()
            kwargs['file'] = tmp_file  # Forces the output to go to the temp file
        print(*largs, **kwargs)
else:
    printc = print  # Regular print

# Main program:

printc('...some text...', 'some more text', sep='/')  # Python3 syntax

# Paging of the current contents of the temp file:
if page:
    tmp_file.flush()  # No need to close the file: you can keep printing to it
    subprocess.call(['less', tmp_file.name])  # Simpler than a full Popen()

This way, you get the flexibility of Python 3's print function, with a code that explicitly shows that you're doing some fancy printing stuff. This scales better with larger programs than modifying the "global" sys.stdout variable in some locations of your code.

Answer 4

I didn't like executing external commands, so I wrote pager in pure Python. It still has a problem - piped input works only for Windows.

Answer 5

How about this:

import pydoc
text = '... some text ... '
pydoc.pager(text)

This (on my opensuse linux box) sends the text to a pager ('less' in my case), and works the same as calling "help(... python command...)" within the Python interpreter.

Answer 6

Use subprocess.Popen instead.

http://docs.python.org/library/subprocess.html#subprocess-replacements

http://docs.python.org/library/subprocess.html#subprocess.Popen

There is even a note about this in the os.popen docs.

http://docs.python.org/library/os.html#os.popen