CODEWITHSUNDEEP
pythonpandasipython
tbc
tbc

Reputation: 1749

python, ipython run interactive script

I have a script named "poc.py" that takes one positional command line argument "inputfile.txt". The poc.py script uses argparse to handle the positional command line argument and then pass the args dict to main(). Once in main(), I read the input file, do some processing, create a pandas DataFrame, and finally plot the data. I am having difficulty manipulating my DataFrame and controlling the exact formatting of the resulting plot, so I'd like to try ipython to explore this interactively and see if I can get a better grasp on the "pythonic" ways to handle pandas/matplotlib.

So I tried experimenting with ipython and running a script, but I can't get ipython to keep my script's namespace.

I have tried this:

$ ipython --pylab -i poc.py inputfile.txt

Which runs my script just fine, and displays the plots (even without the blocking plt.show() call), but when the script is finished, the ipython who and whos commands say Interactive namespace is empty. Likewise if I first enter the ipython shell and then do: 

In [2]: run poc.py inputfile.txt

when the script is done (again, plenty of output, plots show up) I get the same result: an empty interactive namespace.

What am I missing in terms of understanding how to run an external and use ipython to interactively explore the data/objects in my script?

Here is a barebones example of how my script (poc.py) is setup:

import numpy as np
import matplotlib as plt
import pandas as pd

# etc ...more libraries and custom functions here...

def main(args):
  data = np.genfromtxt(args.inputfile)

  # (omitted)...more data processing / manipulation...
  pdata = pd.DataFrame(data)

  # (omitted)...more data processing / manipulation...
  plt.plot(pdata)

  # (omitted)...some formatting of the matplotlib/axes/figure objects
  plt.show()


if __name__ == '__main__':
  parser = argparse.ArgumentParser(description='''some program...''')
  parser.add_argument('inputfile', help='path to input file')
  args = parser.parse_args()
  main(args)

Upvotes: 2

Views: 3217

Answers (2)

tbc
tbc

Reputation: 1749

Answering myself here. I am aware that this could be a comment, but I decided the description warrants a standalone answer.

Basically after spending some time to understand the issues everyone mentioned about my variables going out of scope, and different ways to handle that, I found another solution that worked for me. I ended up using the embed() function from IPython. For the debugging process, I would add 

...
from IPython import embed
embed()
...

at the point in the script where I wanted to stop and look around. Once dropped into the IPython shell, I could investigate variable dimensions and experiment with manipulating things. When I found the combo I wanted, I would copy the commands, drop out of the interactive interpreter and modify the script. The reason this worked for me is it did not involve modifying the structure of the program simply to get debugging information.

Upvotes: 9

abarnert
abarnert

Reputation: 365955

The problem here is that data and pdata aren't in your script's namespace; they're in the local namespace of a function that's already run and completed.

If you want to be able to inspect them after the fact, you'll need to store them somewhere. For example:

    # ...
    plt.show()

    return data, pdata, plt

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='''some program...''')
    parser.add_argument('inputfile', help='path to input file')
    args = parser.parse_args()
    data, pdata, plt = main(args)

(Alternatively, you could just make all of main's variables global, but this way seems cleaner.)

Now, your script's namespace includes variables named data, pdata, and plt that have the values you want. Plus, you can call main again and pass it a different file and get back the values from that file.

Upvotes: 1

Related Questions