CODEWITHSUNDEEP
pythonjupyter-notebookjupyterdata-sciencejupyter-lab
NYCeyes
NYCeyes

Reputation: 5689

Jupyter / IPython SList :: Obtaining non-tokenized output from the shell execute operator "!"

When shell commands are run within a Jupyter Notebook Python Cell, like this:

output = ! some-shell-command

each line emitted to the standard output (stdout) is captured in a list like IPython data-structure called a SList. For instance:

output = !echo -e 'line1\nline2\nline3'
print(output) # A IPython SList data-structure.

['line1', 'line2', 'line3']

Sometimes, however, you want to preserve the original string output format, without tokenization into a list, like this:

print(output)

line1
line2
line3

Example: For structured JSON output -- which is a string containing many newline characters -- you wouldn't want this tokenization to occur.

How, then, can I execute shell commands in Jupyter Notebooks using the ! operator, and retrieve un-tokenized output (like above)?

Ideally, the solution would be something native in Jupyter Notebooks.

Thank you!

Upvotes: 1

Views: 1123

Answers (3)

MilleCodex
MilleCodex

Reputation: 315

For those here looking to turn jupyter standard output (stdout) which is of type SList into JSON this is what I did:

import json
output = !command  #output is an SList
j_out = json.dumps(output)  #jout is a str
out_dict = json.loads(j_out) #out_dict is a list 
out_dict[i][j]... #to access list elements

This assumes that command has appropriately formatted output to be processed into a python dictionary.

Upvotes: 2

hpaulj
hpaulj

Reputation: 231550

The SList has a number of properties, that return it in various forms:

https://gist.github.com/parente/b6ee0efe141822dfa18b6feeda0a45e5

In [151]: ret = !ls *.json                                                                       
In [152]: ret                                                                                    
Out[152]: ['foo1.json', 'foo.json', 'logins.json', 'stack56532806.json']

as list

In [153]: ret.l                                                                                  
Out[153]: ['foo1.json', 'foo.json', 'logins.json', 'stack56532806.json']

as newline separated string:

In [154]: ret.n                                                                                  
Out[154]: 'foo1.json\nfoo.json\nlogins.json\nstack56532806.json'

as space delimited:

In [155]: ret.s                                                                                  
Out[155]: 'foo1.json foo.json logins.json stack56532806.json'
In [156]: type(ret)

its docs

In [158]: ret?                                                                                   
Type:        SList
String form: ['foo1.json', 'foo.json', 'logins.json', 'stack56532806.json']
Length:      4
File:        /usr/local/lib/python3.6/dist-packages/IPython/utils/text.py
Docstring:  
List derivative with a special access attributes.

These are normal lists, but with the special attributes:

* .l (or .list) : value as list (the list itself).
* .n (or .nlstr): value as a string, joined on newlines.
* .s (or .spstr): value as a string, joined on spaces.
* .p (or .paths): list of path objects (requires path.py package)

Any values which require transformations are computed only once and
cached.

Upvotes: 3

Barmar
Barmar

Reputation: 781994

Use join() to combine an iterator into a string.

"\n".join(output)

Upvotes: 1

Related Questions