CODEWITHSUNDEEP
pythonperformancedirectory-listing
melnikovkolya
melnikovkolya

Reputation: 11

Faster way to get a directory listing than invoking "ls" in a subprocess

After a search, and some test runs both os.popen()+read() and subprocess.check_output() seem to be almost equivalent for reading out the contents of a folder. Is there a way to improve either a combination of os.popen()+read() or subprocess.check_output()? I have to ls a number of folders and read the outputs, and using either of the above is similar, but represents the major bottleneck according to profiling results.

Upvotes: 1

Views: 354

Answers (3)

JBernardo
JBernardo

Reputation: 33407

Use glob:

>>> from glob import glob
>>> glob('*')

The syntax is the same.

e.g.

glob('*.txt')  # the same as "ls *.txt"

Upvotes: 0

zwol
zwol

Reputation: 140758

You are looking for os.listdir and/or os.walk, and perhaps also the os.stat family of functions. These are (Python bindings to) the same primitives that ls uses itself, so anything you can do by parsing the output of ls you can do with these. I recommend you read carefully through everything the os, os.path, and stat modules offer; there may be other things you don't need an external program to do.

You should probably also read the documentation for stat, the underlying system call -- it's C-oriented, but it'll help you understand what os.stat does.

Upvotes: 5

Adam Rosenfield
Adam Rosenfield

Reputation: 400462

Why don't you just read the directory contents directly with os.listdir? Why do you need to shell out to ls? If you need more information about files in addition to just the filenames, you can also use os.stat. It's much more efficient to do the system calls yourself than to create subprocesses to do it for you.

For doing entirely directory traversals, there's os.walk. The shutil module also has some useful functions.

Upvotes: 2

Related Questions