Efficient way of creating recursive paths Python

Question

I need a simple function to create a path in Python where the parent may or may not exist.

From python documentation os.makedirs will fail if one of the parents exists.

I have written the below method as which works by makes as many sub directories as necessary.

Does this look efficient?

def create_path(path):
    import os.path as os_path
    paths_to_create = []
    while not os_path.lexists(path):
        paths_to_create.insert(0, path)
        head,tail = os_path.split(path)
        if len(tail.strip())==0: # Just incase path ends with a / or \
            path = head
            head,tail = os_path.split(path)
        path = head

    for path in paths_to_create:
        os.mkdir(path)

Answer 1

This is an old thread, but I was not satisfied with the solutions provided since they were mostly too complicated for a simple task.

From the available functions in the library I believe the cleanest we can do is:

os.path.isdir("mydir") or os.makedirs("mydir")

Answer 2

This code will generate directory tree with given depth and width, using recursive function call:

#!/usr/bin/python2.6

import sys
import os

def build_dir_tree(base, depth, width):
    print("Call #%d" % depth)
    if depth >= 0:
        curr_depth = depth
        depth -= 1
        for i in xrange(width):
                # first creating all folder at current depth
                os.makedirs('%s/Dir_#%d_level_%d' % (base, i, curr_depth))
        dirs = os.walk(base).next()[1]
        for dir in dirs:
                newbase = os.path.join(base,dir)
                build_dir_tree(newbase, depth, width)
    else:
        return

if not sys.argv[1:]:
        print('No base path given')
        sys.exit(1)

print('path: %s, depth: %d, width: %d' % (sys.argv[1], int(sys.argv[2]), int(sys.argv[3])))
build_dir_tree(sys.argv[1], int(sys.argv[2]), int(sys.argv[3]))

Answer 3

With python ( >=3.4.1 ) there is exist_ok parameter for os.makedirs.

If exist_ok is False (the default), an OSError is raised if the target directory already exists.

So if you use like exist_ok=True there won't be any problem for Recursive directory creation.

Note : exist_ok comes with python 3.2 on the other hand there was a bug about raising exception even if you set to True. So try using python >= 3.4.1 ( fixed in that version )

Answer 4

"From python documentation os.makedirs will fail if one of the parents exists."

No, os.makedirs will fail if the directory itself already exists. It won't fail if just any of the parent directories already exists.

Answer 5

I found this question while researching a way to make simple directory trees inside of a project directory.

I am somewhat new to Python, and I struggle when data structures get too complex, i.e. nested. It is much easier on my brain's mental mapping to keep track of small lists of iterables, so I came up with two very basic defs to help me with directory tree creation.

The example takes four objects to create a tree:

1. a root directory path = PROJECT_HOME
2. a home path = home (created if it doesn't exist, not overwritten)
3. an iterable of directory names that will go inside of home = branches (created inside of the home, not overwritten)
4. a dictionary of keyed iterables that map onto the branches = leaves (each value created inside of each mapped branch, not overwritten)

5. If any directory exists, it is not overwritten and the error passes silently.

   import os
   from os.path import join as path_join
   import errno
   
   def make_node(node):
       try:
           os.makedirs(node)
       except OSError, e:
           if e.errno != errno.EEXIST:
               raise
   
   
   def create_tree(home, branches, leaves):
       for branch in branches:
           parent = path_join(home, branch)
           make_node(parent)
           children = leaves.get(branch, [])
           for child in children:
               child = os.path.join(parent, child)
               make_node(child)
   
   if __name__ == "__main__":
       try:  # create inside of PROJECT_HOME if it exists
           PROJECT_HOME = os.environ['PROJECT_HOME']
       except KeyError:  # otherwise in user's home directory
           PROJECT_HOME = os.expanduser('~')
   
       home = os.path.join(PROJECT_HOME, 'test_directory_tree')
       create_tree(home, branches=[], leaves={})
   
       branches = (
           'docs',
           'scripts',
       )
       leaves = (
           ('rst', 'html', ),
           ('python', 'bash', )
       )
       leaves = dict(list(zip(branches, leaves)))
       create_tree(home, branches, leaves)
   
       python_home = os.path.join(home, 'scripts', 'python')
       branches = (
           'os',
           'sys',
           'text_processing',
       )
       leaves = {}
       leaves = dict(list(zip(branches, leaves)))
       create_tree(python_home, branches, leaves)
   
       after_thought_home = os.path.join(home, 'docs', 'after_thought')
       branches = (
           'child_0',
           'child_1',
       )
       leaves = (
           ('sub_0', 'sub_1'),
           (),
       )
       leaves = dict(list(zip(branches, leaves)))
       create_tree(after_thought_home, branches, leaves)
   

The directory tree that this example creates looks like this:

    dev/test_directory_tree/
    ├── docs
    │   ├── after_thought
    │   │   ├── child_0
    │   │   │   ├── sub_0
    │   │   │   └── sub_1
    │   │   └── child_1
    │   ├── html
    │   └── rst
    └── scripts
        ├── bash
        └── python
            ├── os
            ├── sys
            └── text_processing

Answer 6

Here's my take, which lets the system libraries do all the path-wrangling. Any errors other than the directory already existing are propagated.

import os, errno

def ensure_dir(dirname):
    """
    Ensure that a named directory exists; if it does not, attempt to create it.
    """
    try:
        os.makedirs(dirname)
    except OSError, e:
        if e.errno != errno.EEXIST:
            raise

Answer 7

Try this code, it checks if path exists till n sub directory level, and create directory if not exists.

def pathtodir(path):
if not os.path.exists(path):
    l=[]
    p = "/"
    l = path.split("/")
    i = 1
    while i < len(l):
        p = p + l[i] + "/"
        i = i + 1
        if not os.path.exists(p):
            os.mkdir(p, 0755)

Answer 8

Rough draft:

import os


class Path(str):
    """
    A helper class that allows easy contactenation
    of path components, creation of directory trees,
    amongst other things.
    """  
    @property
    def isdir(self):
        return os.path.isdir(self)

    @property
    def isfile(self):
        return os.path.isfile(self)

    def exists(self):
        exists = False
        if self.isfile:
            try:
                f = open(self)
                f.close()
                exists = True
            except IOError:
                exists = False
        else:
            return self.isdir
        return exists

    def mktree(self, dirname):
        """Create a directory tree in this directory."""
        newdir = self + dirname
        if newdir.exists():
            return newdir
        path = dirname.split('/') or [dirname]
        current_path = self + path.pop(0)
        while True:
            try:
                os.mkdir(current_path)
            except OSError as e:
                if not e.args[0] == 17:
                    raise e
                current_path = current_path + path.pop(0)
                continue
            if len(path) == 0:
                break
        return current_path

    def up(self):
        """
        Return a new Path object set a the parent
        directory of the current instance.
        """
        return Path('/'.join(self.split('/')[:-1]))

    def __repr__(self):
        return "<Path: {0}>".format(self)

    def __add__(x, y):
        return Path(x.rstrip('/') + '/' + y.lstrip('/'))