Python: why is my method using os.walk does not return all available paths?

Question

Question as in title and here's the method:

 def walkThroughPath(self , sBasePath, blFolders = True, blFiles = True ):
      aPaths = []
      for sRootDir, aSubFolders, aFiles in os.walk( sBasePath ):
           for sFolder in aSubFolders:
                if blFolders == True:
                     aPaths.append( sRootDir )
                for sFileName in aFiles:
                     if blFiles == True:
                          aPaths.append( sRootDir + "/" + sFileName )

      return aPaths

The method returns a big amount of subfolders and files but definetly not all that I've found.

What's wrong with my method (or is it a wrong usage of os.walk)?

For those who are interested in the Background:

http://www.playonlinux.com/en/topic-10962-centralized_wineprefix_as_preparation_for_debpackages.html

Answer 1

I know that you already solved the problem. But for further references: If you want to walk through dirs as an 32 bit application running on a 64 bit Windows make sure that you check for the redirected directories.

The %windir%\System32 directory is reserved for 64-bit applications. Most DLL file names were not changed when 64-bit versions of the DLLs were created, so 32-bit versions of the DLLs are stored in a different directory. WOW64 hides this difference by using a file system redirector.

File System Redirector on MSDN

Answer 2

both of your hints brought the final solution that looks like that now:

def walkThroughPath(self , sBasePath, blFolders = True, blFiles = True, blFollowSymlinks = True  ):
    aPaths = []
    for sRootDir, aSubFolders, aFiles in os.walk( sBasePath, blFollowSymlinks ):
        for sFolder in aSubFolders:
            if blFolders == True:
                try:
                    aPaths.index( sRootDir )
                    blPathExists = True
                except:
                    blPathExists = False
                    pass

                if blPathExists == False:
                    aPaths.append( sRootDir )
                    self.logDebug("Append: " + sRootDir )
                    self.logDebug("Current content of aPaths: \n" + pprint.pformat(aPaths) )

        for sFileName in aFiles:
            self.logDebug("Current root dir: " + sRootDir )
            if blFiles == True:
                try:
                    aPaths.index( sRootDir + "/" + sFileName )
                    blPathExists = True
                except:
                    blPathExists = False
                    pass

                if  blPathExists == False:
                    aPaths.append( sRootDir + "/" + sFileName )
            if blFolders == True:
                try:
                    aPaths.index( sRootDir )
                    blPathExists = True
                except:
                    blPathExists = False
                    pass

                if blPathExists == False:
                    aPaths.append( sRootDir )
                    self.logDebug("Append: " + sRootDir )
                    self.logDebug("Current content of aPaths: \n" + pprint.pformat(aPaths) )

    self.logDebug("Folders: " + str(blFolders) )    
    self.logDebug("Files  : " + str(blFiles) )  
    self.logDebug("Paths found in " + sBasePath + " : \n" + pprint.pformat(aPaths) )    
    return aPaths

First I indented incorrectly as Steven said.

os.walk seems to handle the lists not as I expected them to be. Folders of files not nessessarily appear in folders list. This caused many folders I left out just because those folder pathes have been in the files list. Additionally I only checked files only in this limited folders list.

Next I added the follow symlinks flag optional as unutbu suggested. Maybe in my case they could be needed as well eventually.

Those method above is surely a candidate for improvement, but it's at least working :-)

Best,

André

Answer 3

Here are two possibilities:

• You don't have permission to read a certain directory.

• By default, os.walk does not follow symbolic links. Use the followlinks=True keyword to follow symbolic links:

  os.walk( sBasePath, followlinks=True )
  

Having skimmed the link you provided, it looks like followlinks=True may be the solution.