Reputation:
I have following Directory structure,
F:\TestData
and TestData contains 20 folders with name node1, node2,..., node20
and each node folder contains file with names log.10.X
I need to access each log file from all node folders, For which I have writen code, but it is saying, File not found - log.*
CODE:
directory = "F:\TestData"
p = subprocess.Popen(["find", "./" + directory, "-name", "log.*"], stdout=subprocess.PIPE)
output, err = p.communicate()
foutput = output.split("\n")
Upvotes: 0
Views: 455
Reputation: 1877
You also use walk
, like this:
import os
directory = "F:\TestData"
for i in os.walk(directory):
# i like this:
# ('F:\\TestData', ['node1', 'node2', 'node3'], [])
# ('F:\\TestData\\node1', [], ['log.1.txt'])
# ('F:\\TestData\\node2', [], ['log.2.txt'])
print i
if i[2] != []:
# TODO: use the path to finish other
# If dictory noden have some log file, you should use i[2][n].
# So, if you only need log.n.txt, you only use i[2][n].
print os.path.join(i[0], i[2][0])
Upvotes: 0
Reputation: 11
Python's glob module may be an option.
import glob
directory = 'F:\TestData'
logcontents = [open(f,'r').read() for f in glob.glob(directory + '\node*\log.*')]
Upvotes: 0
Reputation: 1548
Python, unlike POSIX shells, does not automatically do globbing (interpreting *
and the like as wildcards related to files in the relevant directory) in strings. It does, however, provide a glob
module for that purpose. You can use this to get a list of matching filenames:
import glob
filenames = glob.glob(r'F:\TestData\node*\log.*')
Upvotes: 1
Reputation: 101
You can just use python to get a list of files in the directory
import os
directory = "F:\TestData\"
file_list = os.listdir(directory)
log_list = filter(lambda x: x.startswith("log"), file_list)
oh, you have to code to iterate sub directory.
First os.listdir()
in the parent directory ,and iterate the sub directory to get the files
Upvotes: 0