Reading metadata with Python

Question

For the past two days I have been scanning the Internet to try to find the solution to my problem. I have a folder of different files. They run the gambit of file types. I am trying to write a python script that will read the metadata from each file, if it exists. The intent is to eventually output the data to a file to compare with another program's metadata extraction.

I have found some examples where it worked for a very few number of the files in the directory. All the ways I have found have dealt with opening a Storage Container object. I am new to Python and am not sure what a Storage Container object is. I just know that most of my files error out when trying to use

pythoncom.StgOpenStorage(<File Name>, None, flags)

With the few that actually work, I am able to get the main metadata tags, like Title, Subject, Author, Created, etc.

Does anyone know a way other than Storage Containers to get to the metadata? Also, if there is an easier way to do this with another language, by all means, suggest it.

Thanks

Answer 1

I know this is an old question, but I had the same problem and ended up creating a package to solve my problem: windows-metadata.

An aside, Roger Upole's answer was a good starting point, however it doesn't capture all the attributes a file can have (the break if not colname ends the loop too soon, since Windows skips some column numbers, for whatever reason. So Roger's answer gives the first 30 or so attributes, when there are actually nearly 320).

Now, to answer the question using this package:

from windows_metadata import WindowsAttributes

attr = WindowsAttributes(<File Name>)  # this will load all the filled attributes a file has
title = attr["Title"] # dict-like access
title = attr.title # attribute like access -> these two will return the same value
subject = attr.subject
author = attr.author
...

And so on for any available attributes a file has.

Answer 2

Roger Upole's answer helped immensely. However, I also needed to read the "last saved by" detail in an ".xls" file.

XLS file attributes can be read with win32com. The Workbook object has a BuiltinDocumentProperties. https://gist.github.com/justengel/87bac3355b1a925288c59500d2ce6ef5

import os
import win32com.client  # Requires "pip install pywin32"


__all__ = ['get_xl_properties', 'get_file_details']


# https://learn.microsoft.com/en-us/dotnet/api/microsoft.office.tools.excel.workbook.builtindocumentproperties?view=vsto-2017
BUILTIN_XLS_ATTRS = ['Title', 'Subject', 'Author', 'Keywords', 'Comments', 'Template', 'Last Author', 'Revision Number',
                     'Application Name', 'Last Print Date', 'Creation Date', 'Last Save Time', 'Total Editing Time',
                     'Number of Pages', 'Number of Words', 'Number of Characters', 'Security', 'Category', 'Format',
                     'Manager', 'Company', 'Number of Btyes', 'Number of Lines', 'Number of Paragraphs',
                     'Number of Slides', 'Number of Notes', 'Number of Hidden Slides', 'Number of Multimedia Clips',
                     'Hyperlink Base', 'Number of Characters (with spaces)']


def get_xl_properties(filename, xl=None):
    """Return the known XLS file attributes for the given .xls filename."""
    quit = False
    if xl is None:
        xl = win32com.client.DispatchEx('Excel.Application')
        quit = True

    # Open the workbook
    wb = xl.Workbooks.Open(filename)

    # Save the attributes in a dictionary
    attrs = {}
    for attrname in BUILTIN_XLS_ATTRS:
        try:
            val = wb.BuiltinDocumentProperties(attrname).Value
            if val:
                attrs[attrname] = val
        except:
            pass

    # Quit the excel application
    if quit:
        try:
            xl.Quit()
            del xl
        except:
            pass

    return attrs


def get_file_details(directory, filenames=None):
    """Collect the a file or list of files attributes.
    Args:
        directory (str): Directory or filename to get attributes for
        filenames (str/list/tuple): If the given directory is a directory then a filename or list of files must be given
    Returns:
         file_attrs (dict): Dictionary of {filename: {attribute_name: value}} or dictionary of {attribute_name: value}
            if a single file is given.
    """
    if os.path.isfile(directory):
        directory, filenames = os.path.dirname(directory), [os.path.basename(directory)]
    elif filenames is None:
        filenames = os.listdir(directory)
    elif not isinstance(filenames, (list, tuple)):
        filenames = [filenames]

    if not os.path.exists(directory):
        raise ValueError('The given directory does not exist!')

    # Open the com object
    sh = win32com.client.gencache.EnsureDispatch('Shell.Application', 0)  # Generates local compiled with make.py
    ns = sh.NameSpace(os.path.abspath(directory))

    # Get the directory file attribute column names
    cols = {}
    for i in range(512):  # 308 seemed to be max for excel file
        attrname = ns.GetDetailsOf(None, i)
        if attrname:
            cols[i] = attrname

    # Get the information for the files.
    files = {}
    for file in filenames:
        item = ns.ParseName(os.path.basename(file))
        files[os.path.abspath(item.Path)] = attrs = {}  # Store attributes in dictionary

        # Save attributes
        for i, attrname in cols.items():
            attrs[attrname] = ns.GetDetailsOf(item, i)

        # For xls file save special properties
        if os.path.splitext(file)[-1] == '.xls':
            xls_attrs = get_xl_properties(item.Path)
            attrs.update(xls_attrs)

    # Clean up the com object
    try:
        del sh
    except:
        pass

    if len(files) == 1:
        return files[list(files.keys())[0]]
    return files


if __name__ == '__main__':
    import argparse

    P = argparse.ArgumentParser(description="Read and print file details.")
    P.add_argument('filename', type=str, help='Filename to read and print the details for.')
    P.add_argument('-v', '--show-empty', action='store_true', help='If given print keys with empty values.')
    ARGS = P.parse_args()

    # Argparse Variables
    FILENAME = ARGS.filename
    SHOW_EMPTY = ARGS.show_empty
    DETAILS = get_file_details(FILENAME)

    print(os.path.abspath(FILENAME))
    for k, v in DETAILS.items():
        if v or SHOW_EMPTY:
            print('\t', k, '=', v)

Answer 3

I decided to write my own answer as an attempt to combine and clarify the answers above (which heavily helped me solve my problems).

I'd say there are two approaches to this problem.

Situation 1: you know which metadata the file contains (which metadata you're interested in).

In this case, lets say you have a list of strings, which contains the metadata you're interested in. I assume here that these tags are correct (i.e. you're not interested in the number of pixels of a .txt file).

metadata = ['Name', 'Size', 'Item type', 'Date modified', 'Date created']

Now, using the code provided by Greedo and Roger Upole I created a function which accepts the file's full path and name separately and returns a dictionary containing the metadata of interest:

def get_file_metadata(path, filename, metadata):
    # Path shouldn't end with backslash, i.e. "E:\Images\Paris"
    # filename must include extension, i.e. "PID manual.pdf"
    # Returns dictionary containing all file metadata.
    sh = win32com.client.gencache.EnsureDispatch('Shell.Application', 0)
    ns = sh.NameSpace(path)

    # Enumeration is necessary because ns.GetDetailsOf only accepts an integer as 2nd argument
    file_metadata = dict()
    item = ns.ParseName(str(filename))
    for ind, attribute in enumerate(metadata):
        attr_value = ns.GetDetailsOf(item, ind)
        if attr_value:
            file_metadata[attribute] = attr_value

    return file_metadata

# *Note: you must know the total path to the file.*
# Example usage:
if __name__ == '__main__':
    folder = 'E:\Docs\BMW'
    filename = 'BMW series 1 owners manual.pdf'
    metadata = ['Name', 'Size', 'Item type', 'Date modified', 'Date created']
    print(get_file_metadata(folder, filename, metadata))

Results with:

{'Name': 'BMW series 1 owners manual.pdf', 'Size': '11.4 MB', 'Item type': 'Foxit Reader PDF Document', 'Date modified': '8/30/2020 11:10 PM', 'Date created': '8/30/2020 11:10 PM'}

Which is correct, as I just created the file and I use Foxit PDF reader as my main pdf reader. So this function returns a dictionary, where the keys are the metadata tags and the values are the values of those tags for the given file.

Situtation 2: you don't know which metadata the file contains

This is a somewhat tougher situation, especially in terms of optimality. I analyzed the code proposed by Roger Upole, and well, basically, he attempts to read metadata of a None file, which results in him obtaining a list of all possible metadata tags. So I thought it might just be easier to hardcopy this list and then attempt to read every tag. That way, once you're done, you'll have a dictionary containing all tags the file actually possesses.

Simply copy what I THINK is every possible metadata tag and just attempt to obtain all the tags from the file. Basically, just copy this declaration of a python list, and use the code above (replace metadata with this new list):

metadata = ['Name', 'Size', 'Item type', 'Date modified', 'Date created', 'Date accessed', 'Attributes', 'Offline status', 'Availability', 'Perceived type', 'Owner', 'Kind', 'Date taken', 'Contributing artists', 'Album', 'Year', 'Genre', 'Conductors', 'Tags', 'Rating', 'Authors', 'Title', 'Subject', 'Categories', 'Comments', 'Copyright', '#', 'Length', 'Bit rate', 'Protected', 'Camera model', 'Dimensions', 'Camera maker', 'Company', 'File description', 'Masters keywords', 'Masters keywords']

I don't think this is a great solution, but on the other hand, you can keep this list as a global variable and then use it without needing to pass it to every function call. For the sake of completness, here is the output of the previous function using this new metadata list:

{'Name': 'BMW series 1 owners manual.pdf', 'Size': '11.4 MB', 'Item type': 'Foxit Reader PDF Document', 'Date modified': '8/30/2020 11:10 PM', 'Date created': '8/30/2020 11:10 PM', 'Date accessed': '8/30/2020 11:10 PM', 'Attributes': 'A', 'Perceived type': 'Unspecified', 'Owner': 'KEMALS-ASPIRE-E\\kemal', 'Kind': 'Document', 'Rating': 'Unrated'}

As you can see, the dictionary returned now contains all the metadata that the file contains. The reason this works is because of the if statement:

if attribute_value:

which means that whenever an attribute is equal to None, it won't be added to the returning dictionary.

I'd underline that in case of processing many files it would be better to declare the list as a global/static variable, instead of passing it to the function every time.

Answer 4

Windows API Code Pack may be used with Python for .NET to read/write file metadata.

1. Download the NuGet packages for WindowsAPICodePack-Core and WindowsAPICodePack-Shell

2. Extract the .nupkg files with a compression utility like 7-Zip to the script's path or someplace defined in the system path variable.

3. Install Python for .NET with pip install pythonnet.

Example code to get and set the title of an MP4 video:

import clr
clr.AddReference("Microsoft.WindowsAPICodePack")
clr.AddReference("Microsoft.WindowsAPICodePack.Shell")
from Microsoft.WindowsAPICodePack.Shell import ShellFile

# create shell file object
f = ShellFile.FromFilePath(r'movie..mp4')

# read video title
print(f.Properties.System.Title.Value)

# set video title
f.Properties.System.Title.Value = 'My video'

Hack to check available properties:

dir(f.Properties.System)

Answer 5

You can use the Shell com objects to retrieve any metadata visible in Explorer:

import win32com.client
sh=win32com.client.gencache.EnsureDispatch('Shell.Application',0)
ns = sh.NameSpace(r'm:\music\Aerosmith\Classics Live!')
colnum = 0
columns = []
while True:
    colname=ns.GetDetailsOf(None, colnum)
    if not colname:
        break
    columns.append(colname)
    colnum += 1

for item in ns.Items():
    print (item.Path)
    for colnum in range(len(columns)):
        colval=ns.GetDetailsOf(item, colnum)
        if colval:
            print('\t', columns[colnum], colval)

Answer 6

The problem is that there are two ways that Windows stores file metadata. The approach you're using is suitable for files created by COM applications; this data is included inside the file itself. However, with the introduction of NTFS5, any file can contain metadata as part of an alternate data stream. So it's possible the files that succeed are COM-app created ones, and the ones that are failing aren't.

Here's a possibly more robust way of dealing with the COM-app created files: Get document summary information from any file.

With alternate data streams, it's possible to read them directly:

meta = open('myfile.ext:StreamName').read()

Update: okay, now I see none of this is relevant because you were after document metadata and not file metadata. What a difference clarity in a question can make :|

Try this: How to retrieve author of a office file in python?