Monitoring file IO read and write events on Windows

Question

I want to monitor access to some files on Windows.

What I need:

• Watching ALL connected drives for READ or WRITE events on files (NOT the create/delete events), doing the filtering later
• getting those events 'live' (I can live with a delay of a few seconds, but it should be a monitoring tool running in the background, so analyzing manually saved/exported log files is not an option)
• Get the file name plus number of bytes read or written (and possibly the position inside the file)
• similar to tools like procmon.exe (which AFAIK does not support real-time logging to another process, only manually exporting the logs to a file), just a lot less infos needed
• similar to Windows' resmon's 'Disk' tab
• Windows 10 and above is OK, does not need much of a backwards compatibility

What I do NOT need:

• name or ID of the accessing process
• java.nio.file.WatchService because it cannot raise the read event, nor give info about the amount of bytes
• • nor its Apache couterpart org.apache.commons.io, because that's even less suited

So as you might have noticed, the final thing shall run in Java, but I'm OK with wrapping an app and parsing its output from its stdio's InputStream (and ErrorInputStream). So, the central piece could also be a PowerShell script or .exe file or something alike.

I have scoured the internet and a few Microsoft articles, and there was a plethora of maybe possible solutions, but all quite complicated, like writing a driver in C and JNIing it, etc.

I do not mind using JNI or other means to access the Windows API though.

So I'm hoping there's an easier way to get this done than to write a driver...

Answer 1

ETW would be a good fit for this use case. The FileIo class has subclasses including FileIo_Name, which you can use to temporally associate file names with FileKeys, and FileIo_ReadWrite, which contain information on reads/writes associated with one of those FileKeys.

If you're looking to do this in C and wrap it with JNI, Microsoft has published an example that is similar to what you'd need to do in order to consume these events. Note that instead of EVENT_TRACE_FLAG_NETWORK_TCPIP you'd want to specify EVENT_TRACE_FLAG_DISK_FILE_IO, as mentioned in the FileIo docs. This example also logs to a file; to instead log into memory, see discussion on the docs for EVENT_TRACE_PROPERTIES. An example of parsing the events using the MOF classes can be found here.

You might also take inspiration from ProcMonX, which consumes ETW events in C#. .NET's Microsoft.Diagnostics.Tracing.TraceEvent library (used by ProcMonX) and reflection certainly make this much more straightforward. If you're okay with hooking up .NET to Java (I don't know how feasible this is myself) or especially if you're okay with having a separate process, this is likely the far easier path forward when compared with using the lower-level C APIs.

ProcMonX v2 also exists, which does this in C++, largely using the Tdh family of APIs.