CODEWITHSUNDEEP
excelpowerpivotpowerbipowerquery
sayth
sayth

Reputation: 7048

PowerQuery - Folder Query import new files

If I have created a PowerQuery function that imports XML from a folder, how in the same excel file do I reuse the query when there are new files in the folder to only include data from those new files and append the data to the current table?

Upvotes: 1

Views: 1246

Answers (2)

Mike Honey
Mike Honey

Reputation: 15017

If you start a Power Query using From File / From Folder and browse to your folder, you will see each file represented as a row in a table, with columns such as Date modified. You can filter that list using Date/Time filters on Date modified or by something more complicated if necessary (post your specific requirements and I'll try to steer you in the right direction).

Once you have filtered the query to just the "new files", you can pass the [Content] column into your Function.

Finally Append a new query based on the saved Excel Table output from your pre-existing query together with the "new files" query above to get your combined output. The new query would be set to Load To / Only Create Connection.

Upvotes: 2

PA.
PA.

Reputation: 29339

you can watch a folder for file changes with a simple vba script that uses WMI to poll the directory contents every n seconds.

Something similar to this ...

Sub WatchDirectory(dir as string, every as integer)    
  Set wmisvc = GetObject("winmgmts:\\.\root\cimv2")
  let query = "SELECT * FROM __InstanceOperationEvent " _
        & "WITHIN " & every _
        & " WHERE Targetinstance ISA 'CIM_DirectoryContainsFile' and " _
        & "TargetInstance.GroupComponent='Win32_Directory.Name=" _
        & Chr(34) & dir & Chr(34) & "'"

  Set events = wmisvc.ExecNotificationQuery(query)

  Do While True
    Set event = events.NextEvent()
    if event.Class = "__InstanceCreationEvent" then 
       ....
    end if
  Loop

For more info on wmi see https://sites.google.com/site/beyondexcel/project-updates/exposingsystemsecretswithvbaandwmiapi

For more details on file watching with WMI, see https://blogs.technet.microsoft.com/heyscriptingguy/2005/04/04/how-can-i-monitor-for-different-types-of-events-with-just-one-script/

Upvotes: 1

Related Questions