CODEWITHSUNDEEP
windowsbatch-filecommand-lineinformatica
mgm_data
mgm_data

Reputation: 57

batch script to generate a list with file names newer than a saved date timestamp

I have a landing directory where we receive 10-15 files everyday. I then run a batch script to create a list with file names that have newly landed and then informatica reads the list of file names to process the new source files.

The problem here is, if a file is edited on same day after informatica loads the file. My batch script does not identify the updated file because the file has same date and name.

Is there a way to compare files based on timestamp and generate a file list? Any help will be greatly appreciated. Thanks! The current batch script code: 

rem this batch script is used to list all the files hosted newly to a xyz landing directory
set LandingDir=\\path\to\landing\directory\*.csv

set DateFile=%sys_var%\script\DateFile.txt
set LastRunDateFile=%sys_var%\scripts\LastRunDateFile.txt
set Temp_File_List=%sys_var%\scripts\Temp_File_List.txt
set File_List=%sys_var%\SrcFiles\File_List.txt
set /P _PreviousDate=<%DateFile%
type %DateFile% > %LastRunDateFile%
xcopy "%LandingDir%" /l /s /d:%_PreviousDate% .>%Temp_File_List%
type %Temp_File_List% | findstr /v File(s)>%File_List%
echo %date:~4,2%-%date:~7,2%-%date:~10,4% >%DateFile%

Upvotes: 0

Views: 1967

Answers (1)

Mofi
Mofi

Reputation: 49086

On Windows there is the archive attribute always set on a file automatically if a file is modified in any way.

Using the archive file attribute makes the task much easier than storing last modification files times of all files processed and comparing last modification file times on next run.

All needed to be done is removing archive attribute on file being already processed, i.e. added to the file list.

Example:

@echo off
setlocal
set "FilePattern=*.csv"
set "sys_var=C:\Temp\Test"
set "File_List=%sys_var%\SrcFiles\File_List.txt"
set "LandingDir=\\server\share\path\to\landing\directory"

if exist "%File_List%" del "%File_List%"

for /F "delims=" %%I in ('dir "%LandingDir%\%FilePattern%" /AA-D /B 2^>nul') do (
    echo %LandingDir%\%%I>>"%File_List%"
    %SystemRoot%\System32\attrib.exe -a "%LandingDir%\%%I"
)

if not exist "%File_List%" echo No new file!
endlocal

The command DIR returns because of /AA-D just files (not directories) with archive attribute set in bare format because of /B.

So output by DIR and processed by FOR are just the names of the files with archive attribute set without path and always without surrounding double quotes even if the file name contains a space or another special character.

The file names would be returned by DIR with full path on using additionally DIR option /S for listing recursively all files in specified directory and in all subdirectories matching the file pattern (and having archive attribute set).

Each file name is written into the file list file and then the archive attribute is removed from the file to ignore this file automatically on next run of the batch file except the archive attribute is set again because the file was modified in the meantime.

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

  • attrib /?
  • del /?
  • dir /?
  • echo /?
  • endlocal /?
  • for /?
  • set /?
  • setlocal /?

See also the Microsoft article Using command redirection operators for an explanation of >> and 2>nul with escaping > with ^ to be interpreted on execution of DIR instead of FOR.

Upvotes: 1

Related Questions