Batch File Variable to copy specific files

Question

I'm trying to copy specific files from C: to "X: for example". The files are named with the same format.

A1234_ZZabc123_DT1_F1.tst 
A4567_ZZdef4567_DT2_F2.tst 
A8901_ZZghi1289_DT1.tst 
A2345_ZZjfkbu12_to_modify.tst 
A6789_ZZlmny568_F1_to_modify.tst 
A1234_ZZabc478_DT1.txt 

I want to copy only the .tst files, and with the same format as the first 3 Axxxx_ZZyyyxxx_DTx_Fx.tst where x=number and y=letter.

After ZZ, it might be 4 letters and 3 numbers, or 5 letters and 4 numbers, like a "namecode". Example: ZZkusha122 or ZZkus1551. I need to copy the folders along with the files too.

I'm new to coding and really need some help.

I need to find and copy all those files along 10k+ files together

Answer 1

You claim that the first 3 of your example filenames fit the pattern you describe. I believe that only two do.

@ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=u:\your files"

FOR /f "delims=" %%e IN (
 'dir /b /a-d "%sourcedir%\A*.tst"^|findstr /X /I /R "A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_DT[0-9]_F[0-9].tst" '
 ) DO ECHO COPY "%sourcedir%\%%e" X:
)

GOTO :EOF

Always verify against a test directory before applying to real data.

The required COPY commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO COPY to COPY to actually copy the files. Append >nul to suppress report messages (eg. 1 file copied)

Simply execute a dir command that reports only filenames matching the *.tst mask. Filter this list with findstr which /X exactly matches the regular expression provided. findstr only has a limited implementation of regular expressions. The /I forces a case-insensitive match. If you want case-sensitive, remove the /I and change each [a-z] to [a-zA-Z] (leave as-is if you want lower-case only in these positions.)

See findstr /? from the prompt for more documentation, or search for examples on SO.

---- revision to cater for multiple filemasks and subdirectories ---

@ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=u:\your files"
SET "maskfile=%sourcedir%\q74442552.txt"

FOR /f "tokens=1*delims=" %%e IN (
 'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
 ) DO ECHO COPY "%%e" X:
)

GOTO :EOF

Changes:

1. Establish a file, name irrelevant, as maskfile
   
2. The dir command requires the /s switch to scan subdirectories
   
3. The filemask for the dir command loses the initial A
   
4. The findstr command replaces the /X switch with /E
   
5. The findstr command loses the regex expression. These are transferred to a file and the file is nominated by the /g: switch.
   
6. The copy command loses the source-directory as the directory will be included in %%e
   

The file "q74442552.txt" contains lines that are of the form

A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_DT[0-9]_F[0-9].tst
A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_to.*.tst

This time, %%e acquires the full pathname of the files found. Since the filemask ends .tst, the only filenames to pass the dir filter will be those that end .tst.

The /e switch tells findstr to match string that End with the regex strings in the file specified as /g:.

The strings in the file must comply with Microsoft's partial regex implementation, one to a line.

In summary, findstr uses as regex

• Any character,literally
• [set] any character of a set of characters
• [^set] any character not in a set of characters
• . any character
• .* any number of any character
• prefix any of the special characters with\ to use it literally
• a set may include a range by using low-high

So - you then need to brew-your own using the examples I've supplied. The second line matches Axxxx_ZZyyy_to{anything}.tst for instance.

--- Minor revision to deal with maintaining destination-tree ----- (see notes to final revision for why this doesn't quite work)

@ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=u:\your files"
SET "maskfile=%sourcedir%\q74442552.txt"
SET "destdir=u:\your results"

FOR /f "tokens=1*delims=" %%e IN (
 'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
 ) DO ECHO "%%~nxe"&XCOPY /Y /D /S "%sourcedir%\%%~nxe" "%destdir%\">nul
)
   
GOTO :EOF

This version adds the destination root directory as destdir.

The dir ... findstr... works as before to list the filenames to copy.

The prior version used echo copy to report the proposed copy operation, but the destination was always the same directory.

The replacement XCOPY line maintains the directory structure at the destination.

Note : the XCOPY is "live". The files will be copied to the destination if run as-is. Always verify against a test directory before applying to real data.

To "defuse" the XCOPY, add the /L switch and remove the >nul. This will cause XCOPY to report the source name that would be copied instead of copying it. (The >nul suppresses the report)

The /D only copies source files that eitherr do not exist in the destination of have a later datestamp in the source.

The action is to xcopy each filename found (%%~nxe) from the source directory tree to the destination. Therefore, any file xyz.tst found anywhere in the source tree will be xcopyd to the destination tree. The /D means that once xyz.tst is encountered on the source tree, it will be skipped should it be encountered again.

--- Final (I hope) revision ---

@ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.

SET "sourcedir=U:\Users\tocil\Desktop\aoi"
SET "maskfile=%sourcedir%\q74442552.txt"
SET "destdir=u:\your results"

FOR /f "tokens=1*delims=" %%e IN (
 'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
 ) DO (
  rem drive and path to 'dirname' - has terminal "\"
  SET "dirname=%%~dpe"
  rem remove the sourcedir from dirname
  FOR %%y IN ("%sourcedir%") DO CALL SET "dirname=%%dirname:%%~y=%%"
  rem copy or xcopy the file to the destination. 
  FOR /f "tokens=2delims==" %%y IN ('set dirname') DO XCOPY /Y "%%e" "%destdir%%%y">nul
 )
)

GOTO :EOF

Always verify against a test directory before applying to real data.

Note to self: Only if the filemask provided to XCOPY is ambiguous (ie. contains ? or *) will XCOPY obey the /s switch unless the target file exists in the starting source directory.

hence

xcopy /s sourcedir\myfile destdir

will copy myfile from the entire tree ONLY if sourcedir\myfile exists.

xcopy /s sourcedir\myf?le destdir

will copy myfile from the entire tree regardless. Unfortunately it will also copy myfale and myfule as well.

Hence, the new approach.

First, perform a dir /b /s to get all of the filenames and filter as before. This is being assigned to %%e.

Take the drive and path only of the file and assign to dirname.

The next step is a little complex. First, set the value of %%y to the name of the source directory. Next, use a parser trick to remove that name from dirname. The mechanics are: Parse the %%dirname:%%~y=%% (because the call causes the set to be executed in a sub-shell) whuch does the normal left-to-right evaluation. %% is an escaped-%, so is replaced by %; %%y is an active metavariable so is replaced by (the name of the source directory) and the ~ causes the quotes to be stripped from that name. The resultant command executed is thus SET "dirname=%dirname:nameofsourcedirectory=%"

So now we can construct a copy-class instruction. dirname now contains the relative directory for the destination, which we can extract from the environment by parsing a set listing (Could also be done with delayed expansion) where %%y gets set to the relative directory and has both a leading and trailing backslash, so the destination directory is simply "%destdir%%%y". XCOPY then knows to create that directory if necessary (%%y has a trailing backslash) and we know the source filename is in %%e.

You could also use a copy to do the same thing, but you'd need to create the destination directory first. Another advantage of XCOPY is that you can also specify the /d switch to not copy files that have an earlier date over files that have a later date.