Understanding batch files

Question

I'm trying to understand a pretty complex (to me it is anyhow) batch file that somebody else has written.

The person who wrote it is no longer around so it's down to me to try and figure out what the hell is going on.

To start with, I need to understand what this line is doing: -

for /f "skip=4 tokens=1,5" %%A in ('dir /tc /o-d %i%\PICKS*') do (

Can anybody help?

Answer 1

1. How to get help on Windows commands?

Help on any Windows standard command can be get by running the command with parameter /? from within a command prompt window.

Try it out with executing in a command prompt window dir /? and for /?.

Or run this long command line with multiple commands:

dir /? >"%UserProfile%\Desktop\Help_on_DIR_FOR.txt" & for /? >>"%UserProfile%\Desktop\Help_on_DIR_FOR.txt" & %SystemRoot%\Notepad.exe "%UserProfile%\Desktop\Help_on_DIR_FOR.txt"

This is like running the three command lines:

dir /? >"%UserProfile%\Desktop\Help_on_DIR_FOR.txt"
for /? >>"%UserProfile%\Desktop\Help_on_DIR_FOR.txt"
%SystemRoot%\Notepad.exe "%UserProfile%\Desktop\Help_on_DIR_FOR.txt"

It runs command DIR to output its help with redirecting the help into file Help_on_DIR_FOR.txt on the desktop. Next it runs command FOR to output its help with appending it to file Help_on_DIR_FOR.txt. And last Windows Notepad is started to display the file Help_on_DIR_FOR.txt.

See the Microsoft documentation about Using command redirection operators for the details on the redirection operators > and >>.
See the Wikipedia article about Windows Environment Variables for the details on the predefined environment variables UserProfile and SystemRoot.

Other sources for help on Windows commands are:

1. Microsoft's command-line reference
2. SS64.com - A-Z index of the Windows CMD command line

2. What to know about text encoding of batch files?

The text in a batch file is encoded using one byte per character. That means on using Windows Notepad to create/edit a batch file it is necessary to make sure to save the batch file using ANSI encoding.

Text files using a text encoding with only one byte per character are limited to 256 characters. So there must be a table to define which byte value represents which character. There are many such tables as the mankind on this planet uses more than 256 characters. For text encoding the used code page defines the table used for character encoding.

In a Windows command prompt window (console) is used by default an OEM (Original Equipment Manufacturer) code page while in Windows GUI applications like Windows Notepad is used by default a Windows (ANSI) code page.

Which code page is used depends on Windows region setting as defined for the used account. For example in North American countries code page 437 is used for console and Windows-1252 by Windows GUI applications while in Western European countries code page 850 is used for console and also Windows-1252 by Windows GUI applications.

Run in a command prompt window the command chcp to get displayed which code page is used on console according to region (country) set for your account.

The knowledge about code pages is important on writing a batch file using a Windows GUI editor which contains text with characters with a code value greater 127 decimal because for the upper half of the 256 characters the used code page defines which value represents which character.

It can be easily seen that all non ASCII characters of the help are displayed wrong in Notepad in case of not using English Windows and looking on help output for DIR and FOR to console redirected into a text file viewed with Windows Notepad because of the text file was not created using the Windows (ANSI) code page as expected by Notepad.

3. What does the command line ask for?

Let us analyze the command line

for /f "skip=4 tokens=1,5" %%A in ('dir /tc /o-d %i%\PICKS*') do (

by executing following commented batch file:

@echo off
rem Setup a local environment for the commands below.
setlocal EnableExtensions DisableDelayedExpansion

rem Define the directory to process.
set "i=%TEMP%\UnderstandingDirFor"

rem Create this directory temporarily.
md "%i%" 2>nul

rem Create a file with PICKS at beginning of file name in the directory.
echo PICKS_SampleFile.tmp>"%i%\PICKS Sample File.tmp"

rem Copy the batch file into directory with PICKS in file name.
copy "%~0" "%i%\PICKS_ExampleFile.bat" >nul

rem Copy another file into directory with PICKS in file name.
copy "%SystemRoot%\System32\cmd.exe" "%i%\PICKS cmd.exe" >nul

rem Copy a file into directory with a file name not starting with PICKS.
copy "%SystemRoot%\System32\sort.exe" "%i%" >nul

cls
echo First output is from:
echo(
echo dir "%%i%%\PICKS*"
echo(
dir "%i%\PICKS*"
echo(
pause

echo(
echo ===============================================================================
echo(

echo Second output is from:
echo(
echo dir /tc /o-d "%%i%%\PICKS*"
echo(

dir /tc /o-d "%i%\PICKS*"

echo(
pause

echo(
echo ===============================================================================
echo(

echo Third output is from:
echo(
echo for /f "skip=4 tokens=1,5" %%%%I in ('dir /tc /o-d "%%i%%\PICKS*"')
echo(

for /f "skip=4 tokens=1,5" %%I in ('dir /tc /o-d "%i%\PICKS*"') do (
    echo Token 1 = loop variable I = %%I
    echo Token 5 = loop variable J = %%J
)

echo(
pause

echo(
echo ===============================================================================
echo(

echo Fourth output is from:
echo(
echo for /F "skip=5 tokens=1,4*" %%%%I in ('dir /A-D /O-D /TC "%%i%%\PICKS *" 2^^^>nul')
echo(

for /F "skip=5 tokens=1,4*" %%I in ('dir /A-D /O-D /TC "%i%\PICKS *" 2^>nul') do (
    if /I "%%J" == "PICKS" (
        echo Token 1 = loop variable I = %%I
        echo Token 5 = loop variable K = %%K
    )
)

echo(
pause

rem Delete the created directory with the four files.
rd /S /Q "%i%"

rem Purge the local environment and restore initial environment.
endlocal

Note: The batch file uses "%i%\PICKS*" instead of %i%\PICKS* to work also for directory paths containing a space character or one of these characters &()[]{}^=;!'+,`~. The path of the directory for temporary directories and files referenced with %TEMP% included in %i% could contain a space character which requires the usage of double quotes.

The first output into console window is from:

dir "%i%\PICKS*"

It can be seen on output that the files (or directories) matching the wildcard pattern PICKS* are listed with last modification date at beginning and in order as used internally by the file system.

The file sort.exe is not output because it's file name does not start with PICKS.

What DIR outputs exactly depends on language of Windows. But the first five lines are the header and the last three lines are the summary in language of Windows.

The second output into console window is from:

dir /tc /o-d "%i%\PICKS*"

The output of the list is now in reverse order by date/time because of option /o-d using creation date because of the option /tc.

By default the file/directory with the oldest date is output first and the file/directory with the newest date is output last on using /od. The order is reverse with /o-d which means output is first the newest and last the oldest file/directory.

It can be seen on comparing the two outputs that the creation date of all copied files is newer than their last modification date.

The reason is that the creation date is the date a file/directory was created in current directory and NOT when the file itself was created the first time anywhere.

For that reason the creation date is most often not really useful and using the creation date might be also not useful here. But this can't be determined without knowing what the entire batch file is written for.

The third output into console window is from:

for /f "skip=4 tokens=1,5" %%I in ('dir /tc /o-d "%i%\PICKS*"') do (
    echo Token 1 = loop variable I = %%I
    echo Token 5 = loop variable J = %%J
)

FOR starts a new command process in background using cmd.exe with option /C for an automatic close for executing the specified command line dir /tc /o-d "%i%\PICKS*". To be more precise there is executed by FOR:

%ComSpec% /c dir /tc /o-d "C:\Users\UserName\AppData\Local\Temp\PICKS*"

ComSpec is a predefined environment variable with full qualified file name of Windows command processor which is usually C:\Windows\System32\cmd.exe.

The output of the entire command process to handle STDOUT is captured by FOR and processed line by line after background command process finished and closed itself.

for /F ignores by default all empty lines. The other lines are by default first split up into substrings using normal space and horizontal tab as string delimiters which means no substring contains any space/tab. If the first space/tab delimited substring starts now with a semicolon being the default end of line character, the line is not further processed by FOR and so also ignored like an empty line. Then the first substring is assigned to specified loop variable and the single command or the command block is executed next.

Command FOR is instructed with option skip=4 to skip the first four lines of captured output and start processing the output with line five which is the last empty line of header of DIR output.

Command FOR is instructed with option tokens=1,5 to not assign only first space/tab delimited string to specified loop variable I, but also the fifth space/tab delimited string to the loop variable being the next one after I in ASCII table if there is a fifth substring (= token) at all which is loop variable J.

This substrings/tokens to loop variable assignments feature is the reason why loop variables are case-sensitive while environment variables are not case-sensitive.

Command FOR ignores a line if the first specified token as specified with tokens= cannot be determined from a line. But it does not ignore lines on which not all strings of interest can be found in line as it can be seen on third output.

What fifth space/tab delimited string assigned to loop variable J should be in original batch code depends on what the directory referenced with %i% contains - files or directories - and how the file/directory names starting case-insensitive with PICKS really look like.

For the example I assumed PICKS* matches files with file name starting with PICKS  (note the space at end) and more characters not containing a space character.

By looking on third output it can be seen that the names of the three PICKS* are not assigned all to loop variable J as expected and the summary lines are also processed.

I suppose this does not happen in original batch file because of file/directory names do not contain another space character after PICKS and the FOR loop is exited with command GOTO after processing first file/directory name.

The fourth output into console window is from:

for /F "skip=5 tokens=1,4*" %%I in ('dir /A-D /O-D /TC "%i%\PICKS *" 2^>nul') do (
    if /I "%%J" == "PICKS" (
        echo Token 1 = loop variable I = %%I
        echo Token 5 = loop variable K = %%K
    )
)

This is an improved version suitable to output only the creation date and the name of the files in temporary directory matching the wildcard pattern.

The DIR option /A-D is added to ignore subdirectories and get just a list of files of which file name is matched by the wildcard pattern.

The wildcard pattern is modified to PICKS * which results in command DIR not outputting anymore a line for file PICKS_ExampleFile.bat because of the underscore instead of the space in the file name.

On DIR command line 2^>nul is appended for suppressing a perhaps output error message. Command DIR outputs an error message to handle STDERR not processed by FOR and therefore written to console when it can't find a file matching the wildcard pattern in the specified directory or when the directory does not exist at all. This error message is redirected to device NUL to suppress it. The redirection operator > must be escaped here with caret character ^ to be interpreted first as literal character on parsing FOR command line by Windows command interpreter, but as redirection operator on execution of DIR command line in background command process.

The skip option of command FOR is modified to skip=5 to skip the first 5 lines, i.e. the entire header block of command DIR.

The tokens option of command FOR is modified to tokens=1,4*. This results on lines containing a file name matching the wildcard pattern PICKS * in assigning creation date of file to loop variable I, PICKS to loop variable J and everything after the space(s) after PICKS in line without further splitting up on spaces/tabs to loop variable K.

The IF condition compares case-insensitive, because of /I, the string assigned to loop variable J with PICKS and outputs creation date and file name only if the compared strings are equal. This IF condition filters out the summary lines on output.

Because of * in tokens=1,4* the entire file name with the exception of PICKS  is output even if the file name contains more space characters like in Sample File.tmp.

4. How to debug a batch file?

Windows command processor outputs by default each command line respectively command block defined with ( ... ) after processing it before execution. The output of the command lines/blocks before execution can be disabled with using @echo off at top of the batch file whereby @ prevents already the output of this command line.

But on debugging a batch file it is often necessary to see what is really executed. This can be achieved with either removing @echo off, or changing it to @echo ON or comment this command line out with @rem @echo off or with ::@echo off whereby the last solution changes the command line into an invalid label line.

It is of course also possible to keep @echo off at top of the batch file and use inside the batch file echo on and some lines below @echo off to get displayed on execution just the commands between those two command lines to debug just a specific block of the batch file.

And for debugging the batch file it should not be executed by just double clicking on it in Windows Explorer. This results in starting cmd.exe implicit with option /C for executing the batch file and close the command process and therefore also the console window immediately on exiting the batch file execution.

The closing of the console window is not good for debugging a batch file because the executed commands can't be seen and also the error message output by Windows command processor on detecting a syntax error resulting in an immediate exit of batch file processing is not readable with console window automatically closed.

For that reason debugging a batch file should be done by opening a command prompt window which results in implicit execution of cmd.exe with option /K to keep command process running and its console window open after execution of the batch file which is entered with its file name and if needed with full path in the command prompt window for execution.

Another advantage of running the batch file from within a command prompt window is the possibility to use UP and DOWN keys to re-select a once entered command line or string entered on a user prompt within batch file with set /P from input buffer of command process.

Answer 2

That's a for-loop with the /f option. It loops through a file or command, in this case the command dir /tc /o-d %i%\PICKS*.

Take a look at what dir /tc /o-d word* on its own does (read up on options wtih dir /?), it'll make understanding the loop easier.

Basically, the output of the command is dir (show files in directory) with the timefield showing the creation date and all files sorted by date in descending order. The command applies to the folder %i% (a variable set beforehand). "PICKS*" is the 'mask', i.e. all files starting with "PICKS" are shown. That's what the wildcard (asterisk) means (any characters can follow).

The for-loop now takes that output and assigns it to %%A (temporary iteration variable). It skips the first 4 lines (in the case of dir just header that you don't want to have), and for each following line (the files) picks out the first and the fifth word (delimited by spaces, as space is the default delimiter). %%A is now the date, %%B (the second specified token/fifth word) the second word of the filename (if there is any). Those variables can be used in the do (... part.