Batchfile: read last lines from logfiles and copy them to a new file

Question

This is my first posting so if the format is not as it supposed to be please excuse me for this. (Suggestions for

improvement are welcome.)

I am trying to create a batchfile that will read last lines from logfiles and copy them to a new file.

Until now I have found here a way to read the last line.

Code would be something like:

for /f %%i in ('find /v /c "" ^< someFile.txt') do set /a lines=%%i
set /a startLine=%lines% - 1
more /e +%startLine% someFile.txt > lastLines.txt

The above code works for one file at a time. What I need is to read the last line from all files in a known list and add this line to a new .csv file.

I have been using the following code for getting the 4th entry in the logfiles but it returns every line of every logfile:

for /f %%x in (%list%) do for /f "delims=.txt,  tokens=4" %%i in (%%x.txt) do echo %%x, %%i >> output.csv

What I would need is a sort of combination of both but I don't know how to combine them and make the complete last line be copied to the .csv file.

=== @Magoo: Thanx for your reaction. In every logfile can be 1 to >100 lines with comma separated information. Something like:

"LOGON,6-1-2015,12:43:39,USERNAME,HOSTNAME,,,,192.168.209.242,00:21:5A:2E:64:5E" 

The last code with the 4th entry was used to get a list of all accounts that had logged in to the computers. This code gave me a very large list of all logon/logoff events on all computerlogs I checked in %list%. In %list$ I had all the names of logfiles I wanted to be checked. This returned all lines.

For a new batchfile I need only the last logon/logoff entry and I want the whole last line.

So I have a .txt file with the hostnames of all computers I need to examine. This .txt file will be read line by line via the variable %list%.

From every logfile I need only the last line copied to an output file.

=== I just tried the solution offered by JosefZ. Unfortunately this does not work for me yet. No lastlines are copied to the resultfile. In the code I removed the extra entry for possible lastlines for there are no empty lines in the logs, I also added an entry for the hostname I want to be available in the result. JosefZ had the filename there:

@ECHO OFF >NUL
@SETLOCAL enableextensions disabledelayedexpansion    
type nul>output.csv    
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
  set "host=%%~x" 
  for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"
  call :lline
)    
:endlocal
@ENDLOCAL
goto :eof    
:lline    
  set "filename=.\logs\%filename:&=^&%.txt"
  echo %host%,%lastline%>>output.csv
goto :eof

The resultfile shows only the hostnames. I'll puzzle some more with this but all tips are welcome!

=== Got it!!!

@ECHO OFF >NUL
@SETLOCAL enableextensions disabledelayedexpansion

type nul>output.csv

set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
  set filename=   :: *empty previous filename*
  set lastline=   :: *empty previous lastline* 
  set "host=%%~x" 
  set "filename=.\logs\%host%.txt"  :: *creating the filename from path+hostname+extention*
  for /F "tokens=*" %%G in ('type "%filename%"') do set "lastline=%%G"
  call :lline
)

:endlocal
@ENDLOCAL
goto :eof

:lline

  echo %host%,%lastline%>>output.csv

goto :eof

Answer 1

Your approach with line numbering could fail if a file has more trailing empty lines. Fortunately for /F loop ignores (does not iterate) empty lines; let's put to use this feature: in the script used next practices:

• disabledelayedexpansion to allow ! in file names
• set "list=_listing.txt" where the _listing.txt contains list of file names (full path and extension .txt including), one file name on one line: got by dir /b /s *.txt>_listing.txt
• type nul>files\output.csv to empty the output file (optional)
• set "lastline=!!!file empty!!!" to initialize variable %lastline%; could be set "lastline=" as well
• call :lline to process variables %filename% and %lastline%
• set "filename=%filename:&=^&%" to allow & in file names

The script is as follows:

@ECHO OFF >NUL
@SETLOCAL enableextensions disabledelayedexpansion
type nul>files\output.csv

set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
  set "filename=%%~x"
  set "lastline=!!!file empty!!!"

  rem the whole line
  for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"

  rem the fourth token only
  rem for /F "tokens=4" %%G in ('type "%%~x"') do set "lastline=%%G"

  call :lline
)
:endlocal
@ENDLOCAL
goto :eof

:lline
  set "filename=%filename:&=^&%"
  echo %filename% %lastline%
  rem >>files\output.csv
goto :eof

Sample _listing.txt file:

d:\bat\files\1exclam!ation.txt
d:\bat\files\2exc!lam!ation.txt
d:\bat\files\11per%cent.txt
d:\bat\files\12per%cent%.txt
d:\bat\files\17per%Gcent.txt
d:\bat\files\18per%%Gcent.txt
d:\bat\files\21ampers&nd.txt
d:\bat\files\22ampers&&nd.txt

Output:

d:\bat>lastlines
d:\bat\files\1exclam!ation.txt 0 15.01.2015  1:52:28.48 -15072 20465
d:\bat\files\2exc!lam!ation.txt 6 15.01.2015  1:52:28.50 3250 16741
d:\bat\files\11per%cent.txt -8 15.01.2015  1:52:28.50 -3692 27910
d:\bat\files\12per%cent%.txt !!!file empty!!!
d:\bat\files\17per%Gcent.txt 0 15.01.2015  1:52:28.56 14508 12374
d:\bat\files\18per%%Gcent.txt 1 15.01.2015  1:52:28.56 30540 26959
d:\bat\files\21ampers&nd.txt 15.01.2015  1:22:50.18
d:\bat\files\22ampers&&nd.txt 15.01.2015  1:22:50.18

Honestly, all that ballast is for (possibly) trailing empty lines in files and for (possibly) ! and & in file names only; all could be done with

for /f %%x in (%list%) do for /f "skip=%startLine% tokens=4" %%i in (%%x) do echo %%x, %%i >> output.csv

Answer 2

You should use a simple FOR to iterate a list of values, not FOR /F.

Something like the following should work:

@echo off
setlocal enableDelayedExpansion
>>output.csv (
  for %%F in (
    "file1.log"
    "file2.log"
    "file3.log"
    etc.
  ) do (
    for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
    more +!skip! %%F
  )
)

The quotes around the file names are there in case you get a name with spaces.

You could use your LIST variable if it looks something like
set LIST="file1.log" "file2.log" "file3.log" etc.

@echo off
setlocal enableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.    
>>output.csv (
  for %%F in (%LIST%) do (
    for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
    more +!skip! %%F
  )
)

If any of your file names contain the ! character, then you must toggle delayed expansion ON and OFF within your loop. Otherwise the delayed expansion will corrupt the names when %%F is expanded.

@echo off
setlocal disableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.    
>>output.csv (
  for %%F in (%LIST%) do (
    for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
    setlocal enableDelayedExpansion
    more +!skip! %%F
    endlocal
  )
)