Batch File file output, string manipulation

Question

I am hoping that you can help and explain what the problem is. Like most problems I am sure that this will be very simple to some who uses Batch Scripts regularly.

The challenge which I am trying to solve is to take a list of XML files from a folder, pass them into a Sha1 exe, receive the output, modify it, combine the result and then finally output the result to a file. I was hoping that I didn't have to output to two files but I can't get the Sha1 output to a variable to manipulate it.

What is the problem? Currently when the output of the text comes back in it looks like "size= size:3" and when I try and manipulate string it messes with the output. Example below.

The Code:

set "xmlfilelocation=C:\Users\ADMIN\Documents\test"
set "manifestfile=C:\Users\ADMIN\Documents\manifest.txt"
set "Sha1=C:\Users\ADMIN\\sha1.exe"
set "sha1output=C:\Users\ADMIN\Documents\sha1output.txt"

)
if exist %xmlfilelocation% (
    for /f %%f in ('dir /b %xmlfilelocation%') do (

        rem Takes the file name which is currently been passed to the Sha1 engine
        set filename = %%f 

        rem Run the Sha1 command
        C:\Users\IBM_ADMIN\Documents\Sha1.exe -r %xmlfilelocation%\%%f >> %sha1output%

        rem Waits _for the current Sha1 execution to be run.
        PING 1.1.1.1 -n 1 -w 1000 >NUL
    )
) else (
    echo No folder found
)

::read %THECSVFILE% and loop through each line
for /F "usebackq tokens=* delims= " %%A in (%sha1output%) do (
    set the_line=%%A
    call :process_line
)
goto TheEnd

:process_line
for /F "usebackq tokens=1,2,3,4,5 delims= " %%1 in ('%the_line%') do (
    set OUTPUTLINE=name=%%2 sha1=%%3 size=%%4 url=%xmlfilelocation%\%%2
    echo %OUTPUTLINE% >> %manifestfile%
)

rem del = %sha1output%

Before the string manipulation, the file is displayed correctly apart from the sha1=sha1: which should read just sha1= and the size=size: should read just size= which the manipulation is meant to solve.

name=New6.xml sha1=sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size=size:0 url=C:\Users\IBM_ADMIN\Documents\test\New6.xml 
name=New1.xml sha1=sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size=size:0 url=C:\Users\IBM_ADMIN\Documents\test\New1.xml 
name=New2.xml sha1=sha1:f10e2821bbbea527ea02200352313bc059445190 size=size:3 url=C:\Users\IBM_ADMIN\Documents\test\New2.xml 
name=New3.xml sha1=sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size=size:0 url=C:\Users\IBM_ADMIN\Documents\test\New3.xml 
name=New4.xml sha1=sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size=size:0 url=C:\Users\IBM_ADMIN\Documents\test\New4.xml 
name=New5.xml sha1=sha1:f10e2821bbbea527ea02200352313bc059445190 size=size:3 url=C:\Users\IBM_ADMIN\Documents\test\New5.xml 

When I add in the additional lines to manipulate the string:

    :process_line
for /F "usebackq tokens=1,2,3,4,5 delims= " %%1 in ('%the_line:,=~%') do (
    set sha1=%%3
    set size=%%4
    set url=%xmlfilelocation%\%%2
    set THISLINE=name=%%2 sha1=%sha1:~5% size=%size:~5% url=%url%
    rem echo The Line: %OUTPUTLINE%
    echo %OUTPUTLINE% >> %manifestfile%
)

Returns a file where the output name doesn't display properly. Normally the first or last file, the Sha1 value, for some reason is outputted as a directory. This only happens after I try and manipulate the string.

name=New6.xml sha1=f10e2821bbbea527ea02200352313bc059445190 size=3 url=C:\Users\IBM_ADMIN\Documents\test\New5.xml 
name=New1.xml sha1=ers\IBM_ADMIN\\sha1.exe size=0 url=C:\Users\IBM_ADMIN\Documents\test\New6.xml 
name=New2.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\IBM_ADMIN\Documents\test\New1.xml 
name=New3.xml sha1=f10e2821bbbea527ea02200352313bc059445190 size=3 url=C:\Users\IBM_ADMIN\Documents\test\New2.xml 
name=New4.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\IBM_ADMIN\Documents\test\New3.xml 
name=New5.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\IBM_ADMIN\Documents\test\New4.xml

An example of the Sha1 output, this is what the Sha1.exe outputs with no manipulation from me.

prefetch New1.xml sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size:0 http://EXAMPLEURL/REPLACEME.exe
prefetch New2.xml sha1:f10e2821bbbea527ea02200352313bc059445190 size:3 http://EXAMPLEURL/REPLACEME.exe    
prefetch New3.xml sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size:0 http://EXAMPLEURL/REPLACEME.exe    
prefetch New4.xml sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size:0 http://EXAMPLEURL/REPLACEME.exe    
prefetch New5.xml sha1:f10e2821bbbea527ea02200352313bc059445190 size:3 http://EXAMPLEURL/REPLACEME.exe    
prefetch New6.xml sha1:da39a3ee5e6b4b0d3255bfef95601890afd80709 size:0 http://EXAMPLEURL/REPLACEME.exe

What the final output should look like for the 6 files is:

name=New6.xml sha1=f10e2821bbbea527ea02200352313bc059445190 size=3 url=C:\Users\ADMIN\Documents\test\New5.xml 
    name=New1.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\ADMIN\Documents\test\New6.xml 
    name=New2.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\ADMIN\Documents\test\New1.xml 
    name=New3.xml sha1=f10e2821bbbea527ea02200352313bc059445190 size=3 url=C:\Users\ADMIN\Documents\test\New2.xml 
    name=New4.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\ADMIN\Documents\test\New3.xml 
    name=New5.xml sha1=da39a3ee5e6b4b0d3255bfef95601890afd80709 size=0 url=C:\Users\ADMIN\Documents\test\New4.xml

Answer 1

This does what you have shown:

This uses a helper batch file called repl.bat (by dbenham) - download from: https://www.dropbox.com/s/qidqwztmetbvklt/repl.bat

Place repl.bat in the same folder as the batch file or in a folder that is on the path.

The input file is file.txt and the output file is newfile.txt

@echo off
type "file.txt" | repl "prefetch (.*?) sha1:(.*?) size:(.).*" "name=$1 sha1=$2 size=$3 url=C:\Users\IBM_ADMIN\Documents\test\$1" >"newfile.txt"
pause