Download only today's files from FTP server using Windows batch file

Question

I am trying to copy XML files from an FTP location to an application server - but not all files. New files are archived after every half an hour in the ftp location. I need to transfer only the new files based on the timestamp and date.

I am currently using the below 2 files to copy all the files from FTP location.

Batch file:

ftp -i -s:D:\ftp_commands.txt -n <host name>

Text file (ftp_commands.txt):

user <username> <password>
cd <source path>
lcd <destination path>
mget *
bye

Can anyone please help me copy files based on timestamp or date?

Answer 1

It's a pretty complex task to implement with Windows batch-file and the built-in FTP client (ftp.exe).

It would be more easier with PowerShell or other more powerful language.

---

And even easier using a more capable FTP client.

For example WinSCP FTP client supports time constraints.

With WinSCP, a batch file to download today's files is as trivial as:

winscp.com /ini=nul /log=todays.log /command ^
    "open ftp://username:password@ftp.example.com/" ^
    "get /remote/path/*>=%%TIMESTAMP#yyyy-mm-dd%% C:\local\path\" ^
    "exit"

With WinSCP 5.13 and newer, the syntax is even easier:

    "get /remote/path/*>=today C:\local\path\" ^

This uses the %TIMESTAMP% syntax and a file mask with a time-constraint.

See also:

• guide for Converting Windows FTP script to WinSCP script;
• Downloading the most recent file with WinSCP.

(I'm the author of WinSCP)

Answer 2

As user Martin Prikryl pointed out in a comment, it is possible to use Windows' native FTP command ftp.exe to accomplish your goal, although it might not be that easy. So I had to take the challenge…

Here is a script that downloads a given number of the most recent (newest) files. This could be modified to regard the timestamp rather than a count, but since the date/time format may depend on the FTP host and/or the local machine, and I do not know what format you are receiving when executing the FTP command dir, I decided to go for the count at the first place. So here it is:

@echo off
setlocal EnableExtensions DisableDelayedexpansion

rem // Define constants here:
set "_FTP_LIST=%TEMP%\ftp_list_%RANDOM%.txt" & rem // (FTP script for listing files)
set "_FTP_RECV=%TEMP%\ftp_recv_%RANDOM%.txt" & rem // (FTP script for getting files)
set "_FTP_LTMP=%TEMP%\ftp_list_%RANDOM%.tmp" & rem // (file to store remote file list)
set "_FTP_HOST=<host name>"        & rem // (name of FTP host)
set "_FTP_USER=<username>"         & rem // (user name to login to the FTP host)
set "_FTP_PASS=<password>"         & rem // (pass word to login to the FTP host)
set "_FTP_RSRC=<source path>"      & rem // (path to remote source location)
set "_FTP_LDST=<destination path>" & rem // (path to local destination location)
set "_REVERSE="   & rem // (set to any value to get the oldest not the newest files)
set /A "_COUNT=1" & rem // (number of most recent or newest remote files to receive)

rem // Check if revert flag is set, force sort order to be in decreasing age in case:
if defined _REVERSE (set "REV=r") else (set "REV=")

rem // Build FTP script for listing remote files sorted by age:
> "%_FTP_LIST%" (
    rem // Check whether use name is given:
    if defined _FTP_USER (
        rem // Avoid auto-login:
        set "SWITCH=-n"
        rem // Write command to login:
        setlocal EnableDelayedExpansion
        echo user "!_FTP_USER!" !_FTP_PASS!
        endlocal
    ) else (
        rem // Attempt to login anonymously:
        set "SWITCH=-A"
    )
    rem // Write command to change to the desired remote location:
    echo cd "%_FTP_RSRC%"
    rem // Write command to list remote files sorted by increasing age:
    echo ls -t%REV% "%_FTP_LTMP%"
    rem // Write command to leave the FTP host:
    echo bye
)

rem /* Execute FTP script to list remote files sorted by age and write result to a
rem    temporary file, which is going to be read and parsed later: */
ftp -i -v %SWITCH% -s:"%_FTP_LIST%" "%_FTP_HOST%"

rem // Build FTP script for downloading the newest remote files:
> "%_FTP_RECV%" (
    rem // Check whether use name is given:
    if defined _FTP_USER (
        rem // Write command to login:
        setlocal EnableDelayedExpansion
        echo user "!_FTP_USER!" !_FTP_PASS!
        endlocal
    )
    rem // Write command to change to the desired remote location:
    echo cd "%_FTP_RSRC%"
    rem // Write command to change to the desired local location:
    echo lcd "%_FTP_LDST%"
    rem // Reset index used to extract the listed remote files:
    set /A "INDEX=0"
    rem /* Read the temporary file containing the list of remote files sorted by age,
    rem    loop through them and dynamically build the download commands: */
    for /F usebackq^ delims^=^ eol^= %%L in ("%_FTP_LTMP%") do (
        rem // Increment index:
        for /F %%K in ('set /A "INDEX+1"') do (
            set /A "INDEX=%%K"
            rem /* Check whether index already reached given count of remote files and
            rem    if not, write command to download a single file: */
            if %%K LEQ %_COUNT% echo get "%%L"
        )
    )
    rem // Write command to leave the FTP host:
    echo bye
)

rem // Ensure that the local destination directory exists:
md "%_FTP_LDST%" 2> nul

rem // Execute FTP script to download the newest remote files:
ftp -i -v %SWITCH% -s:"%_FTP_RECV%" "%_FTP_HOST%"

rem // Clean up temporary files:
del "%_FTP_LIST%" "%_FTP_RECV%" "%_FTP_LTMP%"

endlocal
exit /B