Monitor FTP directory in ASP.NET/C#

Question

I have FileSystem watcher for a local directory. It's working fine. I want same to implement for FTP. Is there any way I can achieve it? I have checked many solutions but it's not clear.

Logic: Want to get files from FTP later than some timestamp.
Problem faced: Getting all files from FTP and then filtering the result is hitting the performance (used FtpWebRequest).

Is there any right way to do this? (WinSCP is on hold. Cant use it now.)

FileSystemWatcher oFsWatcher = new FileSystemWatcher();
OFSWatchers.Add(oFsWatcher);
oFsWatcher.Path = sFilePath;

oFsWatcher.Filter = string.IsNullOrWhiteSpace(sFileFilter) ? "*.*" : sFileFilter;
oFsWatcher.NotifyFilter = NotifyFilters.FileName;

oFsWatcher.EnableRaisingEvents = true;
oFsWatcher.IncludeSubdirectories = bIncludeSubdirectories;
oFsWatcher.Created += new FileSystemEventHandler(OFsWatcher_Created);

Answer 1

I have got an alternative solution to do my functionality.

Explanation:

I am downloading the files from FTP (Read permission reqd.) with same folder structure.

So everytime the job/service runs I can check into the physical path same file(Full Path) exists or not If not exists then it can be consider as a new file. And Ii can do some action for the same and download as well.

Its just an alternative solution.

Code Changes:

 private static void GetFiles()
 {
    using (FtpClient conn = new FtpClient())
    {
        string ftpPath = "ftp://myftp/";

        string downloadFileName = @"C:\temp\FTPTest\";

        downloadFileName +=  "\\";

        conn.Host = ftpPath;
        //conn.Credentials = new NetworkCredential("ftptest", "ftptest");
        conn.Connect();

        //Get all directories

        foreach (FtpListItem item in conn.GetListing(conn.GetWorkingDirectory(),
            FtpListOption.Modify | FtpListOption.Recursive))
        {
            // if this is a file
            if (item.Type == FtpFileSystemObjectType.File)
            {
                string localFilePath = downloadFileName + item.FullName;

                //Only newly created files will be downloaded.
                if (!File.Exists(localFilePath))
                {
                    conn.DownloadFile(localFilePath, item.FullName);
                    //Do any action here.
                    Console.WriteLine(item.FullName);
                }
            }
        }
    }
}

Answer 2

You cannot use the FileSystemWatcher or any other way, because the FTP protocol does not have any API to notify a client about changes in the remote directory.

All you can do is to periodically iterate the remote tree and find changes.

It's actually rather easy to implement, if you use an FTP client library that supports recursive listing of a remote tree. Unfortunately, the built-in .NET FTP client, the FtpWebRequest does not. But for example with WinSCP .NET assembly, you can use the Session.EnumerateRemoteFiles method.

See the article Watching for changes in SFTP/FTP server:

// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
    Protocol = Protocol.Ftp,
    HostName = "example.com",
    UserName = "user",
    Password = "password",
};

using (Session session = new Session())
{
    // Connect
    session.Open(sessionOptions);

    List<string> prevFiles = null;

    while (true)
    {
        // Collect file list
        List<string> files =
            session.EnumerateRemoteFiles(
                "/remote/path", "*.*", EnumerationOptions.AllDirectories)
            .Select(fileInfo => fileInfo.FullName)
            .ToList();
        if (prevFiles == null)
        {
            // In the first round, just print number of files found
            Console.WriteLine("Found {0} files", files.Count);
        }
        else
        {
            // Then look for differences against the previous list
            IEnumerable<string> added = files.Except(prevFiles);
            if (added.Any())
            {
                Console.WriteLine("Added files:");
                foreach (string path in added)
                {
                    Console.WriteLine(path);
                }
            }

            IEnumerable<string> removed = prevFiles.Except(files);
            if (removed.Any())
            {
                Console.WriteLine("Removed files:");
                foreach (string path in removed)
                {
                    Console.WriteLine(path);
                }
            }
        }

        prevFiles = files;

        Console.WriteLine("Sleeping 10s...");
        Thread.Sleep(10000);
    }
}

(I'm the author of WinSCP)

---

Though, if you actually want to just download the changes, it's a way easier. Just use the Session.SynchronizeDirectories in the loop.

while (true)
{
    SynchronizationResult result =
        session.SynchronizeDirectories(
            SynchronizationMode.Local, "/remote/path", @"C:\local\path", true);
    result.Check();
    // You can inspect result.Downloads for a list for updated files

    Console.WriteLine("Sleeping 10s...");
    Thread.Sleep(10000);
}

This will update even modified files, not only new files.

---

Though using WinSCP .NET assembly from a web application might be problematic. If you do not want to use a 3rd party library, you have to do with limitations of the FtpWebRequest. For an example how to recursively list a remote directory tree with the FtpWebRequest, see my answer to List names of files in FTP directory and its subdirectories.

---

You have edited your question to say that you have performance problems with the solutions I've suggested. Though you have already asked a new question that covers this:
Get FTP file details based on datetime in C#

Answer 3

Unless you have access to the OS which hosts the service; it will be a bit harder.

FileSystemWatcher places a hook on the filesystem, which will notify your application as soon as something happened.

FTP command specifications does not have such a hook. Besides that it's always initiated by the client.

Therefor, to implement such logic you should periodical perform a NLST to list the FTP-directory contents and track the changes (or hashes, perhaps (MDTM)) yourself.

More info:

• FTP return codes
• FTP