.Net Core Background Services and Long Running Tasks

Question

I'm using .Net Core to run long-running background Services and long-running tasks in my ASP .Net Core in order to get Files generated by External running Process in Linux Ubuntu 18.04. Using FileSystemWatcher.

After 24 hours or more, things get worst. The FileSystemWatch works properly but the long-running threads and background service are not working well and sometimes they stop with no error.

Anyone has an idea about Background services or long-running tasks issues in .Net Core 3.1.

Should I Use Hangfire to avoid those issues?

Example : This Thread may Works up to 72 hours with no stop.

   public class DataByteCollector{

    public readonly BytesMemoryCache DataBytesMemoryCache;

    public DataByteCollector(BytesMemoryCache DataBytesMemoryCache)
    {
        DataBytesMemoryCache = DataBytesMemoryCache;
    }

    public BufferBlock<ByteData> ByteStream { get; set; }

    public async Task SubscribeToStream(string bytesId, CancellationToken cancellationToken)
    {
        // Avoid Capturing members in anonymous methods

        var DataBytesMemoryCache = DataBytesMemoryCache;
        ByteStream = new BufferBlock<ByteData>(
            new DataflowBlockOptions
            {
                BoundedCapacity = 8,
                EnsureOrdered = true,
                CancellationToken = cancellationToken
            });
        var byteStream = ByteStream;

        var streamDataCache = DataBytesMemoryCache.GetBytesCache(bytesId);
        if (streamDataCache == null)
        {
            ByteStream.Complete();
            return;
        }

        var bytesData = streamDataCache.Streams;

        async Task DataGathring()
        {
            try
            {
                DateTime? lastDataRead = null;

                var hasStarted = true;
                while (!cancellationToken.IsCancellationRequested && DataBytesMemoryCache.DataCacheExists(bytesId))
                {
                    IEnumerable<ByteData> stat;
                    if (hasStarted)
                    {
                        stat = bytesData.ToArray();
                        hasStarted = false;
                    }
                    else
                    {
                        stat = bytesData.TakeLast(1).Where(x => x.DateTime > lastDataRead).ToArray();
                    }

                    if (stat.Any())
                    {
                        foreach (var farge in stat)
                        {
                            lastDataRead = DateTime.Now;
                            await Channel.SendAsync(farge, cancellationToken);
                        }
                    }
                    else
                    {
                        await byteStream.SendAsync(new ByteData(new byte[100], "dummy"), cancellationToken);
                    }

                 
                    if (lastDataRead == null)
                    {
                        break;
                    }
                    if (lastDataRead < DateTime.Now.AddSeconds(-30))
                    {
                        break;
                    }

                    await Task.Delay(5000, cancellationToken);
                  
                }
                cancellationToken.ThrowIfCancellationRequested();
                byteStream.Complete();
                await byteStream.Completion;
            }
            catch (Exception e)
            {

                byteStream.Complete();
                await byteStream.Completion;
            }
        }

        await Task.Factory.StartNew(DataGathring, cancellationToken);
    }
}

Answer 1

Why not use an IHostedService for this?

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.1&tabs=visual-studio