.NET Core Worker Service as Windows service not stopping after calling StopAsync

Question

I have a worker service deployed as windows service and I am having problem stopping windows services, it shows stopped from the logs but when I logon to the server it still says running.

public class Worker : BackgroundService
{
  private readonly ILogger<Worker> _logger;
  

  public Worker(ILogger<Worker> logger)
  {
    this._logger = logger;
  }

  public override Task StartAsync(CancellationToken cancellationToken)
  {
    _logger.LogInformation("Windows Service is Starting....   ");
    return base.StartAsync(cancellationToken);
  }
  protected override async Task ExecuteAsync(CancellationToken stoppingToken)
  {
    //initial value coming as false
    _logger.LogInformation(string.Format("Cancellation token initial value : {0}", stoppingToken.IsCancellationRequested));

        while (!stoppingToken.IsCancellationRequested)
        {
           try
           {
              _logger.LogInformation("Starting the Process .....");
               
              //Dummy task used here, so it can cancel token.
              await Task.Delay(1000);       
         
              //Deliberately failing in the string.Format to test windows service stopping
              string.Format("Failing with exception here : {1}", "Test");
            }
           
            catch (Exception ex)
            {
              _logger.LogError(ex, ex.Message);
               var canclSrc = new CancellationTokenSource();
               canclSrc.Cancel();
               await StopAsync(canclSrc.Token);
                   
               //Cancellation token here is coming as true..
               _logger.LogInformation(string.Format("Cancellation token value : {0}", stoppingToken.IsCancellationRequested));
               _logger.LogInformation("Windows Service is stopped..");
             }
         }
    //Windows service failed and it is logging outside the while loop.
    _logger.LogInformation("Came out of while loop as windows service stopped!");
  }

 public override Task StopAsync(CancellationToken cancellationToken)
 {
   _logger.LogInformation("Stopping Windows Service...");
   return base.StopAsync(cancellationToken);
 }

It is giving cancellation token as true in the logs, successfully calling the StopAsync method.

When I login into the server and see the windows service it is showing running and the windows service got struck somewhere I don't see any logs anything the service just hungs..

Any suggestion on why my windows service (which is a worker service) is not stopping on the server even when stopAsync is invoked.

Answer 1

I'd stick with Microsoft's example using Environment.Exit(1):

namespace App.WindowsService;

public sealed class WindowsBackgroundService : BackgroundService
{
    private readonly JokeService _jokeService;
    private readonly ILogger<WindowsBackgroundService> _logger;

    public WindowsBackgroundService(
        JokeService jokeService,
        ILogger<WindowsBackgroundService> logger) =>
        (_jokeService, _logger) = (jokeService, logger);

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        try
        {
            while (!stoppingToken.IsCancellationRequested)
            {
                string joke = _jokeService.GetJoke();
                _logger.LogWarning("{Joke}", joke);

                await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
            }
        }
        catch (TaskCanceledException)
        {
            // When the stopping token is canceled, for example, a call made from services.msc,
            // we shouldn't exit with a non-zero exit code. In other words, this is expected...
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "{Message}", ex.Message);

            // Terminates this process and returns an exit code to the operating system.
            // This is required to avoid the 'BackgroundServiceExceptionBehavior', which
            // performs one of two scenarios:
            // 1. When set to "Ignore": will do nothing at all, errors cause zombie services.
            // 2. When set to "StopHost": will cleanly stop the host, and log errors.
            //
            // In order for the Windows Service Management system to leverage configured
            // recovery options, we need to terminate the process with a non-zero exit code.
            Environment.Exit(1);
        }
    }
}

If you're using your own cancellation tokens in the app to signal it's time to restart, then:

try
{
    // Implementation
}
catch (TaskCanceledException ex)
{
    // perhaps log debug / trace
}
catch (Exception ex)
{
    // log error
}
finally
{
    // Note that `stoppingToken.IsCancellationRequested` being true means the service was
    // manually stopped using services.msc, so returning a non-zero exit code is not desired.
    if (!stoppingToken.IsCancellationRequested)
        Environment.Exit(1);
}

Here's why:

When using _hostApplicationLifetime.StopApplication(); The windows service manager will not restart the service when configured to restart on error because it's not getting a non-zero exit code.

From my testing:

Environment.Exit(1) must be done before BackgroundService ExecuteAsync exits.

What doesn't work from my testing:

• returning non-zero int from Main.
• Environment.Exit(1) in Main after kestrel stops.
• Canceling a cancellation token passed to kestrel's RunAsync method.
• Setting Environment.ExitCode = 1 before ExecuteAsync returns.

I haven't looked at Microsoft's dotnet runtime source to see why the above is the case.

from https://blog.stephencleary.com/2020/06/servicebase-gotcha-recovery-actions.html:

public class MyBackgroundService : BackgroundService
{
  private readonly IHostLifetime _hostLifetime;
  public MyBackgroundService(IHostLifetime hostLifetime) =>
      _hostLifetime = hostLifetime;

  protected override Task ExecuteAsync(CancellationToken stoppingToken)
  {
    try
    {
      // Implementation
    }
    catch (Exception)
    {
      if (_hostLifetime is ServiceBase serviceLifetime)
        serviceLifetime.ExitCode = -1;
      else
        Environment.ExitCode = -1;
    }
  }
}

Setting serviceLifetime.ExitCode = -1 might work (I haven't tested this). However, as I stated before Environment.ExitCode = -1 doesn't work. Note serviceLifetime.ExitCode = -1 throws when not in Windows.

In addition, a problem with the above is kestrel keeps running when ExecuteAsync exits.

As pointed out from the same blog:

https://blog.stephencleary.com/2020/06/backgroundservice-gotcha-application-lifetime.html

public class MyBackgroundService : BackgroundService
{
    private readonly IHostApplicationLifetime _hostApplicationLifetime;
    public MyBackgroundService(IHostApplicationLifetime hostApplicationLifetime) =>
        _hostApplicationLifetime = hostApplicationLifetime;

    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        try
        {
            // Implementation
        }
        finally
        {
            _hostApplicationLifetime.StopApplication();
        }
    }
}

Which is why I prefer Environment.Exit(1) at least until I get away from using windows services.

Answer 2

await StopAsync(canclSrc.Token);

Your background service shouldn't stop itself.

If you want to tear down your host (i.e., the Win32 Service), then you should call IHostApplicationLifetime.StopApplication (as described on my blog). Something like this:

public class Worker : BackgroundService
{
  private readonly ILogger<Worker> _logger;
  private readonly IHostApplicationLifetime _hostApplicationLifetime;

  public Worker(ILogger<Worker> logger, IHostApplicationLifetime hostApplicationLifetime) => (_logger, _hostApplicationLifetime) = (logger, hostApplicationLifetime);

  protected override async Task ExecuteAsync(CancellationToken stoppingToken)
  {
    _logger.LogInformation("Windows Service is Starting....   ");

    _logger.LogInformation(string.Format("Cancellation token initial value : {0}", stoppingToken.IsCancellationRequested));

    try
    {
      while (!stoppingToken.IsCancellationRequested)
      {
        _logger.LogInformation("Starting the Process .....");
               
        await Task.Delay(1000);       
         
        //Deliberately failing in the string.Format to test windows service stopping
        string.Format("Failing with exception here : {1}", "Test");
      }
    }
    catch (Exception ex)
    {
      _logger.LogError(ex, ex.Message);
    }
    finally
    {
      _logger.LogInformation("Windows Service is stopped..");
      _hostApplicationLifetime.StopApplication();
    }
  }
}