Launching several long running background services at the same time

Question

I am experimenting with IHostedService in a dotnet core 2.2. My task is to create 2 background long-running tasks.

• The first of them is to manage Selenium browser session (open/close tabs, parse DOM) and put email messages in a queue inside ConcurrentBag.
• Second background worker is sending out email notifications once in 10 minutes, for the messages that exist in ConcurrentBag (which first task added). It also groups them together so that only 1 message is sent.

However, I am having troubles running 2 of the hosted processes at the same time. It seems like only first hosted process if being executed, while the second process awaits for the first to be fully executed. But since I never expect it to be finished - second process never starts...

Am I misusing the IHostedService? If yes, then what would be the best architectural approach of accomplishing my task?

Here is the code that I am currently using (trying to finish):

using System;
// ..

namespace WebPageMonitor
{
    class Program
    {
        public static ConcurrentBag<string> Messages = new ConcurrentBag<string>();

        static void Main(string[] args)
        {
            BuildWebHost(args)
                .Run();

            Console.ReadKey();
        }

        private static IHost BuildWebHost(string[] args)
        {
            var hostBuilder = new HostBuilder()
                .ConfigureHostConfiguration(config =>
                {
                    config.AddJsonFile("emailSettings.json", optional: true);
                    config.AddEnvironmentVariables();
                })
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddOptions();

                    var bindConfig = new EmailSettings();
                    hostContext.Configuration.GetSection("EmailSettings").Bind(bindConfig);
                    services.AddSingleton<EmailSettings>(bindConfig);

                    services.AddTransient<EmailSender>();

                    services.AddHostedService<BrowserWorkerHostedService>();
                    services.AddHostedService<EmailWorkerHostedService>();
                });

            return hostBuilder.Build();
        }

    }
}

BrowserWorkerHostedService

public class BrowserWorkerHostedService : BackgroundService
{
    private static IWebDriver _driver;

    public BrowserWorkerHostedService()
    {
        InitializeDriver();
    }

    private void InitializeDriver()
    {
        try
        {
            ChromeOptions options = new ChromeOptions();
            options.AddArgument("start-maximized");
            options.AddArgument("--disable-infobars");
            options.AddArgument("no-sandbox");

            _driver = new ChromeDriver(options);
        }
        catch (Exception ex)
        {
            Program.Messages.Add("Exception: " + ex.ToString());

            Console.WriteLine($" Exception:{ex.ToString()}");
            throw ex;
        }
    }

    protected override async Task ExecuteAsync(CancellationToken stopToken)
    {
        while (!stopToken.IsCancellationRequested)
        {
            try
            {
                _driver.Navigate().GoToUrl("https://www.google.com");
                Program.Messages.Add("Successfully opened a website!");
                // rest of the processing here

                Thread.Sleep(60_000);
            }
            catch (Exception ex)
            {
                Program.Messages.Add("Exception: " + ex.ToString());

                Console.WriteLine(ex.ToString());
                Thread.Sleep(120_000);
            }
        }

        _driver?.Quit();
        _driver?.Dispose();
    }
}

EmailWorkerHostedService

public class EmailWorkerHostedService : BackgroundService
{
    private readonly EmailSender _emailSender;
    private readonly IHostingEnvironment _env;

    public EmailWorkerHostedService(
        EmailSender emailSender,
        IHostingEnvironment env)
    {
        _emailSender = emailSender;
        _env = env;
    }

    protected override async Task ExecuteAsync(CancellationToken stopToken)
    {
        while (!stopToken.IsCancellationRequested)
        {
            var builder = new StringBuilder();

            List<string> exceptionMessages = new List<string>();
            string exceptionMessage;
            while (Program.Messages.TryTake(out exceptionMessage))
                exceptionMessages.Add(exceptionMessage);

            if (exceptionMessages.Any())
            {
                foreach (var message in exceptionMessages)
                {
                    builder.AppendLine(new string(message.Take(200).ToArray()));
                    builder.AppendLine();
                }

                string messageToSend = builder.ToString();
                await _emailSender.SendEmailAsync(messageToSend);
            }

            Thread.Sleep(10000);
        }
    }
}

EDIT: After applying changes suggested in answer, here is the current version of the code that works. Adding await helped.

Answer 1

First DO NEVER use Thread.Sleep() in async context as it's blocking action. Use Task.Delay() instead. And I believe here is your problem. Look at BackgroundService.StartAsync implementation:

    public virtual Task StartAsync(CancellationToken cancellationToken)
    {
        // Store the task we're executing
        _executingTask = ExecuteAsync(_stoppingCts.Token);

        // If the task is completed then return it, this will bubble cancellation and failure to the caller
        if (_executingTask.IsCompleted)
        {
            return _executingTask;
        }

        // Otherwise it's running
        return Task.CompletedTask;
    }

When asyc method called actually it performed synchronously until first true async operation. Your true async operation is

await _emailSender.SendEmailAsync(messageToSend);

but it will be called only when meeting the condition

if (exceptionMessages.Any())

It means your ExecuteAsync method will never return and so StartAsync. Task.Delay is also true async method (Thread.Sleep is not), hence after hitting it the StartAsync will continue and exit and your second service will have the chance to start.