Correct use of Worker Service and registration of its services from Web Api (same solution)

Question

I am trying to find the right architecture to solve this problem: I need to build an API that serves a REST controller. This REST controller is simply accessing the values stored in a Dictionary and serves them to the user. What's in the Dictionary data structure? There is data which are the result of scraping an external api.

Solution 1: I build no worker service but only my API. My API when it starts will run a different thread asynchronously that keeps scraping the external API and saves the data in this Dictionary.

Solution 2 - The one I adopted cause I think better (correct me if I am wrong): I build a Worker Service that gets called when my API starts and whose role is to scrape the external API and save data in the Dictionary. Such data will be served by the controller in my API.

So, first of all I don't know if my choice is correct with respect to the first one. If it is... I am having troubles configuring the worker.

Program.cs of my API:

public class Program
{
    public static void Main(string[] args)
    {
        var webHost = CreateHostBuilder(args).Build();

        var configuration = new ConfigurationBuilder()
            .SetBasePath(Directory.GetCurrentDirectory())
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
            .AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Production"}.json", optional: true)
            .Build();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
}

Startup.cs of my API:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();

    services.AddHostedService<Worker>(); //here I register my worker service
}

Program.cs of my worker:

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostContext, services) =>
            {
                services.AddSingleton<IScraperService, ScraperService>();

                // Inject IHttpClientFactory
                services.AddHttpClient();
                services.AddHostedService<Worker>();
            });
}

My worker:

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;
    private readonly IScraperService _scraperService;

    public Worker(ILogger<Worker> logger, IScraperService scraperService)
    {
        _logger = logger;
        _scraperService = scraperService;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            _scraperService.ScrapeObjects();
        }
    }
}

so when my API starts will call the worker (or at least try to) but complaints. It says that my worker doesn't know anything about the registration of IScraperService. Of course not.. because I registered in the ConfigureServices of my Worker Service project, which is never called because I don't start it on its own.

Possible solution: is it correct to make the registration of IScraperService (services.AddSingleton();) at Startup of my API ? Please let me know if the whole is wrong. Thank you!

Answer 1

Have the two working together

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostContext, services) => {
                services.AddSingleton<IScraperService, ScraperService>();

                // Inject IHttpClientFactory
                services.AddHttpClient();
                services.AddHostedService<Worker>();
            })
            .ConfigureWebHostDefaults(webBuilder => {
                webBuilder.UseStartup<Startup>();
            });
}

Start up

public void ConfigureServices(IServiceCollection services) {
    services.AddControllers();    
    //services.AddHostedService<Worker>(); //worker already registered in Main

    //...
}

All services should now be known by the service container.

When the host is run, API will listen for requests and the worker will be started.

public class Worker : BackgroundService {
    private readonly ILogger<Worker> _logger;
    private readonly IScraperService _scraperService;

    public Worker(ILogger<Worker> logger, IScraperService scraperService) {
        _logger = logger;
        _scraperService = scraperService;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken) {
        while (!stoppingToken.IsCancellationRequested) {
            await Task.Run(() => _scraperService.ScrapeObjects());
        }
    }
}