Reputation: 9
I have a hosted service that executes a job/function periodically (a minute after execution). That job reads possibly millions of data and checks if needs to be modified by reading some data from the cache.
My question is that how to check those data in parallel and execute as fast as I can?
should I scale the hosted service horizontally? In that case how to make it fairly parted the data for each service instance?
or should I query the data and split it into two/three and check in thread-level parallelism? In that case, what would be the side effect in the future if the system load is increased?
or should I leave hosted services and do it in another way?
Any suggestion is more than welcome.
What I am doing right now is to query records by their types and check in parallel and I have one instance.
simple example of the hosted service
public async Task DoJob(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
var lossTask = WatchSlOrders(cancellationToken);
var profitTask = WatchTpOrders(cancellationToken);
Task.WaitAll(new Task[] { lossTask, profitTask }, cancellationToken: cancellationToken);
await Task.Delay(1000 * 60, cancellationToken);
}
}
public async Task WatchSlOrders(CancellationToken cancellationToken)
{
var scope = Services.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<DbContext>();
var orders = // Get SL Orders
foreach (var order in orders)
{
var symbolInfo = // Read data from cache
if (// Order does not need to be triggered) continue;
// Modify the order as triggered and send to message queue.
}
}
// Table Schema
public class ADOrder : IEntity
{
[Key]
public Guid Id { get; set; }
public Guid PositionId { get; set; }
public string Symbol { get; set; }
public OrderTriggerType TriggerType { get; set; } // Stop Loss, Take Profit
public decimal TriggerPrice { get; set; }
public OrderStatus Status { get; set; } // Open, Triggered, Filled, Rejected
}
Edit: I have a Redis cache that is fed by symbol info from various exchanges (BTCUSDT, ETHUSDT) and orders in the Postgres database (Binance, BTCUSDT, Stop Loss, 32.000, Open). The hosted service, query open orders and check if the order should be triggered (BTC current price is less than 32.00) modify the order state as triggered and send to message queue, and this record will not be queried in the next iteration (so, write latency is important otherwise record will be sent to message queue twice).
Upvotes: 0
Views: 234