CODEWITHSUNDEEP
c#async-await
Lenny D
Lenny D

Reputation: 2044

Avoiding async/await if I know most of the time the result will be cached

Which of these is the best option when working with caching in C#?

I am interested at the compiler level which of these is the most elegant / performant solution.

E.g does the .net compiler use any tricks to know that when code will run synchronously and avoid creating/running unnecessary async await code?

Option 1, Use async/await and use Task.FromResult for cached values;

        public async Task<T> GetValue<T>(string key)
        {
            if (_cache.containsKey(key))
            {
                // 99% of the time will hit this
                return Task.FromResult(_cache.GetItem(key));
            }

            return  await _api.GetValue(key);

        }

Option 2, Avoid async/await and use something like GetAwaiter().GetResult() for the few times the API Endpoint will be hit.

        public  T GetValue<T>(string key)
        {
            if (_cache.containsKey(key))
            {
                // 99% of the time will hit this
                return _cache.GetItem(key);
            }

            return _api.GetValue(key).GetAwaiter().GetResult();

        }

Any insights would be very much appreciated.

Upvotes: 1

Views: 602

Answers (4)

Jon Hanna
Jon Hanna

Reputation: 113342

Your first won't work. The simplest, and the one to go for most of the time is:

public async Task<T> GetValueAsync<T>(string key)
{
  if (_cache.ContainsKey(key))
  {
    return _cache.GetItem(key);
  }

  T result = await _api.GetValueAysnc(key);
  _cache.Add(key, result);
  return result;
}

Or better still if possible:

public async Task<T> GetValueAsync<T>(string key)
{
  if (_cache.TryGet(key, out T result))
  {
    return result;
  }

  result = await _api.GetValueAysnc(key);
  _cache.Add(key, result);
  return result;
}

This works fine and will return an already-completed task when the value was in the cache, so awaiting it will continue immediately.

However if the value is in the cache much of the time and the method is called often enough for the extra apparatus around async to make a difference then you can avoid it entirely in such a case:

public Task<T> GetValueAsync<T>(string key)
{
  if (_cache.TryGet(key, out Task<T> result))
  {
    return result;
  }

  return GetAndCacheValueAsync(string key);
}

private async Task<T> GetAndCacheValueAsync<T>(string key)
{
  var task = _api.GetValueAysnc(key);
  result = await task;
  _cache.Add(key, task);
  return result;
}

Here if the value is cached we avoid both the state-machine around async and also the creation of a new Task<T> since we have stored an actual Task. Each of these are only done in the first case.

Upvotes: 4

canton7
canton7

Reputation: 42330

The official approach is to cache the Task<T>, and not the T.

This also has the advantage that if someone requests the value, you can kick off the request to fetch the value and then cache the resulting, in-progress Task<T>. If someone else requests the cached value before the request has completed, they're also given the same in-progress Task<T>, and you don't end up making two requests.

For example:

public Task<T> GetValue<T>(string key)
{
    // Prefer a TryGet pattern if you can, to halve the number of lookups
    if (_cache.containsKey(key))
    {
        return _cache.GetItem(key);
    }

    var task = _api.GetValue(key);
    _cache.Add(key, task);
    return task;
}

Note that you need to think about failure in this case: if the request to the API fails, then you'll be caching a Task which contains an exception. This might be what you want, but it might not be.

If for some reason you can't do this, then the official advice is to use ValueTask<T> for high-performance scenarios. This type has some gotchas (such as you can't await it twice), so I recommend reading this. If you don't have high performance requirements, Task.FromResult is fine.

Upvotes: 4

iury
iury

Reputation: 118

What you are looking for probably is memoization.

A implementation might be something like this:

public static Func<T, TResult> Memoize<T, TResult>(this Func<T, TResult> f)
{
    var cache = new ConcurrentDictionary<T, TResult>();
    return a => cache.GetOrAdd(a, f);
}

Measure(() => slowSquare(2));   // 00:00:00.1009680
Measure(() => slowSquare(2));   // 00:00:00.1006473
Measure(() => slowSquare(2));   // 00:00:00.1006373
var memoizedSlow = slowSquare.Memoize();
Measure(() => memoizedSlow(2)); // 00:00:00.1070149
Measure(() => memoizedSlow(2)); // 00:00:00.0005227
Measure(() => memoizedSlow(2)); // 00:00:00.0004159

Source

Upvotes: 1

Christopher
Christopher

Reputation: 9824

First of all, this calls for linking the speed rant:

https://ericlippert.com/2012/12/17/performance-rant/

Microoptimisations like these are usually left to the JiT. My rule of thumb is that if you really need that difference, then you are propably dealing with realtime programming. And for Realtime Proramming a Garbage Collected runtime like .NET was propably the wrong environment to begin with. Something with direct memory management like unsafe code - even native C++ or Assembler - would have been better.

Secondly, a task might just be the wrong tool here. Maybe what you actually want is something like Lazy[T]? Or any of the 5 different Chache classes? (as with timer, there is about one for specific User Interface technology).

It is possible to use any tool for many purposes. But tasks are for Multitasking and there are better tools for caching and lazy initialisation. And Lazy[T] is even inherently Thread save.

Upvotes: -2

Related Questions