HybridCache: Unit Testing Support

[sdepouw]

Currently, writing tests with the HybridCache abstraction is a little clunky. Essentially, you have to mock out the more complex GetOrCreateAsync() method. While this isn't hard to write a simple set of extensions to do so (which I'll include below), having something akin to what TimeProvider has, Microsoft.Extensions.TimeProvider.Testing that provides a FakeTimeProvider, would be most welcome.

I don't foresee a hypothetical FakeHybridCache having to do much, as the only contracted method is GetOrCreateAsync(). There's no need to add support for building out fake memory / distributed caches to control cache hits/misses. Rather, having a simple way to control what gets returned when called with a specific key (or tags perhaps too) would be all that's required.

Here's a little something I whipped up, using NSubstitute, to support both "I want to mock some fake return data from my cache" and "I want to assert my cache was called / used." Obviously this is quick and dirty and I just pass null or "any argument allowed" for the majority of things, but you can see where something like this would be beneficial in an official capacity. (Not strictly for NSubstitute, mind; methods that simply let us set up fake data or test that calls happened (a simple counter) would be all that's needed.)

using Microsoft.Extensions.Caching.Hybrid;
using NSubstitute;
using NSubstitute.Core;

namespace Example;

/// <summary>
/// Extension methods to make testing <see cref="HybridCache" /> easier in NSubstitute
/// </summary>
public static class NSubstituteHybridCacheExtensions
{
  public static ConfiguredCall SetupGetOrCreateAsync(this HybridCache mockCache, string key, string expectedValue)
  {
    return mockCache.GetOrCreateAsync(
      key,
      Arg.Any<object>(),
      Arg.Any<Func<object, CancellationToken, ValueTask<string>>>(),
      Arg.Any<HybridCacheEntryOptions?>(), 
      Arg.Any<IEnumerable<string>?>(), 
      Arg.Any<CancellationToken>()
    ).Returns(expectedValue);
  }
  
  public static async Task AssertGetOrCreateAsyncCalledAsync(this HybridCache mockCache, string key, int requiredNumberOfCalls)
  {
    await mockCache.Received(requiredNumberOfCalls).GetOrCreateAsync(
      key,
      Arg.Any<object>(),
      Arg.Any<Func<object, CancellationToken, ValueTask<string>>>(),
      null,
      null,
      Arg.Any<CancellationToken>()
    );
  }
}

And my unit test code has something like this:

using FluentAssertions;
using Microsoft.Extensions.Caching.Hybrid;
using NSubstitute;
using XUnit;

namespace UnitTesting;

public class MyTests
{
  private readonly HybridCache _mockCache = Substitute.For<HybridCache>();

  [Fact]
  public async Task Something()
  {
    // Mocking a return value
    _mockCache.SetupGetOrCreateAsync("some-key", expectedValue);

    // Asserting the cache was called
    await _mockCache.AssertGetOrCreateAsyncCalledAsync("some-key", 1);
  }
}

There's tons to improve here, but here's a super basic not-fully-implemented FakeHybridCache that could be a jumping off point.

using Microsoft.Extensions.Caching.Hybrid;

namespace Microsoft.Extensions.Caching.Hybrid.Testing;

public class FakeHybridCache : HybridCache
{
  // Not sure if this should be static, nor do I know what to do about Tags
  // Could make public since it's a fake cache anyway, and that could be useful for testing purposes
  private readonly Dictionary<string, object?> _cache = new();
  
  public override async ValueTask<T> GetOrCreateAsync<TState, T>(string key, TState state, Func<TState, CancellationToken, ValueTask<T>> factory, HybridCacheEntryOptions? options = null,
    IEnumerable<string>? tags = null, CancellationToken cancellationToken = default)
  {
    bool cached = _cache.TryGetValue(key, out object? value);
    if (cached) return (T?)value!;
    _cache.Add(key, await factory(state, cancellationToken));
    return (T?)_cache[key]!;
  }

  public override ValueTask SetAsync<T>(string key, T value, HybridCacheEntryOptions? options = null, IEnumerable<string>? tags = null,
    CancellationToken cancellationToken = default)
  {
    _cache[key] = value;
    return ValueTask.CompletedTask;
  }

  public override ValueTask RemoveAsync(string key, CancellationToken cancellationToken = default)
  {
    _cache.Remove(key);
    return ValueTask.CompletedTask;
  }

  public override ValueTask RemoveByTagAsync(string tag, CancellationToken cancellationToken = default)
  {
    throw new NotImplementedException();
  }
}

[tyler-boyd]

Related, I think it would be nice if there was some sort of NopCache provided which can be used in tests where some services require a HybridCache but you don't actually want any caching to happen.

[mgravell]

Rather, having a simple way to control what gets returned when called with a specific key (or tags perhaps too) would be all that's required.

That's ... not that simple. I can imagine a home-brew implementation something like:

    sealed class MockHybridCache : HybridCache
    {
        private readonly ConcurrentDictionary<string, object> _values = new();

        public object? this[string key]
        {
            get => _values.TryGetValue(key, out var val) ? val : null;
            set => _values[key] = value;
        }

        public override ValueTask<T> GetOrCreateAsync<TState, T>(string key, TState state, Func<TState, CancellationToken, ValueTask<T>> factory,
            HybridCacheEntryOptions? options = null, IEnumerable<string>? tags = null, CancellationToken cancellationToken = default)
            => _values.TryGetValue(key, out var value) ? new((T)value) : factory(state, cancellationToken);

        public override ValueTask RemoveAsync(string key, CancellationToken cancellationToken = default) { _values.TryRemove(key); return default; }
        public override ValueTask RemoveByTagAsync(string tag, CancellationToken cancellationToken = default) => default;
        public override ValueTask SetAsync<T>(string key, T value, HybridCacheEntryOptions? options = null, IEnumerable<string>? tags = null,
            CancellationToken cancellationToken = default) => default;
    }

But I suspect the requirements on a per-scenario basis would be more bespoke and unique. If anything, I think we'd need to actually try and gather some requirements, i.e. how is this meant to be used? what features does it need to support? I'd be disinclined to simply rush in and throw together random ideas, without exploring it first. So: what do we need here?

Related, I think it would be nice if there was some sort of NopCache provided which can be used in tests where some services require a HybridCache but you don't actually want any caching to happen.

That's pretty easy to implement if needed:

    sealed class FakeHybridCache : HybridCache
    {
        public override ValueTask<T> GetOrCreateAsync<TState, T>(string key, TState state, Func<TState, CancellationToken, ValueTask<T>> factory,
            HybridCacheEntryOptions? options = null, IEnumerable<string>? tags = null, CancellationToken cancellationToken = default)
            => factory(state, cancellationToken);

        public override ValueTask RemoveAsync(string key, CancellationToken cancellationToken = default) => default;
        public override ValueTask RemoveByTagAsync(string tag, CancellationToken cancellationToken = default) => default;
        public override ValueTask SetAsync<T>(string key, T value, HybridCacheEntryOptions? options = null, IEnumerable<string>? tags = null,
            CancellationToken cancellationToken = default) => default;
    }

I'm ... not sure how useful that would be to add, and I certainly wouldn't want to make that the base implementation, as that's a foot-shotgun waiting to happen.

[sdepouw]

My use cases are:

• In a unit test, be able to tell the abstraction to return something when I give the abstraction a key. Essentially, let me tell the abstraction's GetOrCreateAsync what to return, when I give it a string key.
• In a unit test, be able to count the number of calls to GetOrCreateAsync (even something as simple as "I was called with these keys: ["foo", "foo", "bar"] would be sufficient). Note this does not have to communicate cache hits vs misses. This is simply "assert cache was called N time(s)."

The first case is more important than anything, so that when I'm using the abstraction I can stub in what should be returned without having to do anything more complex than _fakeCache.KeyReturns<T>(string key, T returnValue).

[mgravell]

Actually, for preloading data: SetAsync is our friend, whether using L1 or L1+L2; - is SetAsync sufficient?

Does that just leave the "what was accessed"? We do have some inbuilt metrics, but these are currently using the usual metric APIs - so we don't retain much data that we can access directly. Since we defer L1 to IMemoryCache, we also don't have ready access to all the values we're caching, which makes things a little awkward. We'd need to either add explicit counting, or some better way of wrapping/subclassing. Maybe the real thing here is "make the API accessible" and if someone wants to subclass it and log more, fine. Thoughts? We could store a counter in CacheItem, and check L1 for MemoryCache, iterate the keys, fetch them all iteratively, check whether they're CacheItem, and fetch the counts.... but... #awkward

[sdepouw]

Implementing an official FakeHybridCache would be ideal. One that does not inherit or try to use the real implementations. Ideally, because it's an abstraction, we should not be touching the real implementation in any way, shape, or form. (In a unit test, mocking a dependency doesn't mean shortcutting the internals of the real deal. It means the real deal is never touched.) Because if the real HybridCache implementation changes, it could break unrelated unit tests, which is bad medicine.

Start with the abstraction, and implement a new, fake, implementation of HybridCache. It's the approach I took in my initial sample above. I feel like I could adopt it myself at this point, but the FakeTimeProvider over in Microsoft.Extensions.TimeProvider.Testing has spoiled me in terms of expectations. Note how it doesn't reach into the actual time provider code, but just implements its own version of the TimeProvider abstract base class, providing any helper methods needed.

A hypothetical FakeHybridCache would go the same route. It can start small too. Just beginning with the first case ("a way to directly set cache items, so that if you Set<T>("foo", myT) the fake GetOrCreateAsync<T>("foo") will echo it back out), and incrementally adding stuff as needed. No need to attempt to subclass the real deal, to attempt to negotiate L1/L2 or any of the real guts.

If this proves too much for dotnet to handle, I can run my own version of course. It's just always nice to see official support written by folks who understand the abstraction better than a newcomer (like me!) might.