fix: fixes race conditions in EntitlementPolicyCache

[imdominicreed]

Proposed Changes

ristretto is eventually consistent.. causing a race condition on the first request, updates cache, fetches from cache, nothings there. using a cache here doesn't seem right since we are pulling these in anyways. if the policy is too big that it gets evicted from the cache then what is the cache doing.

This PR removes the ristretto cache, and just stores the entitlement policy. also, sync is added on the read write operations.

Checklist

• I have added or updated unit tests
• I have added or updated integration tests (if appropriate)
• I have added or updated documentation

Testing Instructions

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	177.61626ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	106.590182ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	359.229053ms
Throughput	278.37 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	37.418751385s
Average Latency	372.582364ms
Throughput	133.62 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.865470237s
Average Latency	257.692926ms
Throughput	193.31 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	185.004495ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	117.937769ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	354.68495ms
Throughput	281.94 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	38.671017518s
Average Latency	384.322092ms
Throughput	129.30 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	26.980913838s
Average Latency	268.829582ms
Throughput	185.32 requests/second

[github-actions]

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	172.507366ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	92.284208ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	341.922144ms
Throughput	292.46 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.427441212s
Average Latency	362.458897ms
Throughput	137.26 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.382811927s
Average Latency	252.855485ms
Throughput	196.98 requests/second

[jrschumacher]

@imdominicreed this feels pretty aggressive. By shifting the cache to in memory managed by policy it removes the centralized control which will have long term impact on admins from managing cache behavior across the platform.

It is true that ristretto is eventually consistent so the first few hits might be fetching from the DB until the cache is available, but this is expected. If you're encountering some unexpected behavior I'd prefer to address the defect rather than tossing the baby with the bathwater.

I think its important to consider the eviction case, but I don't think its realistic that the policy would be too large at this point unless the admin uses an extremely small cache size.

One thing we're trying to set ourselves up for is support for other cache solutions such as distributive cache via Redis. The cache manager uses https://github.com/eko/gocache so that it will be possible to give admins the option to choose the cache layer they want.