Introduce proper compiler-wise caches #17790
Description
Right now, we mostly use dictionaries (and concurrent dictionaries) for caching.
We need to introduce one compiler-wise cache type(s) with different strategies, and plugged into our activity reporting (reporting metrics, exposed to dotnet-collect and otel), report hits, misses, key collisions, etc.
Implementation is up to whoever is going to do it. MemoryCache is untyped, and isn't designed for storing one type of objects in cache. We might need to implement it ourselves.