feat: record desired state in Context

[metacosm]

Signed-off-by: Chris Laprun [email protected]

[csviri]

I think we should agree that on these optimization issues we should see the numbers, so we can decide if it is worth it.
First of all on large resources this will have a huge memory overhead.
So we need to see, how faster is it (if it is faster at all), and how much more memory we consume.

But as we I think discussed before, there should be no blocking operation in desired state computation, that goes agains the pattern, calculating the desired (thus creating a pojo) is extremely fast - might be faster than getting a resource from a hash map (we need to measure), we should think though what we are actually optimizing with this.

[metacosm]

Agreed but it might also be a good idea to look at the different cache layers which are becoming too complex to properly maintain.

[csviri]

Note that without this:
#2893
if the reconciler does not implement cleaner, there is no place where to clean those caches, since we don't receive the delete event now, when it should happen.

[metacosm]

Thinking about this some more, the memory usage should actually be minimal and there is no need to clean things because this isn't a new cache per se as the desired states are put in the Context object which doesn't survive past the reconciliation so these are short-lived versions of objects that are already computed so it shouldn't incur any additional memory than what is already used during the reconciliation.

[csviri]

Thinking about this some more, the memory usage should actually be minimal and there is no need to clean things because this isn't a new cache per se as the desired states are put in the Context object which doesn't survive past the reconciliation so these are short-lived versions of objects that are already computed so it shouldn't incur any additional memory than what is already used during the reconciliation.

if it does not survive the reconiliation, why to cache it?
Now it does no use the context by a HashMap

[metacosm]

Thinking about this some more, the memory usage should actually be minimal and there is no need to clean things because this isn't a new cache per se as the desired states are put in the Context object which doesn't survive past the reconciliation so these are short-lived versions of objects that are already computed so it shouldn't incur any additional memory than what is already used during the reconciliation.

if it does not survive the reconiliation, why to cache it?
To avoid recomputing a potentially costly and non-idempotent desired state, which is something that people have run into several times already.

[csviri]

To avoid recomputing a potentially costly and non-idempotent desired state, which is something that people have run into several times already.

In what use case is the desired state cumputation costy?
Could you pls link the issue here, so we have better clarity?
All that information should come from EventSource caches.

[csviri]

TBH I don't like the fact that we are calling the desired either - and by definition idempotent function. But I'm not convinced that this is the way to solving or should be solved in generic way, users are able to optimize that in some edge cases already. But this is not a strong opinion, so if we are able to solve this elegantly withing the Context I'm open for that.

also maybe there is something I just don't see, so happy to dicuss further. Would be nice to see the users specific issue.

[csviri]

This probably should be removed.