Is current CVI fixable with minimum effort?

[albertpod]

Hello team,

I want to highlight an ongoing issue regarding the suboptimal and inaccurate implementation of CVI within RxInfer ecosystem. This component is crucial for executing message passing when the delta function receives distributions from the exponential family, not just Gaussian ones.

Originally, we considered rewriting it in ExpectationsApproximation.jl and then integrating this into RxInfer.jl. There are issues opened on ExpectationsApproximation.jl. However, this approach seems less than ideal, especially given our roadmap's timeline. According to the roadmap, we aim to introduce Automated inference with ExponentialFamily.jl within the first two quarters. With Q1 nearly complete, fully integrating this package into the RxInfer ecosystem by Q2 appears to be a stretch.

Given our deeper understanding of the issue, thanks to @ismailsenoz, @bvdmitri and @Nimrais, I wonder if a temporary fix is feasible. Alternatively, could we implement another method to provide a similar solution without relying on the current CVI version in RxInfer.jl? My concern is that in its present form, CVI shouldn't be in RxInfer.jl, although this is more an feeling than a definitive stance. I just don't like broken stuff that is supposed to solve many problems of newcomers, but can't deliver that.

I'm keen to hear your thoughts on the matter, especially from @ismailsenoz, @Nimrais, @ThijsvdLaar, @bvdmitri, @wouterwln, @bartvanerp, @HoangMHNguyen, and @bertdv. Do you think a quick fix is possible, or should we consider full integration only after completing GraphPPL.jl 4.0? The upcoming changes in ReactiveMP.jl suggest that waiting might lead to a smoother integration process.

Everyone's opinions are important in deciding the next steps, including potentially redistributing tasks to speed up the process.

Looking forward to your feedback.

Cheers!

[bvdmitri]

I can guess what do you mean, but for the sake of the public open discussion can we summarize what exactly is wrong with the current CVI and what do we want as the team to fix. Currently it reads like "we have an issue can you fix it please"? Its not a productive discussion. Can we at least highlight some bullets of what is wrong and what is not and why it is labeled as urgent? I can start as an example, but better if @ismailsenoz or @Nimrais do that as they have a better picture.

• Does CVI support all members of exponential family from ExponentialFamily.jl. Yes. (with a very few exceptions)
• Does CVI return some estimates for all members of exponential family ExponentialFamily. Yes.
• Is CVI robust enough and error-free? No
• Is CVI efficient enough? Maybe, depends

These are just guesses, but it would not be possible to fix anything without a proper understanding of what exactly are we talking about when saying that something needs to be (quick/temporarily) fixed.

Alternatively, could we implement another method to provide a similar solution without relying on the current CVI version in RxInfer.jl?

I would rather ask do we have another method? CVI has problems, that is true, but some of the problems are beyond its implementation. Implementing a different method does not necessarily solves all the problems. For example we still don't have any method to run inference around arbitrary stochastic nodes, fixing CVI or implementing a new method to replace it won't really help newcomers on that matter.

[albertpod]

I agree with you about this thread. Let's see how it goes. I've tagged few people, hopefully they jump in to provide theirs opinion.