-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved Integrators #300
Improved Integrators #300
Conversation
…compiled backends
… of some non-MSE losses sometimes
That already looks really good! Which integrator do you think should we use as a default or should I run some experiments? |
@stefanradev93 In my tests, RK45 runs in slightly fewer function evaluations with dynamic step size on. For a static step size, I would still choose Euler, since Flow Matching tends to have straight trajectories. But we can leave RK45 as the default, I think, or even implement more algorithms. That said, there is still an issue somewhere, presumably with negative step sizes (when |
fix small deviations in some backends add (required) min and max step number selection for dynamic integration improve dispatch remove step size selection (users can compute this if they need it, but exposing the argument is ambiguous w.r.t. fixed vs adaptive integration)
add default integrate kwargs
This PR:
The current code for integration is still a little messy due to lack of proper encapsulation. However, we can update this as we go without much impact to the user, if any.
I tested this running Flow Matching on the Two Moons example, and on some toy examples, both in non-compiled and compiled contexts.