You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current Implementation of Persistence.Query may not scale well on certain queries.
The biggest problem is that we are using .ToList() in places that in large systems may return large results.
We have two options. The first of which is a 'universal' fix and can easily be implemented as an opt-in feature, the second will require a little more work but will be better for users long term.
Refactor methods that can use the 'batched chunk' read style used by MessagesWithBatch
Option 1 is the 'safest' and should work with all DBs with the possible exception of SQLite (because of it's tendencies for readers and writers to block each other.) Option 2 will be better performing.
I'm expecting that the end state will be that most queries will be better under 'batched' reads, but there will be one or two (I'm thinking mainly CurrentPersistenceIds()) that will still benefit from the option of switching.
The text was updated successfully, but these errors were encountered:
The current Implementation of Persistence.Query may not scale well on certain queries.
The biggest problem is that we are using
.ToList()
in places that in large systems may return large results.We have two options. The first of which is a 'universal' fix and can easily be implemented as an opt-in feature, the second will require a little more work but will be better for users long term.
.ToListAsync()
to instead stream off anIAsyncEnumerable
, via https://gist.github.com/to11mtm/dc9a350080fcbcb14098c14509d70e7fMessagesWithBatch
Option 1 is the 'safest' and should work with all DBs with the possible exception of SQLite (because of it's tendencies for readers and writers to block each other.) Option 2 will be better performing.
I'm expecting that the end state will be that most queries will be better under 'batched' reads, but there will be one or two (I'm thinking mainly
CurrentPersistenceIds()
) that will still benefit from the option of switching.The text was updated successfully, but these errors were encountered: