Optimizations & Improvements: Control Flow Redesign, Polars DataFrames, and Margin Level Prioritizations

[martinezjorge]

Hello my name is Jorge and I'm happy to present some ideas and improvements on the liquidator bot!

PR Content and Context

I've been trying to improve the performance of the liquidator bot and I was experimenting with a few different mechanisms and strategies so if this is too big of a PR or there are some ideas that don't make sense I'm happy to separate the changes into more specific features and omit unwanted features. Also would love to have a discussion about the different ways that liquidator bot could be optimized, improved, redesigned, or rewritten; especially if its completely different to what I'm presenting here or if I'm missing something obvious.

The main effort was to prioritize accounts that are closer to being liquidated but without letting the overall margin account dataset get stale. With that in mind, I made the following high level changes:

1. Reduce the amount of accounts being scanned
2. Increase the frequency of the smaller scans (configured with environment variables)
3. At longer periodic intervals for full margin account scans (configured with environment variables)
4. Collect data on the accounts being scanned
5. Utilize the data to curate the priority margin accounts

What I noticed was that all margin accounts were being read in and parsed at the specified interval configured in .env. At the time of this writing there are ~222k margin accounts. By filtering out margin accounts that don't have any required margin I was able to bring down the relevant amount of accounts to ~17k margin accounts. Then by sorting by margin levels in increasing order we can see the accounts that are closest to being liquidated. We can then further bound the margin accounts with arbitrary margin level bounds that approach liquidation margin level. With this method, I was able to further reduce the amount of accounts scanned to ~11k accounts which is about 5% of all margin accounts and further reduction is possible.

I changed the interval environment variable from INTERVAL to PRICE_FEEDS_UPDATE_INTERVAL and ALL_MARGIN_ACCOUNTS_SCAN_INTERVAL. I moved the timer in the bottom to a general one second timer to allow for simultaneous countdowns for routines (price update routine, full margin account scan routine, ...etc) . To separate full account scans from priority account scans after a fresh price update.

So now the ~11k prioritized accounts can get scanned more frequently. Prior to these changes scanning all the accounts took 40-80 seconds and sometimes longer, but now the priority account scans, ~11k accounts, take about 7-10 seconds and since they happen each time the priceFeeds are updated they now will happen more frequently.

I added some tests to develop and integrate the changes but they are very much proof of concepts, experimental, and more like one-off scripts than a test suite. They are better off being run with -g flag to grep specific unit tests.

I refactored a lot of the code in liquidator.ts to utils directories to try to clean up the main loop but this is still a work in progress.

I used polar dataframes to analyze the margin account data. I'm not sure if its faster in every case but it facilitates data engineering, querying, and the writing and saving of CSV files.

Future Work

1. With the ability to collect data on accounts the collected data could be mined in order to find features that could possibly predict which accounts might liquidate or are most/more likely to get liquidated in order increase liquidator performance.

2. There are likely unnecessary account readings that could be done at lower frequencies to reduce the possible of throttling of a user's RPC node.

3. Further refine the routine flows.