Binary Ninja uses too-low buffer size when reading bndb

[comex]

Version and Platform (required):

• Binary Ninja Version: 4.3.6879-dev (333734f4)
• OS: macOS
• OS Version: 15.3 (24D60)
• CPU Architecture: Apple M3 Max

Bug Description:
One of my bndbs has grown to 11GB. I've noticed that when I open the bndb, Binary Ninja spends almost 30 seconds just calling pread to load the bndb into memory. During this time, the UI is frozen. For the pread calls, the offsets are not always sequential but often are, while the read size is always 1024 bytes.

1024 bytes seems like way too low a read size. Here is a simplistic comparison using dd (although this is sequential):

% dd if=main_301.bndb of=/dev/null bs=1024 
[..]
11609607168 bytes transferred in 13.495961 secs (860228269 bytes/sec)
% dd if=main_301.bndb of=/dev/null bs=1048576
[..]
11609607168 bytes transferred in 0.684243 secs (16967082116 bytes/sec)

(After this phase of loading is done, the UI becomes responsive again and Binary Ninja starts using CPU on multiple threads while the progress bar ticks up.)

Consider implementing some sort of cache so that you can use a larger read size… or just use mmap.

Also consider fixing the UI to not be unresponsive while reading data.

[WeiN76LQh]

This was in part mentioned in #6288, in fact it might just be the entire problem in that issue too.

[negasora]

All the 1024 byte reads are because sqlite operates on blocks, as mentioned in https://sqlite.org/lockingv3.html

... From the point of view of the pager the database consists of a single file of uniform-sized blocks. Each block is called a "page" and is usually 1024 bytes in size. The pages are numbered beginning with 1. So the first 1024 bytes of the database are called "page 1" and the second 1024 bytes are call "page 2" and so forth. ...

The default block size changed to 4096 with sqlite 3.12.0, which we switched to in binja build 2904 - any database created before that uses the old value of 1024.

Can you make a copy of your database and check the value of the page_size pragma?

sqlite3 x.bndb "PRAGMA page_size"

We definitely shouldn't have the UI hang at all when opening databases, but that might explain why it's so bad.