Can Gramine Batch Multiple File Read/Write Requests in One OCall?

[BaoSKY]

My program involves file read/write operations, and to my knowledge, these typically require OCalls in SGX. How does Gramine handle this? Does every file read/write operation (e.g., C++ filestream operations) trigger an SGX OCall in Gramine as well?

I’ve conducted performance tests comparing file access times across three environments: REE, Gramine-direct, and Gramine-SGX. Below are the results for a single file read operation (in ns, averaged over multiple random accesses of varying file sizes):

• REE: 799, 815, 836, 847, 856, 941, 963, 1028, 1096, 1151, 1237, 1477, 1891

• Gramine-direct: 1428, 1425, 1445, 1522, 1542, 1562, 1572, 1638, 1707, 1752, 1834, 2074, 2489

• Gramine-SGX: 9256, 9407, 9394, 9401, 9445, 9442, 9494, 9634, 9654, 9715, 9827, 10134, 10495

As shown, there is a significant performance drop (~8000 ns) in Gramine-SGX, which I assume is due to the overhead introduced by SGX OCalls.

In my scenario, however, I can batch multiple read/write requests together, so ideally, only a single OCall would be required. Is there a way to optimize this in Gramine and reduce the number of OCalls?

[mkow]

In theory yes, but it's not implemented and would require the following:

• Your app would need to use scatter-gather I/O syscalls, like readv.
• The corresponding filesystem you use in Gramine would need to implement readv and writev callbacks in its struct libos_fs_ops.
• PAL API would need to get scatter-gather I/O API calls.
• ocall API would also need such a call added.

So, it's quite some work. Not complicated, but adds a bunch of new APIs we'd need to support. Not sure we want that, but I guess we should consider it if it actually significantly helps with performance.

[BaoSKY]

Thank you for your response. Although it doesn't directly resolve my issue, I appreciate your input.