add off-heap tracking allocator to support overall memory quota

[privateJiangyaokai]

Forgive me I am mostly a Java programmer.
So I noticed that --heap-size-limit or --max-old-space-size has no effect on arrays. I did a bit of investigation and it seems like the proper way of actually limiting the total memory consumption of an Isolate is via a custom allocator.
I wrote a preliminary implementation, could you check it out?

[caoccao]

I like your idea. It seems it comes out of an agent, right? Here are my feedback for your reference.

1. That would bring breaking changes. Breaking implies 'Off the ECMA spec'.
2. It's easy to have that kind of check or protection in pure JavaScript. You may redefine Array.push() and other similar API so that they check the length, measures the memory, etc. That would be much more flexible. That's also why I always want to avoid changing V8 source code, replacing V8's built-in features.
3. ArrayBuffer's memory can be directly from Java. In other words, the backing store can be in JVM. This calculation would be mixing the JVM and non-JVM memory.
4. swc4j provides compelling static analysis at TS/JS AST level. It can be used to prevent malicious scripts.

[privateJiangyaokai]

It is not from an agent. Though I did use AI to generate a lot of the code. The core idea was to enable accurately letting V8 GC to know the size of off-heap memory by calling AdjustAmountOfExternalAllocatedMemory.

1. AFAIK this is the proper way of doing it in V8 and does not break ECMA spec?
2. Ditto
3. You are right. I didn't know about Javet's handling of ArrayBuffer's memory allocation.
4. If I understood correctly, swc4j, being a static analsis tool, can not catch malicious scripts inside "eval"s, and sometimes these are legitimate uses.

I'll see if I can combine all the sources of off-heap memory allocation when running V8 via Javet. Do you think that's viable?

[caoccao]

Thank you sharing your thoughts.

• swc4j can look inside eval() with some limitations. swc4j has limited ability of statically evaluating expressions. Please review my blog posts about decoding jsfxxk. In those blog posts eval() is evaluated in Java.
• A public SaaS should disable eval() in all cases. So the issue itself doesn't exist.
• ArrayBuffer, TypedArray can be virtualized in JS as well. We can check the requested memory size in JS and reject the requests that exceed the limit. This is a much better solution because the scripts get a native JS error indicating why this allocation is not allowed. This even works inside eval().

In summary, there's no need to hack V8 for such feature. In JS, you have plenty of ways that are much more flexible, powerful, user friendly.

[privateJiangyaokai]

I see!!
Thanks for your input.
I'll look into virtualizing ArrayBuffer and TypedArray. Looks like a better place to start than solely relying on AdjustAmountOfExternalAllocatedMemory.