A method for cranelift code-gen to write compiled code on an already allocated memory

[farooqkz]

Hello Hello.

I have an application in which I need to compiled a huge number of programs of different sizes, and then execute each one of them on a large dataset. Also note that the programs change rapidly, so each time I should compile them again.

Currently, I do:

• Compile each program using cranelift, so I have a CompiledCode
• Allocate a memory mapped region
• Copy the bytes of compiled code from buffer of CompiledCode to the memory mapped allocation
• Make the memmap executable
• Transmute the memmap into a function
• Execute the function on the dataset

Typically, there are 1k-10k programs. And memset has become a very big bottleneck in performance.

What I need to do, is allocate a memmap, and then ask cranelift to write bytes on that region. There is compile_and_emit. But it has got two problems, first you are deprecating this method. Second, this wants a Vec<u8> instead of &mut [u8].

The immediate solution to me as an outsider seems keeping the method and changing it to accept a slice instead of a vec. The right solution, I don't know :)

[bjorn3]

But it has got two problems, first you are deprecating this method. Second, this wants a Vec instead of &mut [u8].

compile_and_emit is also doing a memcpy. This is the implementation:

let compiled_code = self.compile(isa, ctrl_plane)?;
mem.extend_from_slice(compiled_code.code_buffer());
Ok(compiled_code)

Avoiding the memcpy would require threading an allocator deep into the machine code emission code as we don't know the machine code size until after we are fully done emitting it.

And memset has become a very big bottleneck in performance.

Hang on, you are saying that memset is the bottleneck, not memcpy? What is memset even being used for? Are you clearing out the executable region of memory every time or something?

[cfallin]

As bjorn3 mentions, this is quite a bit more complex than "accept a slice instead of a vec", unfortunately. We don't know the final size (how do we communicate "out of space, try again" to the embedder?); and the resulting lifetime propagation would be very infectious. There's a high bar for such a change.

And on that: I am quite surprised that you're seeing raw memory throughput as a bottleneck and not "the rest of the compiler" -- Cranelift certainly cannot generate machine code at peak machine memory bandwidth. Can you share a profile (e.g. a flamegraph) to help us understand?