std: rework uefi allocators #22818

RossComputerGuy · 2025-02-08T17:38:34Z

Split off from #22226 as suggested by @linusg. This reworks the allocators to be more friendly.

linusg

Thank you, this is the right size for a PR like this :)

Two high level concerns:

All allocators, including platform-specific ones like WasmAllocator currently live in std.heap. If we're moving these around that should be where.
Why are we adding additional abstractions on top of the UEFI-provided allocator functions? In general that's a good thing if it improves ergonomics of those APIs from a Zig perspective, but in this case no one but the allocator implementation should touch those, and I feel in that case the extra layer is not justified.

RossComputerGuy · 2025-02-09T01:11:06Z

All allocators, including platform-specific ones like WasmAllocator currently live in std.heap. If we're moving these around that should be where.

Will do

Why are we adding additional abstractions on top of the UEFI-provided allocator functions? In general that's a good thing if it improves ergonomics of those APIs from a Zig perspective, but in this case no one but the allocator implementation should touch those, and I feel in that case the extra layer is not justified.

I'm not 100% sure, this originally came from @truemedian.

truemedian · 2025-02-10T02:42:40Z

Why are we adding additional abstractions

Which additional abstractions are you referring to?

PageAllocator.resize requesting new pages at the end of the current allocation happens because it's essentially free and part of the UEFI page allocation interface

PoolAllocator exists because RawPoolAllocator cannot handle the full range of possible alignment values the Allocator interface requires

linusg · 2025-02-10T10:44:44Z

Which additional abstractions are you referring to?

The ones in boot_services.zig.

truemedian · 2025-02-10T13:26:24Z

Ah, thats because this came out of a branch where the goal was to make std.os.uefi a more idiomatic interface instead of just a loose C library binding.

linusg

Please rebase this after #22820 and provide a compelling use case for the custom layer in boot_services.zig or remove it.

linusg · 2025-02-20T00:44:14Z

lib/std/heap/uefi_allocators.zig

+/// Allocates memory in pages.
+///
+/// This allocator is backed by `allocatePages` and is therefore only suitable for usage when Boot Services are available.
+pub const PageAllocator = struct {


The namespacing here is awkward, this ought to be a file-as-struct (UefiPageAllocator.zig).

Why file as a struct when there's multiple allocators?

Because std.heap.UefiPageAllocator is much nicer than std.heap.uefi_allocators.PageAllocator (which violates https://ziglang.org/documentation/master/#Avoid-Redundant-Names-in-Fully-Qualified-Namespaces). Why do you want both allocators in the same file?

Because std.heap.UefiPageAllocator is much nicer than std.heap.uefi_allocators.PageAllocator

Wouldn't std.heap.uefi.Page/Pool be nicer?

Why do you want both allocators in the same file?

Just so there isn't 3 more files that could fit into 1.

Wouldn't std.heap.uefi.Page/Pool be nicer?

Not really, a "page" and a "page allocator" are different things. Avoiding redundancy does not mean to leave out important bits of information until the name has a different meaning.

Just so there isn't 3 more files that could fit into 1.

That's not a good reason.

Ok, I've managed to find the time and split the allocators into their own files.

RossComputerGuy · 2025-02-20T02:20:43Z

Please rebase this after #22820

Already been rebased.

provide a compelling use case for the custom layer in boot_services.zig or remove it.

I don't have one unless @truemedian does. I think it just makes the API easier to use and it's already there. Shouldn't need additional maintenance since UEFI's API doesn't really change in that section. I probably will just remove it.

linusg · 2025-02-20T02:30:38Z

Already been rebased.

I see at least one reference to .LoaderData which has been renamed.

RossComputerGuy · 2025-02-20T02:34:20Z

I see at least one reference to .LoaderData which has been renamed.

Fixed, strange that one didn't get caught when I tried compiling lib/init.

linusg · 2025-02-20T02:39:47Z

I think it just makes the API easier to use and it's already there. Shouldn't need additional maintenance

For the most part Zig follows YAGNI principles.

linusg

Parts of this will be superseded by #23441.

RossComputerGuy · 2025-04-02T19:48:30Z

Great, I'll rebase once that lands. Though I'm still wondering about #22818 (comment).

dotcarmen

this pr should now be unblocked as well since #23441 landed :)

dotcarmen · 2025-07-18T12:22:05Z

lib/std/heap.zig

@@ -24,6 +24,8 @@ pub const GeneralPurposeAllocatorConfig = DebugAllocatorConfig;
 /// Deprecated; to be removed after 0.14.0 is tagged.
 pub const GeneralPurposeAllocator = DebugAllocator;

+pub const uefi = @import("heap/uefi_allocators.zig");


i prefer to keep these in std.os.uefi, any reason to move them?

I forget who mentioned it but someone here mentioned that this is what should be done.

It was me, but I've changed my mind - the split into file-as-struct is nice but let's keep them in std.os.uefi for now.

dotcarmen · 2025-07-18T12:47:01Z

lib/std/os/uefi/tables/boot_services.zig

@@ -38,19 +40,19 @@ pub const BootServices = extern struct {
    restoreTpl: *const fn (old_tpl: usize) callconv(cc) void,

    /// Allocates memory pages from the system.
-    allocatePages: *const fn (alloc_type: AllocateType, mem_type: MemoryType, pages: usize, memory: *[*]align(4096) u8) callconv(cc) Status,
+    _allocatePages: *const fn (alloc_type: AllocateType.Enum, mem_type: MemoryType, pages: usize, memory: PhysicalAddress) callconv(cc) Status,


oh that's a good point that i didn't address in my pr: the memory returned is a physical address, which shouldn't be used directly as a pointer

once you rebase i think it might make sense to keep that change (as opposed to *[*]uefi.Page), with the caveat that it should be memory: *PhysicalAddress and to return that PhysicalAddress from allocatePages. idk though, that would imply changing eg convertPointer which could get hairy

alternatively it would be nice if there was std.builtin.AddressSpace.physical but that could come with a cascade of changes that aren't necessarily relevant here.

RossComputerGuy · 2025-07-18T21:27:27Z

Rebased, I haven't adjusted much but hopefully I resolved the conflicts well.

dotcarmen · 2025-07-20T18:11:37Z

lib/std/os/uefi/tables/boot_services.zig

@@ -377,7 +379,7 @@ pub const BootServices = extern struct {
        mem_type: MemoryType,
        pages: usize,
    ) AllocatePagesError![]align(4096) Page {
-        var ptr: [*]align(4096) Page = switch (location) {
+        var ptr: PhysicalAddress = switch (location) {


you also have to fix the return type of the function and the .success arm of the switch below

RossComputerGuy mentioned this pull request Feb 8, 2025

std: implement basic io for UEFI #22226

Open

linusg reviewed Feb 8, 2025

View reviewed changes

RossComputerGuy force-pushed the feat/uefi-alloc-rework branch from e0ab1e2 to da5fad6 Compare February 10, 2025 02:25

RossComputerGuy force-pushed the feat/uefi-alloc-rework branch 2 times, most recently from 7efc07c to a9862db Compare February 10, 2025 03:35

linusg requested changes Feb 20, 2025

View reviewed changes

RossComputerGuy force-pushed the feat/uefi-alloc-rework branch from a9862db to 33af113 Compare February 20, 2025 02:33

linusg mentioned this pull request Apr 2, 2025

std.os.uefi.tables: ziggify boot and runtime services #23441

Merged

linusg reviewed Apr 2, 2025

View reviewed changes

RossComputerGuy force-pushed the feat/uefi-alloc-rework branch from 33af113 to 9611057 Compare April 10, 2025 07:24

dotcarmen reviewed Jul 18, 2025

View reviewed changes

RossComputerGuy added 4 commits July 18, 2025 14:12

std.os.uefi.tables.boot_services: adjust allocate page

b287b34

std.os.uefi: refactor allocators

789a932

std.os.uefi: move allocators to heap

7952530

std.os.heap.uefi_allocators: split each allocator into a separate file

d5966c8

RossComputerGuy force-pushed the feat/uefi-alloc-rework branch from 9611057 to d5966c8 Compare July 18, 2025 21:27

dotcarmen reviewed Jul 20, 2025

View reviewed changes

Uh oh!

std: rework uefi allocators #22818

Are you sure you want to change the base?

std: rework uefi allocators #22818

Uh oh!

Conversation

RossComputerGuy commented Feb 8, 2025

Uh oh!

linusg left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented Feb 9, 2025

Uh oh!

truemedian commented Feb 10, 2025

Uh oh!

linusg commented Feb 10, 2025

Uh oh!

truemedian commented Feb 10, 2025

Uh oh!

linusg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented Feb 20, 2025

Uh oh!

linusg commented Feb 20, 2025

Uh oh!

RossComputerGuy commented Feb 20, 2025

Uh oh!

linusg commented Feb 20, 2025

Uh oh!

linusg left a comment

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented Apr 2, 2025

Uh oh!

dotcarmen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented Jul 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

linusg left a comment •

edited

Loading