[Question] Is there a reason to not support pagemaps level 5 ?

[linkdd]

PML5 is an Intel extension to increase addressable space from 256TB to 128PB (see wikipedia).

I admit it is a niche use case 🙂 But I was still wondering if there is any reason to not support it?

Since it uses bits from 48 to 57 for the new level, I guess it would break the safety checks for the VirtAddr type, are there other things that would break?

NB: The limine bootloader provides a way to detect if the CPU has the PML5 extension.

[Freax13]

AFAIK there is no reason we don't support 5-level paging, it's just that no one has implemented it yet. It might require redesigning some of the abstractions to support both 4-level and 5-level paging. Feel free to come up with ideas.

Since it uses bits from 48 to 57 for the new level, I guess it would break the safety checks for the VirtAddr type, are there other things that would break?

This would also break/require redesign of the page table related abstraction including Mapper and its implementations.

[phil-opp]

For VirtAddr, we could add a new generic parameter that allows specifying 4-level or 5-level paging. If we default to 4-level paging, we might be even able to avoid a breaking change. Maybe it's also possible to something similar for OffsetPageTable, etc.

[sarahspberrypi]

I noticed that the bitmask of the addr() function of PageTableEntry (see here) already supports 5-level paging (with 52 Bits) but the VirtAddr::new() function still only expects a 4-level address which is 47 bits long. This broke in my code when using SEV due to the C-Bit-Position being at either bit 47 (for older devices) or at bit 51 (for never AMD CPUs). I think we should adapt VirtAddr to allow bits 47-51 to be non-machting as well.
Side note: If we align the bitmasks here with VirtAddrs requirements, we can safely use VirtAddr::new_unchecked() at this location.

[Freax13]

I noticed that the bitmask of the addr() function of PageTableEntry (see here) already supports 5-level paging (with 52 Bits) but the VirtAddr::new() function still only expects a 4-level address which is 47 bits long. This broke in my code when using SEV due to the C-Bit-Position being at either bit 47 (for older devices) or at bit 51 (for never AMD CPUs). I think we should adapt VirtAddr to allow bits 47-51 to be non-machting as well. Side note: If we align the bitmasks here with VirtAddrs requirements, we can safely use VirtAddr::new_unchecked() at this location.

(This misunderstands a few concepts/our APIs, there's been some discussion on this in #511).

[Freax13]

For VirtAddr, we could add a new generic parameter that allows specifying 4-level or 5-level paging. If we default to 4-level paging, we might be even able to avoid a breaking change. Maybe it's also possible to something similar for OffsetPageTable, etc.

Unfortunately, I don't think this can work without a breaking change because Rust doesn't allow specifying a default for generic parameters in functions.

[Freax13]

Here's another idea: We could add a cargo feature to enable 5 level paging. When this feature is enabled, the Page and VirtAddr address types would use 5 level paging. If enabled, we'd expose additional functions like p5_index(). The Mapper implementations (in this crate) would be aware of this and map one more level when the feature is active.

I wrote up a proof of concept for this a while ago and didn't come across any major issues.

One downside of this approach is that it's not possible to select between 4 level and 5 level paging at runtime. That said, I'm not sure if it's feasible to duplicate all code to have a generic instantiation for 4 level paging and one for 5 level paging. At best, we can probably make the mapper implementations aware of the current paging configuration, but I'm not sure that we can do this for the canonality checks.

Given that it's unlikely that we can avoid a breaking change while adding simultaneous 4 and 5 level paging support, I'd suggest that we go with adding a cargo feature for now and reevaluate this when there's a need to support both at the same time. Personally, I'm not really sure what users would want/need from us to support this.
Adding a cargo feature to enable 5 level paging shouldn't be a breaking change.

[phil-opp]

I don't think that that a cargo feature would be a good idea since cargo unifies features across dependencies. So if your kernel enables the 5 level paging feature, the bahavior of VirtAddr would also change in all dependencies that use x86_64 too.

The addional generic parameter on VirtAddr and the page table types still seems like the cleanest solution to me. We could add type aliases to avoid spilling the generics into normal user code.

[Freax13]

This goes back to my assumption that users will know at compile times whether they want to use 4 or 5 level paging. In that case, users could set the feature in the top level crate and all dependencies would automatically use the correct paging config. Library authors would not need to add explicit support for 4 and 5 level paging because the address types would automatically do the right thing (thanks to feature unification). Library crates should not (unconditionally) set the feature.

Another way to phrase this would be that the feature doesn't indicate support, but the desired configuration. Libraries wouldn't set this flag because they support 5 level paging, they'd only set it if they require 5 level paging.

To be fair, I'm not sure that it's fair to assume that users will know the paging configuration at compile time, but wouldn't they also need to know this when using generics? Sure, they have the option to use types for both configurations, but in practice, I don't think it's feasible to make a whole kernel/user program generic over the paging config, so they'd likely have to pick one.

One downside of using generics to differentiate between 4 and 5 level paging is that library authors will have to explicitly opt-in by adding generic parameters to all of their types/traits/functions. Depending on how heavily they use the address types, that could be quite a large change. If we can't make adding support for 5 level page a non-breaking change, chances are libraries would have to make breaking changes as well if they wanted to add support.

[linkdd]

5 level paging is only usable if the CPU supports it. Using a compile-time flag to enable it would surely reduce the compatibility of the OS you're writing, no?

[josephlr]

I agree that Cargo features are not the right approach here. They are additive and have a tendency to get virally enabled.

For VirtAddr and PhysAddr (and structures containing them like Page and PhysFrame), I think it would make more sense to have generic parameters encoding the validity range. Something like:

trait VirtValidity {
    fn check_virt_addr(v: u64) -> Result<(), VirtAddrNotValid>;
}
trait PhysValidity {
    fn check_phys_addr(v: u64) -> Result<(), PhysAddrNotValid>;
}

Then VirtAddr becomes VirtAddr<V: VirtValidity = Virt48> with:

T: VirtValidity	check_virt_addr behavior
Virt48	Bits 48-64 must match bit 47
Virt57	Bits 57-64 must match bit 56
VirtRt	Checks Cr4Flags::L5_PAGING at runtime to get allowed range

And PhysAddr becomes PhysAddr<V: PhysValidity = Phys52> with:

T: PhysValidity	check_phys_addr behavior
Phys52	Bits 52-64 must be 0
Phys48	Bits 48-64 must be 0
PhysEnc	Requires the encryption bit to be clear if memory encryption is on.
PhysRt	Checks CPUID[0x80000008] at runtime to get allowed range

This allows us to support extensions and different Physical/Virtual address ranges, while keeping a nice and ergonomic default.

[josephlr]

However, such a generic approach would likely require rust-lang/rust#143874 (#![feature(const_trait_impl)]), as we would want check_virt_addr to be const for Virt48, but it cannot be const for VirtRt.

[Freax13]

I started sketching out the generic parameter approach. I opened a draft PR: #586. It's still very much work in progress, but I gives us something to look at when discussing the right approach.

FWIW, I found my old WIP patch that uses a cargo feature instead: Freax13@d288b37. That patch is also not yet completely finished up, but it is fairly close because that approach doesn't require touching as many parts of the code base. I think that main thing that was missing is that I didn't do a whole lot of testing yet.

[Freax13]

5 level paging is only usable if the CPU supports it. Using a compile-time flag to enable it would surely reduce the compatibility of the OS you're writing, no?

Yes, that's fair. That said, the OS itself could also have a build flag for 5 level paging and could forward that to the x86_64 crate. That way, it'd still be possible to support both although not in the same binary.
AFAICT that's what windows is doing with ntoskrnl.exe and ntkrla57.exe. The former only supports 4 level paging and the later supports 5 level paging (I'm not completely sure if it only supports 5 level paging or if it supports both).
Linux supports 4 and 5 level paging in the same kernel binary.