RFC: Unsafe Set Enum Discriminants

[jamesmunns]

Summary

This RFC proposes a way to write the discriminant of an enum when building it "from scratch". This introduces two new library components, an unsafe set_discriminant function, and a discriminant_of! macro, which can be used with any enum, regardless of repr or any other details.

This RFC is a follow-on to the offset_of_enum feature, tracked by rust-lang/rust#120141.

Rendered

[jamesmunns]

@RalfJung I would appreciate if you could let me know if I got this right! I don't totally understand the "mandatory readback", so if this could be stated better, please feel free to propose alternate wording.

[traviscross]

It'd be good also for this (or the following) section to be extended with more details about why this is the correct semantic. I think I see the case for and agree that setting a MaybeUninit<Option<&u8>> to Some with set_discriminant should represent an assertion that the relevant bits aren't zero at the point of that call, but it's natural to wonder about this, and so it'd be good to describe it in some detail.

[jamesmunns]

Addressed all review comments so far. Link to the diff of changes since the initial version: https://github.com/rust-lang/rfcs/pull/3727/files/5c74b691972d36d30e27f253e8e2673d9930316f..eb486ef529eea98b7d2d9b0ffcd2254ac16a4ea0

[coolreader18]

What's the current legality of initializing niche-optimized enums?

let mut x: Option<NonNull<u8>> = None;
(&raw mut x).cast::<NonNull<u8>>().write(NonNull::dangling()); // UB?

I suppose there are guarantees made about the layout of Option for FFI purposes, but this feels... sketchy (though miri says it's fine, including with more complex niches like char). That to say, unless these semantics are already defined, I feel like the section about calling set_discriminant on niche-optimized enums should be strengthened from "encouraged for explicitness" to "should"/"must". Maybe with an exception for Option/Result-like enums with a single niche at 0, since FFI-compatibility guarantees are already made for those, but I'd err on not allowing it if possible.

[matthieu-m]

I was thinking about the (possible) future alternative receivers for set_discriminant:

• Having to call .as_mut_ptr() is not so bad, what is more annoying is the loss of guarantees. Yes my &mut MaybeUninit has given me a non-null & sufficiently-aligned pointer, thank you very much. This is the true boilerplate.
• Having set_discriminant_mu and set_discriminant_nn (or whatever) feels... heavy. Remembering the specific names is going to be annoying.
• A SetDiscriminant::set_discriminant trait or a set_discriminant<T: SealedDiscriminant>(t: T, ..) method muddle safety guarantees: it makes it harder, at the call site, to ensure that the expected receiver is used, and it makes it brittle with regard to refactorings. Change the "receiver" from NonNull<T> to *mut T? Make sure the pointer is non-null now!
• Inherent methods on NonNull and MaybeUninit directly is another possibility. If just called set_discriminant they would have some of the disadvantages of the trait-based approach, but for MaybeUninit a write_discriminant inherent method could be more in keeping with the current write... API and the different name would prevent a maintenance mistake.

So, all in all, for now I see two alternatives I like:

• Don't introduce any variant. Bit boilerplatey in safety guarantees, but very explicit, and explicit is not so bad for unsafe.
• Only introduce a single variant: MaybeUninit::<T>::write_discriminant(&mut self, disc: Discriminant<T>). It's natural to have, and offers a lot of guarantees that *mut T doesn't -- non null, sufficiently aligned, exclusive access -- leaving only "correctly initialized" to fulfill.

[traviscross]

Unresolved questions are things that are intended to be required to be answered before stabilization. In this case, the other forms could always be added later, so it may be worth moving this to a future possibility instead.

That is, unless you mean for there to be an open question about whether we'd want the *mut T form at all, but I think @RalfJung sufficiently addressed that in the Zulip thread about how it would be inconsistent with our many other such functions for there to not be a *mut T form of this.