add later credits

[web.git] / personal / _posts / 2022-04-11-provenance-exposed.md

diff --git a/personal/_posts/2022-04-11-provenance-exposed.md b/personal/_posts/2022-04-11-provenance-exposed.md
index e5560e275e1556d0a4cec4ab1430ecb7f79808cf..709dbc6cae1958444bc4ffaf4f53d6919700d42c 100644 (file)
--- a/personal/_posts/2022-04-11-provenance-exposed.md
+++ b/personal/_posts/2022-04-11-provenance-exposed.md
@@ -277,13 +277,21 @@ Because of that, I think we should move towards discouraging, deprecating, or ev
 That means a cast is the only legal way to turn a pointer into an integer, and after the discussion above we got our casts covered.
 A [first careful step](https://github.com/rust-lang/rust/pull/95547) has recently been taken on this journey; the `mem::transmute` documentation now cautions against using this function to turn pointers into integers.
 
 That means a cast is the only legal way to turn a pointer into an integer, and after the discussion above we got our casts covered.
 A [first careful step](https://github.com/rust-lang/rust/pull/95547) has recently been taken on this journey; the `mem::transmute` documentation now cautions against using this function to turn pointers into integers.
 

+**Update (2022-09-14):** After a lot more discussion, the current model pursued by the Unsafe Code Guidelines WG is to say that pointer-to-integer transmutation is permitted, but just strips provenance without exposing it.
+That means the program with the casts replaced by transmutation is UB, because the `ptr` it ends up dereferencing has invalid provenance.
+However, the transmutation itself is not UB.
+Basically, pointer-to-integer transmutation is equivalent to [the `addr` method](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.addr), with all its caveats -- in particular, transmuting a pointer to an integer and back is like calling `addr` and then calling [`ptr::invalid`](https://doc.rust-lang.org/nightly/std/ptr/fn.invalid.html).
+That is a *lossy* round-trip: it loses provenance information, making the resulting pointer invalid to dereference.
+It is lossy even if we use a regular integer-to-pointer cast (or `from_exposed_addr`) for the conversion back to a pointer, since the original provenance might never have been exposed.
+Compared to declaring the transmutation itself UB, this model has some nice properties that help compiler optimizations (such as removing unnecessary store-load round-trips). **/Update**
+

 ## A new hope for Rust
 
 All in all, while the situation may be very complicated, I am actually more hopeful than ever that we can have both -- a precise memory model for Rust *and* all the optimizations we can hope for!
 The three core pillars of this approach are:
 - making pointer-integer casts "expose" the pointer's provenance,
 - offering `ptr.addr()` to learn a pointer's address *without* exposing its provenance,
 ## A new hope for Rust
 
 All in all, while the situation may be very complicated, I am actually more hopeful than ever that we can have both -- a precise memory model for Rust *and* all the optimizations we can hope for!
 The three core pillars of this approach are:
 - making pointer-integer casts "expose" the pointer's provenance,
 - offering `ptr.addr()` to learn a pointer's address *without* exposing its provenance,

-- and disallowing pointer-integer transmutation.
+- and making pointer-integer transmutation round-trips lossy (such that the resulting pointer cannot be dereferenced).

 
 Together, they imply that we can optimize "nice" code (that follows Strict Provenance, and does not "expose" or use integer-pointer casts) perfectly, without any risk of breaking code that does use pointer-integer round-trips.
 In the easiest possible approach, the compiler can simply treat pointer-integer and integer-pointer casts as calls to some opaque external function.
 
 Together, they imply that we can optimize "nice" code (that follows Strict Provenance, and does not "expose" or use integer-pointer casts) perfectly, without any risk of breaking code that does use pointer-integer round-trips.
 In the easiest possible approach, the compiler can simply treat pointer-integer and integer-pointer casts as calls to some opaque external function.


@@ -328,7 +336,7 @@ Compositionality at its finest!
 
 I have talked a lot about my vision for "solving" pointer provenance in Rust.
 What about other languages?
 
 I have talked a lot about my vision for "solving" pointer provenance in Rust.
 What about other languages?

-As you might have heard, C is moving towards making [PNVI-ae-udi](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2577.pdf) an official recommendation for how to interpret the C memory model.
+As you might have heard, C is moving towards making [PNVI-ae-udi](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2676.pdf) an official recommendation for how to interpret the C memory model.

 With C having so much more legacy code to care about and many more stakeholders than Rust does, this is an impressive achievement!
 How does it compare to all I said above?
 
 With C having so much more legacy code to care about and many more stakeholders than Rust does, this is an impressive achievement!
 How does it compare to all I said above?
 


@@ -367,8 +375,8 @@ Because of all that, I think it is reasonable for Rust to make a different choic
 
 This was a long post, but I hope you found it worth reading. :)
 To summarize, my concrete calls for action in Rust are:
 
 This was a long post, but I hope you found it worth reading. :)
 To summarize, my concrete calls for action in Rust are:

-- Code that uses pointer-integer transmutation should migrate to regular casts or `MaybeUninit` transmutation ASAP.
-  I think we should declare pointer-integer transmutation Undefined Behavior and not accept such code as well-defined.
+- Code that uses pointer-integer transmutation round-trips should migrate to regular casts or `MaybeUninit` transmutation ASAP.
+  I think we should declare pointer-integer transmutation as "losing" provenance, so code that assumes a lossless transmutation round-trip has Undefined Behavior.

 - Code that uses pointer-integer or integer-pointer *casts* might consider migrating to the Strict Provenance APIs.
   You can do this even on stable with [this polyfill crate](https://crates.io/crates/sptr).
   However, such code *is and remains* well-defined. It just might not be optimized as well as one could hope, it might not compile on CHERI, and Miri will probably miss some bugs.
 - Code that uses pointer-integer or integer-pointer *casts* might consider migrating to the Strict Provenance APIs.
   You can do this even on stable with [this polyfill crate](https://crates.io/crates/sptr).
   However, such code *is and remains* well-defined. It just might not be optimized as well as one could hope, it might not compile on CHERI, and Miri will probably miss some bugs.


@@ -421,6 +429,10 @@ My personal stance is that we should not let the cast synthesize a new provenanc
 This would entirely lose the benefit I discussed above of making pointer-integer round-trips a *local* concern -- if these round-trips produce new, never-before-seen kinds of provenance, then the entire rest of the memory model has to define how it deals with those provenances.
 We already have no choice but treat pointer-integer casts as an operation with side-effects; let's just do the same with integer-pointer casts and remain sure that no matter what the aliasing rules are, they will work fine even in the presence of pointer-integer round-trips.
 
 This would entirely lose the benefit I discussed above of making pointer-integer round-trips a *local* concern -- if these round-trips produce new, never-before-seen kinds of provenance, then the entire rest of the memory model has to define how it deals with those provenances.
 We already have no choice but treat pointer-integer casts as an operation with side-effects; let's just do the same with integer-pointer casts and remain sure that no matter what the aliasing rules are, they will work fine even in the presence of pointer-integer round-trips.
 

+That said, under this model integer-pointer casts still have no side-effect, in the sense that just removing them (if their result is unused) is fine.
+Hence, it *could* make sense to implicitly perform integer-pointer casts in some situations, like when an integer value (without provenance) is used in a pointer operation (due to an integer-to-pointer transmutation).
+This breaks some optimizations like load fusion (turning two loads into one assumes the same provenance was picked both times), but most optimizations (in particular dead code elimination) are unaffected.
+

 #### What about LLVM?
 
 I discussed above how my vision for Rust relates to the direction C is moving towards.
 #### What about LLVM?
 
 I discussed above how my vision for Rust relates to the direction C is moving towards.