From: Ralf Jung Date: Tue, 12 Feb 2019 08:56:31 +0000 (+0100) Subject: all-hands post X-Git-Url: https://git.ralfj.de/web.git/commitdiff_plain/ca23ad10f06218fd7c963557e3218a8d8db0658c all-hands post --- diff --git a/personal/_posts/2019-02-12-all-hands-recap.md b/personal/_posts/2019-02-12-all-hands-recap.md new file mode 100644 index 0000000..36a6be3 --- /dev/null +++ b/personal/_posts/2019-02-12-all-hands-recap.md @@ -0,0 +1,70 @@ +--- +title: "All-Hands 2019 Recap" +categories: rust +--- + +Last week, I was in Berlin at the Rust All-Hands 2019. +It was great! +I will miss nerding out in discussions about type theory and having every question answered by just going to the person who's the expert in that area, and asking them. +In this post, I am summarizing the progress we made in my main areas of interest and the discussions I was involved in---this is obviously just a small slice of all the things that happened. + + + +## Validity Invariants and `MaybeUninit` + +We had a session to talk about [validity invariants](https://github.com/rust-rfcs/unsafe-code-guidelines/blob/master/active_discussion/validity.md) ([meeting notes here](https://paper.dropbox.com/doc/Topic-Validity-Invariants--AXWuzG6GFwiTyUDukuPV7vDmAg-Wpl6mhrqNBStyrCHwhpYI#:uid=357396828564014720965680&h2=NOTES)). +No firm conclusions were reached, but we got input from people that haven't been part of the discussions inside the UCG (unsafe code guidelines) WG yet and that was very interesting. +It seems that deciding about the [validity invariant for references](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/77) and (to a lesser extend) the [validity invariant for unions](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/73) will be a slow process---there are lots of different options, and some of the trade-offs come down to prioritizing one value over another (like prioritizing checkable UB over optimizations, or vice versa). +Probably the closest to a conclusion we reached was that [uninitialized integers should probably not be UB](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/71#issuecomment-460599057) until you perform any operation on them. + +That said, it also got clear that the sooner we can stabilize (parts of) [`MaybeUninit`](https://github.com/rust-lang/rust/issues/53491), the better. +And as @japaric pointed out, we do not have to wait until all of these questions get answered! +We can stabilize a subset of the API, and leave (in particular) `get_ref` and `get_mut` out of that. +I think the biggest blocker for that is to get [RFC 2582](https://github.com/rust-lang/rfcs/pull/2582) accepted (which is very close to beginning its FCP). +Once that is done, I think I'll just beef up the documentation a bit and submit a stabilization PR. + +What worries me a bit is that we did not get much feedback from people using the API. +It seems like everyone is waiting for it to become stable, but of course then it will be too late to fix API mistakes that we made! +Given the importance of the API for certain classes of unsafe code, I think it would be great to get some more feedback before we set things in stone. + +To finish up the API, I went ahead and renamed the method that you call when a `MaybeUninit` is fully initialized to [`into_initialized`](https://doc.rust-lang.org/nightly/std/mem/union.MaybeUninit.html#method.into_initialized), which seems, uh, good enough? +I also like `assume_initialized` but that does not sound like it would return anything. +One thing I wonder about is whether some of these methods should be "downgraded" to functions, forcing callers to write `MaybeUninit::into_initialized` or so. +If you have an opinion on this, please [join us in the tracking issue](https://github.com/rust-lang/rust/issues/53491)! + +## Uninitialized Memory + +@stjepang, @Amanieu and me talked a bit about [uninitialized data in atomic operations](https://github.com/crossbeam-rs/crossbeam/issues/315), and how to make sense of the [C++ spec](https://en.cppreference.com/w/cpp/atomic/atomic/compare_exchange), in particular in the presence of mixed atomic and non-atomic accesses to the same object using [`atomic_ref`](https://en.cppreference.com/w/cpp/atomic/atomic_ref). +We did not make any notable progress, other than the possibility of a sound `AtomicCell` *without* `get_mut`. + +Another discussion was around using [`Read::read`](https://doc.rust-lang.org/nightly/std/io/trait.Read.html#tymethod.read) to initialize memory. +The problem here is that with an *unknown* `Read` implementation, we cannot just pass in an uninitialized buffer. +Uninitialized data is strictly different from any particular bit pattern, and using it in any non-trivial way is UB. +Since there is no way to know what the unknown `Read::read` function does, we cannot pass such bad data to it. +In [my terminology](({% post_url 2018-08-22-two-kinds-of-invariants%})), even though integers with uninitialized data are (likely) *valid*, uninitialized data violates the *safety invariant* of our integer types. +To solve this, the proposal is to add a [`freeze` method](https://github.com/rust-lang/rust/pull/58363) that turns all uninitialized memory into arbitrary but fixed bits. +Such frozen memory admits fewer optimizations (because it must observably have a consistent value when accessed multiple times), but it is also safe to use at any integer type (for the same reason).
+The main reason I like this proposal is that it officially acknowledges the subtle but important distinction between "arbitrary bit pattern" and "uninitialized", and that should help a lot with teaching these concepts to unsafe code authors. + +## Stacked Borrows + +Finally, there was some discussion about [Stacked Borrows](https://github.com/RalfJung/unsafe-code-guidelines/blob/stacked-borrows/wip/stacked-borrows.md) ([meeting notes here](https://paper.dropbox.com/doc/Topic-Stacked-borrows--AXUlwiiCbkUsrhfLWPy43~yiAg-2q57v4UM7cIkxCq9PQc22#:uid=304192866787878821395738&h2=NOTES)). +The feedback was very positive, and (to my surprise) everybody agreed that the things I had to change in the standard library to make it conform with the model were appropriate bugfixes. +@arielb1 and @matthewjesper helped a lot with detailed discussions about some of the [remaining issues](https://github.com/rust-rfcs/unsafe-code-guidelines/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-stacked-borrows) in the model, as well as figuring out a plan to fix a problem [found by @Amanieu](https://github.com/solson/miri/issues/615). +However, [two-phase borrows remain a problem](https://github.com/rust-lang/rust/issues/56254). + +## Miri now tests libcore and liballoc + +I am particularly happy about the progress I made on [Miri](https://github.com/solson/miri/) during this week. +With help from @eddyb and @alexcrichton, I got Miri to run the libcore and liballoc unit test suites. +This has already helped to [uncover a bug](https://github.com/rust-lang/rust/pull/58200) and some [subtle undocumented invariants](https://github.com/rust-lang/rust/issues/58320) in the standard library, as well as [several](https://github.com/solson/miri/pull/625) bugs [in Miri](https://github.com/rust-lang/rust/pull/58324). +I have [set things up](https://github.com/RalfJung/miri-test-libstd) such that Miri will run these tests every night, giving us higher confidence that the standard library is free of undefined behavior---or rather, that the parts of the standard library that are covered by unit tests are free of undefined behavior that Miri can detect. +I've been partially working towards this goal for several months now, so it is really satisfying to see it all come together. :-) + +Moreover, @Amanieu ran Miri on hashbrown, not discovering a bug in his crate but running into [several](https://github.com/solson/miri/issues/612) bugs [in Miri](https://github.com/solson/miri/issues/614) which I then fixed. + +And last but not least, Miri can now [pass arguments to the interpreted program](https://github.com/solson/miri/pull/624), which is particularly useful when running test suites to run only the test one is currently debugging. + +I think that's it! +Lots of exciting progress as well as lots of grounds for further discussion. +This won't get boring any time soon. :D