research page moved to new location

[web.git] / ralf / _posts / 2019-11-25-how-to-panic-in-rust.md

diff --git a/ralf/_posts/2019-11-25-how-to-panic-in-rust.md b/ralf/_posts/2019-11-25-how-to-panic-in-rust.md
index 0a9d2ba21b7bba82c4ba7cf740acde09360788a6..f840aa65ece45539d8284c5e53ada40db5ac5be7 100644 (file)
--- a/ralf/_posts/2019-11-25-how-to-panic-in-rust.md
+++ b/ralf/_posts/2019-11-25-how-to-panic-in-rust.md
@@ -18,6 +18,9 @@ There [are still some rough edges](https://github.com/rust-lang/miri/issues?q=is
 The purpose of this post is to document the high-level structure and the relevant interfaces that come into play on the Rust side of this.
 The actual mechanism of unwinding is a totally different matter (and one that I am not qualified to speak about).
 
 The purpose of this post is to document the high-level structure and the relevant interfaces that come into play on the Rust side of this.
 The actual mechanism of unwinding is a totally different matter (and one that I am not qualified to speak about).
 

+*Note:* This post describes panicking as of [this commit](https://github.com/rust-lang/rust/commit/4007d4ef26eab44bdabc2b7574d032152264d3ad).
+Many of the interfaces described here are unstable internal details of libstd, and subject to change any time.
+

 ## High-level structure
 
 When trying to figure out how panicking works by reading the code in libstd, one can easily get lost in the maze.
 ## High-level structure
 
 When trying to figure out how panicking works by reading the code in libstd, one can easily get lost in the maze.


@@ -57,21 +60,21 @@ Let us now look at these pieces in a bit more detail.
 
 ## Panic Runtime
 
 
 ## Panic Runtime
 

-The [interface to the panic runtime](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L48) (introduced by [this RFC](https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md)) is a function `__rust_start_panic(payload: usize) -> u32` that gets imported by libstd, and that is later resolved by the linker.
+The [interface to the panic runtime](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L53) (introduced by [this RFC](https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md)) is a function `__rust_start_panic(payload: usize) -> u32` that gets imported by libstd, and that is later resolved by the linker.

 
 The `usize` argument here actually is a `*mut &mut dyn core::panic::BoxMeUp` -- this is where the "payload" of the panic (the information available when it gets caught) gets passed in.
 
 The `usize` argument here actually is a `*mut &mut dyn core::panic::BoxMeUp` -- this is where the "payload" of the panic (the information available when it gets caught) gets passed in.

-`BoxMeUp` is an unstable internal implementation detail, but [looking at the trait](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panic.rs#L268-L271) we can see that all it really does is wrap a `dyn Any + Send`, which is the [type of the panic payload](https://doc.rust-lang.org/std/thread/type.Result.html) as returned by `catch_unwind` and `thread::spawn`.
-`BoxMeUp::box_me_up` returns a `Box<dyn Any + Send>`, but as a raw pointer (because `Box` is not available in the context where this trait is defined); `BoxMeUp::get` just borrows the contents.
+`BoxMeUp` is an unstable internal implementation detail, but [looking at the trait](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L268-L281) we can see that all it really does is wrap a `dyn Any + Send`, which is the [type of the panic payload](https://doc.rust-lang.org/std/thread/type.Result.html) as returned by `catch_unwind` and `thread::spawn`.
+`BoxMeUp::take_box` returns a `Box<dyn Any + Send>`, but as a raw pointer (because `Box` is not available in the context where this trait is defined); `BoxMeUp::get` just borrows the contents.

 
 

-The two implementations of this interface Rust ships with are [`libpanic_unwind`](https://github.com/rust-lang/rust/tree/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libpanic_unwind) for `-C panic=unwind` (the default on most platforms) and [`libpanic_abort`](https://github.com/rust-lang/rust/tree/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libpanic_abort) for `-C panic=abort`.
+The two implementations of this interface Rust ships with are [`libpanic_unwind`](https://github.com/rust-lang/rust/tree/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libpanic_unwind) for `-C panic=unwind` (the default on most platforms) and [`libpanic_abort`](https://github.com/rust-lang/rust/tree/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libpanic_abort) for `-C panic=abort`.

 
 ## `std::panic!`
 
 
 ## `std::panic!`
 

-On top of the panic *runtime* interface, libstd implements the default Rust panic machinery in the internal [`std::panicking`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs) module.
+On top of the panic *runtime* interface, libstd implements the default Rust panic machinery in the internal [`std::panicking`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs) module.

 
 #### `rust_panic_with_hook`
 
 
 #### `rust_panic_with_hook`
 

-The key function that everything passes through is [`rust_panic_with_hook`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L435-L437):
+The key function that almost everything passes through is [`rust_panic_with_hook`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L439-L441):

 {% highlight rust %}
 fn rust_panic_with_hook(
     payload: &mut dyn BoxMeUp,
 {% highlight rust %}
 fn rust_panic_with_hook(
     payload: &mut dyn BoxMeUp,


@@ -86,7 +89,7 @@ Panic hooks have a [`PanicInfo`](https://doc.rust-lang.org/core/panic/struct.Pan
 This matches `rust_panic_with_hook`'s arguments quite nicely!
 `file_line_col` and `message` can be used directly for the first two elements; `payload` gets turned into `&(dyn Any + Send)` through the `BoxMeUp` interface.
 
 This matches `rust_panic_with_hook`'s arguments quite nicely!
 `file_line_col` and `message` can be used directly for the first two elements; `payload` gets turned into `&(dyn Any + Send)` through the `BoxMeUp` interface.
 

-Interestingly, the *default* panic hook entirely ignores the `message`; what you actually see printed is [the `payload` downcast to `&str` or `String`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L171-L177) (whatever works).
+Interestingly, the *default* panic hook entirely ignores the `message`; what you actually see printed is [the `payload` downcast to `&str` or `String`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L176-L182) (whatever works).

 Supposedly, the caller should ensure that formatting `message`, if present, gives the same result.
 (And the ones we discuss below do ensure this.)
 
 Supposedly, the caller should ensure that formatting `message`, if present, gives the same result.
 (And the ones we discuss below do ensure this.)
 


@@ -96,54 +99,59 @@ The `'static` bound is quite well hidden there, but after a while I realized tha
 
 #### libstd panicking entry points
 
 
 #### libstd panicking entry points
 

-`rust_panic_with_hook` is a private function to `std::panicking`; the module provides three separate entry points on top of this central function:
+`rust_panic_with_hook` is a private function to `std::panicking`; the module provides three entry points on top of this central function, and one that circumvents it:

 
 

-* the [default panic handler implementation](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L301), backing (as we will see) panics from `core::panic!` and built-in panics (from arithmetic overflow or out-of-bounds array/slice indexing).
+* the [`begin_panic_handler`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L332), the default panic handler implementation that backs (as we will see) panics from `core::panic!` and built-in panics (from arithmetic overflow or out-of-bounds array/slice indexing).

   This obtains as input a [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html), and it has to turn that into arguments for `rust_panic_with_hook`.
   Curiously, even though the components of `PanicInfo` and the arguments of `rust_panic_with_hook` match up pretty well and seem like they could just be forwarded, that is *not* what happens.
   This obtains as input a [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html), and it has to turn that into arguments for `rust_panic_with_hook`.
   Curiously, even though the components of `PanicInfo` and the arguments of `rust_panic_with_hook` match up pretty well and seem like they could just be forwarded, that is *not* what happens.

-  Instead, libstd entirely *ignores* the `payload` component of the `PanicInfo`, and sets up the actual payload (passed to `rust_panic_with_hook`) such that it contains [the formatted `message`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L348).
+  Instead, libstd entirely *ignores* the `payload` component of the `PanicInfo`, and sets up the actual payload (passed to `rust_panic_with_hook`) such that it contains [the formatted `message`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L350).

 
     In particular, this means that the panic *runtime* is irrelevant for `no_std` applications.
   It only comes into play when libstd's panic handler implementation is used.
   (The panic *strategy* selected via `-C panic` still has an effect as it also influences code generation.
   For example, with `-C panic=abort` code can become simpler as it does not need to support unwinding.)
 
 
     In particular, this means that the panic *runtime* is irrelevant for `no_std` applications.
   It only comes into play when libstd's panic handler implementation is used.
   (The panic *strategy* selected via `-C panic` still has an effect as it also influences code generation.
   For example, with `-C panic=abort` code can become simpler as it does not need to support unwinding.)
 

-* [`begin_panic_fmt`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L319), backing the [format string version of `std::panic!`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/macros.rs#L22-L23) (i.e., this is used when you pass multiple arguments to the macro).
-  This basically just packages the format string arguments into a `PanicInfo` (with a [dummy payload](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panic.rs#L56)) and calls the default panic handler that we just discussed.
+* [`begin_panic_fmt`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L316), backing the [format string version of `std::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L22-L23) (i.e., this is used when you pass multiple arguments to the macro).
+  This basically just packages the format string arguments into a `PanicInfo` (with a [dummy payload](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L56)) and calls the default panic handler that we just discussed.

 
 

-* [`begin_panic`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L388), backing the [single-argument version of `std::panic!`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/macros.rs#L16).
+* [`begin_panic`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L392), backing the [single-argument version of `std::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L16).

   Interestingly, this uses a very different code path than the other two entry points!
   In particular, this is the only entry point that permits passing in an *arbitrary payload*.
   Interestingly, this uses a very different code path than the other two entry points!
   In particular, this is the only entry point that permits passing in an *arbitrary payload*.

-  That payload is just [converted into a `Box<dyn Any + Send>`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/panicking.rs#L415) so that it can be passed to `rust_panic_with_hook`, and that's it.
+  That payload is just [converted into a `Box<dyn Any + Send>`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L419) so that it can be passed to `rust_panic_with_hook`, and that's it.

   
     In particular, a panic hook that looks at the `message` field of the `PanicData` it is passed will *not* be able to see the message in a `std::panic!("do panic")`, but it *will* see the message in a `std::panic!("panic with data: {}", data)` as the latter passes through `begin_panic_fmt` instead.
   That seems quite surprising. (But also note that `PanicData::message()` is not stable yet.)
 
   
     In particular, a panic hook that looks at the `message` field of the `PanicData` it is passed will *not* be able to see the message in a `std::panic!("do panic")`, but it *will* see the message in a `std::panic!("panic with data: {}", data)` as the latter passes through `begin_panic_fmt` instead.
   That seems quite surprising. (But also note that `PanicData::message()` is not stable yet.)
 

+* [`rust_panic_without_hook`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L496) is the odd one out: this entry point backs [`resume_unwind`](https://doc.rust-lang.org/nightly/std/panic/fn.resume_unwind.html), and it actually does *not* call the panic hook.
+  Instead, it dispatches to the panic runtime immediately.
+  Like, `begin_panic`, it lets the caller pick an arbitrary payload.
+  Unlike `begin_panic`, the caller is responsible for boxing and unsizing the payload; `update_count_then_panic` just forwards that pretty much verbatim to the panic runtime.
+

 ## Panic Handler
 
 All of the `std::panic!` machinery is really useful, but it relies on heap allocations through `Box` which is not always available.
 To give libcore a way to cause panics, [panic handlers were introduced](https://github.com/rust-lang/rfcs/blob/master/text/2070-panic-implementation.md).
 As we have seen, if libstd is available, it provides an implementation of that interface to wire `core::panic!` into the libstd panic machinery.
 
 ## Panic Handler
 
 All of the `std::panic!` machinery is really useful, but it relies on heap allocations through `Box` which is not always available.
 To give libcore a way to cause panics, [panic handlers were introduced](https://github.com/rust-lang/rfcs/blob/master/text/2070-panic-implementation.md).
 As we have seen, if libstd is available, it provides an implementation of that interface to wire `core::panic!` into the libstd panic machinery.
 

-The [interface to the panic handler](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panicking.rs#L78) is a function `fn panic(info: &core::panic::PanicInfo) -> !` that libcore imports and that is later resolved by the linker.
+The [interface to the panic handler](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L78) is a function `fn panic(info: &core::panic::PanicInfo) -> !` that libcore imports and that is later resolved by the linker.

 The [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html) type is the same as for panic hooks: it contains a panic source location, a panic message, and a payload (a `dyn Any + Send`).
 The panic message is represented as [`fmt::Arguments`](https://doc.rust-lang.org/std/fmt/struct.Arguments.html), i.e., a format string with its arguments that has not been formatted yet.
 
 ## `core::panic!`
 
 The [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html) type is the same as for panic hooks: it contains a panic source location, a panic message, and a payload (a `dyn Any + Send`).
 The panic message is represented as [`fmt::Arguments`](https://doc.rust-lang.org/std/fmt/struct.Arguments.html), i.e., a format string with its arguments that has not been formatted yet.
 
 ## `core::panic!`
 

-On top of the panic handler interface, libcore provides a [minimal panic API](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panicking.rs).
-The [`core::panic!`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/macros/mod.rs#L10-L26) macro creates a `fmt::Arguments` which is then [passed to the panic handler](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panicking.rs#L82).
+On top of the panic handler interface, libcore provides a [minimal panic API](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs).
+The [`core::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L10-L26) macro creates a `fmt::Arguments` which is then [passed to the panic handler](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L82).

 No formatting happens here as that would require heap allocations; this is why `PanicInfo` contains an "uninterpreted" format string with its arguments.
 
 No formatting happens here as that would require heap allocations; this is why `PanicInfo` contains an "uninterpreted" format string with its arguments.
 

-Curiously, the `payload` field of the `PanicInfo` that gets passed to the panic handler is always set to [a dummy value](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panic.rs#L56).
+Curiously, the `payload` field of the `PanicInfo` that gets passed to the panic handler is always set to [a dummy value](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L56).

 This explains why the libstd panic handler ignores the payload (and instead constructs a new payload from the `message`), but that makes me wonder why that field is part of the panic handler API in the first place.
 This explains why the libstd panic handler ignores the payload (and instead constructs a new payload from the `message`), but that makes me wonder why that field is part of the panic handler API in the first place.

-Another consequence of this is that [`core::panic!("message")`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/macros/mod.rs#L15) and [`std::panic!("message")`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libstd/macros.rs#L16) (the variants without any formatting) actually result in very different panics: the former gets turned into `fmt::Arguments`, passed through the panic handler interface, and then libstd creates a `String` payload by formatting it.
+Another consequence of this is that [`core::panic!("message")`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L15) and [`std::panic!("message")`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L16) (the variants without any formatting) actually result in very different panics: the former gets turned into `fmt::Arguments`, passed through the panic handler interface, and then libstd creates a `String` payload by formatting it.

 The latter, however, directly uses the `&str` as a payload, and the `message` field remains `None` (as already mentioned).
 
 Some elements of the libcore panic API are lang items because the compiler inserts calls to these functions during code generation:
 The latter, however, directly uses the `&str` as a payload, and the `message` field remains `None` (as already mentioned).
 
 Some elements of the libcore panic API are lang items because the compiler inserts calls to these functions during code generation:

-* The [`panic` lang item](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panicking.rs#L39) is called when the compiler needs to raise a panic that does not require any formatting (such as arithmetic overflow); this is the same function that also backs single-argument [`core::panic!`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/macros/mod.rs#L15).
-* The [`panic_bounds_check`](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/panicking.rs#L55) lang item is called on a failed array/slice bounds check.
-  It calls into the same method as [`core::panic!` with formatting](https://github.com/rust-lang/rust/blob/7d761fe0462ba0f671a237d0bb35e3579b8ba0e8/src/libcore/macros/mod.rs#L21-L24).
+* The [`panic` lang item](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L39) is called when the compiler needs to raise a panic that does not require any formatting (such as arithmetic overflow); this is the same function that also backs single-argument [`core::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L15).
+* The [`panic_bounds_check`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L55) lang item is called on a failed array/slice bounds check.
+  It calls into the same method as [`core::panic!` with formatting](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L21-L24).

 
 ## Conclusion
 
 
 ## Conclusion