add RefinedRust paper

[web.git] / personal / _posts / 2017-07-14-undefined-behavior.md

diff --git a/personal/_posts/2017-07-14-undefined-behavior.md b/personal/_posts/2017-07-14-undefined-behavior.md
index ae5463e70cffd98b73a46cb54495a436893f26b8..26171ac992ea597aafd92d74ee38cd3b252f044e 100644 (file)
--- a/personal/_posts/2017-07-14-undefined-behavior.md
+++ b/personal/_posts/2017-07-14-undefined-behavior.md
@@ -8,7 +8,7 @@ So, finally, just one year later, this post is my take at what the purpose of th
 <!-- MORE -->
 Warning:  This post may contain opinions.  You have been warned.
 
 <!-- MORE -->
 Warning:  This post may contain opinions.  You have been warned.
 

-## When are optimizations legal?
+## When are Optimizations Legal?

 
 Currently, we have a pretty good understanding of what the intended behavior of *safe* Rust is.
 That is, there is general agreement (modulo some [bugs](https://github.com/rust-lang/rust/issues/27868)) about the order in which operations are to be performed, and about what each individual operation does.
 
 Currently, we have a pretty good understanding of what the intended behavior of *safe* Rust is.
 That is, there is general agreement (modulo some [bugs](https://github.com/rust-lang/rust/issues/27868)) about the order in which operations are to be performed, and about what each individual operation does.


@@ -37,9 +37,9 @@ After all, if there is any execution for which the assumption does *not* hold, t
 Now, it turns out that it is often really hard to obtain precise aliasing information.
 This could be the end of the game:  No alias information, no way to verify our assumptions, no optimizations.
 
 Now, it turns out that it is often really hard to obtain precise aliasing information.
 This could be the end of the game:  No alias information, no way to verify our assumptions, no optimizations.
 

-## Shifting responsibility
+## Shifting Responsibility

 
 

-However, it turns out that compilers writers consider these optimizations important enough that they came up with an alternative solution:
+However, it turns out that compiler writers consider these optimizations important enough that they came up with an alternative solution:

 Instead of having the compiler verify such assumptions, they declared the programmer responsible.
 
 For example, the C standard says that memory accesses have to happen with the right "effective type":  If data was stored with a `float` pointer, it must not be read with an `int` pointer.
 Instead of having the compiler verify such assumptions, they declared the programmer responsible.
 
 For example, the C standard says that memory accesses have to happen with the right "effective type":  If data was stored with a `float` pointer, it must not be read with an `int` pointer.


@@ -52,11 +52,18 @@ However, if they *do* alias, that would violate the effective type restriction,
 As we have seen, in both of the possible cases, the reordering is correct; the compiler is thus free to perform the transformation.
 
 Undefined behavior moves the burden of proving the correctness of this optimization from the compiler to the programmer.
 As we have seen, in both of the possible cases, the reordering is correct; the compiler is thus free to perform the transformation.
 
 Undefined behavior moves the burden of proving the correctness of this optimization from the compiler to the programmer.

+In the example above, what the "effective type" rule really means is that every single memory read of a `float` comes with a *proof obligation*:
+The programmer has to show that that the last write to this memory actually happened through a `float` pointer (baring some exceptions around union and character pointers).
+Similarly, the (in)famous rule that [signed integer overflow is undefined behavior](https://stackoverflow.com/questions/16188263/is-signed-integer-overflow-still-undefined-behavior-in-c) means that every single arithmetic operation on signed integers comes with the proof obligation that this operation will never, ever, overflow.
+The compiler performs its optimization under the assumption that the programmer actually went through the effort and convinced itself that this is the case.
+

 Considering that the compiler can only be so smart, this is a great way to justify optimizations that would otherwise be difficult or impossible to perform.
 Unfortunately, it is often not easy to say whether a program has undefined behavior or not -- after all, such an analysis being difficult is the entire reason compilers have to rely on UB to perform their optimizations.
 Furthermore, while C compilers are happy to exploit the fact that a particular program *has* UB, they do not provide a way to test that executing a program *does not* trigger UB.
 Considering that the compiler can only be so smart, this is a great way to justify optimizations that would otherwise be difficult or impossible to perform.
 Unfortunately, it is often not easy to say whether a program has undefined behavior or not -- after all, such an analysis being difficult is the entire reason compilers have to rely on UB to perform their optimizations.
 Furthermore, while C compilers are happy to exploit the fact that a particular program *has* UB, they do not provide a way to test that executing a program *does not* trigger UB.

-It also turns out that programmers' intuition often [does not match what the compiler does](https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-discussion.html), which leads to miscompilations (in the eye of the programmer) and sometimes to security [vulerabilities](https://lwn.net/Articles/342330/).
+It also turns out that programmers' intuition often [does not match what the compiler does](https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-discussion.html), which leads to miscompilations (in the eye of the programmer) and sometimes to security [vulnerabilities](https://lwn.net/Articles/342330/).

 As a consequence, UB has a pretty bad reputation.
 As a consequence, UB has a pretty bad reputation.

+(The fact that most people will not expect an innocent-looking `+` operation to come with subtle proof obligations concerning overflow probably also plays a role in this.
+In other words, this is also an API design problem.)

 
 There are various sanitizers that watch a program while it is being executed and try to detect UB, but they are not able to catch all possible sources of UB.
 Part of the reason this is so hard is that the standard has not been written with such sanitizers in mind.
 
 There are various sanitizers that watch a program while it is being executed and try to detect UB, but they are not able to catch all possible sources of UB.
 Part of the reason this is so hard is that the standard has not been written with such sanitizers in mind.


@@ -67,7 +74,7 @@ That is not very satisfying.
 ## Undefined Behavior in Rust
 
 Coming back to Rust, where are we at?
 ## Undefined Behavior in Rust
 
 Coming back to Rust, where are we at?

-Safe Rust is [free from UB]({{ site.baseurl }}{% post_url 2017-07-08-rustbelt %}), but we still have to worry about unsafe Rust.
+Safe Rust is [free from UB]({% post_url 2017-07-08-rustbelt %}), but we still have to worry about unsafe Rust.

 For example, what if unsafe code crafts two aliasing mutable references (something that is prevented in safe Rust) and passes them to our `simple` function?
 This violates the assumptions we made when we reordered the two writes.
 If we want to permit this optimization (which we do!), we have to argue why it cannot change program behavior.
 For example, what if unsafe code crafts two aliasing mutable references (something that is prevented in safe Rust) and passes them to our `simple` function?
 This violates the assumptions we made when we reordered the two writes.
 If we want to permit this optimization (which we do!), we have to argue why it cannot change program behavior.


@@ -84,7 +91,11 @@ I also think that tooling to *detect* UB is of paramount importance, and can hel
 To this end, the specification should be written in a way that such tooling is feasible.
 In fact, specifying a dynamic UB checker is a very good way to specify UB!
 Such a specification would describe the additional state that is needed at run-time to then *check* at every operation whether we are running into UB.
 To this end, the specification should be written in a way that such tooling is feasible.
 In fact, specifying a dynamic UB checker is a very good way to specify UB!
 Such a specification would describe the additional state that is needed at run-time to then *check* at every operation whether we are running into UB.

-It is with such considerations in my mind that I have previously written about [miri as an executable specification]({{ site.baseurl }}{% post_url 2017-06-06-MIR-semantics %}).
+It is with such considerations in my mind that I have previously written about [miri as an executable specification]({% post_url 2017-06-06-MIR-semantics %}).
+
+Coming up next on this channel:  During my [internship]({% post_url 2017-05-23-internship-starting %}), I am working on such a specification.
+My ideas are concrete enough now that I can write down a draft, which I will share with the world to see what the world thinks about it.
+
+**Update:** [Writing down has happened]({% post_url 2017-07-17-types-as-contracts %}).

 
 

-Coming up next on this channel:  During my [internship]({{ site.baseurl }}{% post_url 2017-05-23-internship-starting %}), I am working on such a specification.
-I have a draft ready now, and I want to share it with the world to see what the world thinks about it.
+**Update:** Clarified "Shifting Responsibility".