--- /dev/null
+RedirectMatch "^/cs/(index.html?)?$" "/research"
+RedirectMatch "^/cs/bachelor(.*)$" "/research/bachelor/bachelor$1"
--- /dev/null
+-----BEGIN PGP PUBLIC KEY BLOCK-----\r
+\r
+xsFNBFNAJO0BEAC8Rr0Q4AC4X2l0a09GQSCIib9Htj5N+21vHZ3EYAG0gQ1QAOX3\r
+QHrtu26Dn1nvpQ8hhsp86YHK7pD4IXPesdJjbEmF0glidhPMJJ6pq8vQvMywg7sD\r
+X1XOhDKP/ejSOXmS6G5cgMGrYad2ys5eB9Q18Bj310zwZTeDTPwxM5GqIg7/cX66\r
+LCITZOi88ZNOJgysuCLioCU2j4bBgJbrmClwi5IipfclsWZwACmgEGHc/STcEyrT\r
+AolLdtW3L3Ghaz4gZRGqV606plWewdFIqkVgJpDOWou9yIDoeSpKc2sHY2PL3xoz\r
+GfftBj+gJkwsBovHzxZUbU4oTETVGpPmyExiIL79ickHwqdDBZV9PzyHioK7AhMQ\r
+ys0K1yHn0x/pTMWrOFLQxJ5tPNWstTXcAYIbJMJaAKTmtN0d4sj4/U8Q9S2D+ItF\r
+EyZOUyL480xHzX4STTF1ibiDm9tDvpZEsFHwbb7yA7nfXg4F82FIVI1R16n3IqdK\r
+ROFsToE0MM2FTyApFIpVx2KGdbtRsCOty07TWrenRDMMCj2YJcBkCOy2u1loULEm\r
+/n8cyiyhxEE4EkzIsChngB3+wgfQQCLAUEjbwUyWztvvo8EzvmZDVDKGBPHhQtx8\r
+EOL/Qpa0zVCU5eJ+JeSc2rMRhFP1+tL9GWu7Gxmfb8gv0yBhtJIXTcT5awARAQAB\r
+zRlSYWxmIEp1bmcgPHBvc3RAcmFsZmouZGU+wsGTBBMBCgA9AhsDBAsJCAcFFQoJ\r
+CAsFFgIDAQACHgECF4AWIQS9P1r22V13xIqFr2gMsY5SGyTz/wUCYNBuWAUJD2rl\r
+agAKCRAMsY5SGyTz/7PJD/0Qol0yc/O/tX/lBsB/TGnPJa8t59ChgYvDayyjZEet\r
+nPAuONBaElp9VxVfmcNOzuV6kuJTwZj6Yuq77XlSuslOcBkoPiXCZqlg3ELhF2E9\r
+3lVXwCZ7xeA73hoaCwwV9JXYI4H8MxiFQzMkwX0uaB24GNMKPayMivKfRYLKgt0T\r
+M6wdC5IF3qiA8OJF3HtEznojvYQ8kzG0yV9A543wr0Pw6A3kGZdRQ2yLy7CoPQsH\r
+gz/T8RvtMUVhp6oBGSk2zQlzRnJ4ln9276LpuVB5UIyVkF+QgRS5YVc7U09N9vhT\r
+cv+AVmYf8g736YHPREE1BxMgMy96Z46HGyDn9cOXpmdPg018lXdg+Di5LUlzQPzS\r
+6BgmIV7ikX3r5UEWK7cUvyW5tq4SO/b0acEqJdkKpxpccmEbvRJgwOyhQLdNTlAe\r
+FAQ+lUHEhNNl5zFXnj+U5n+k+3eKQomAX9Baav5Zv04OL9CkvNZ7+0i1hU62jrlp\r
+VwdQ0dXTYl2YRAKseB4ik71qLbZ+8J1Ld9njfB8PWaNK1979rCPdENZtRIz8aML5\r
+CWOd2nT7nYkTtYqPJ03SLzKSYM+TxwJPu0//OLdbD2LZ8FIiAxt0li/2cAuEvsoD\r
+mM2ANcXV0KajIpsMnruZKY5oBfklxFz3cB1yg6gJX+nyoEBDSfuOujTRT7A50JgB\r
+9MLAXAQTAQoABgUCU0EXlQAKCRBAHU2dJowdVidwCAC9mBsnmkxJPjk0RB53WIzW\r
+7+t8XCKjpIx0cSlsjdE3j49cdsMllz0S9ffNrtqsS/qjdtyXb5t2rXj7G9EoSpxE\r
+ygds6d2JZh6nJZ4jII/FCtcJ1av7tNoC3RS0dU3vr7LFHLNDCdvW28haZuoxMT7E\r
+paE14moTvMriSIAQ5LKzDqvQfDT1IInMVj4s5FpDWsH+YoWU5DazeCTk686YFtyw\r
+3emvgO3VUiDdKGfwrfhYf2Lp/t0+Ot2f0/kZJZrqWKaqkOEdM505BP8bmsX4K7xW\r
+4TCTkp6szJhifXv7i/kz+U0BrKrfpLhyIZ1H2jHdZoUfD3Z+myLK9RDWlVRnNxNX\r
+wsDcBBABAgAGBQJTQVfnAAoJEKAmVAtPJmh7yN8MAIl7yLmKOu2mUpnGl0EtH1K+\r
+sIrjuU1UnLdVR5v0UzKqxX8dUCl4pv5/c49ZsZ+pfB1E48wGM+eXP3eLci34fsUM\r
+2QO1Ue6rxJx9RYYC9eZrKL7+QETFiNhVARlA2ZfTAStOcKoSWUs8gHFZXCT/+gHh\r
+V5ONS82L5LTFLJgGHBSh90B24e+pFQEhuuWT3QqyszbijfvsM0J9oILs0Bqc3lmU\r
+Z3sJudJz3yGSFzjCbLcv6SBRrhjTfbwoH0Gb3RhIFfThLX+e+euBoDWHmr0Qk4gC\r
+6a+XqzEER9bF8fajFXdBaSMFW600BKHD1LCU0ihqtZJ+cSOoG+N6rDj1Ug8Kisnd\r
+Dwvzz0td/GrjA0oM6kUIYolUFcwj73zT4T56Xc+4DF5U0UVXCIujhH0d0LW4tAHG\r
+eeu687DrGm/I6eyj8pKJEVj+fYyKZghnfceBxS7PO68w/e54irzxkC9a7NIq22uQ\r
+FAfaM6vcFEGtIRaulAhyeISBfzvJ+Bu/DVs3AcX+KMLAXAQTAQIABgUCU0KCEwAK\r
+CRCfM0zgXZjKAHmHB/0fprnD3OOV7MjBGMd+9j0JX9a4Ghsg7wa9GId7T4vu5HEE\r
+7nCmANl6LIjyaVPoa7PKsWb1vvXZA5sh24keNRkbvbIeecyXIYC0XuvCvBw286T7\r
+yS1vB5+cPnyfpTPq8/JQ102L6GblvZl4nruwYMUBq/nNJ/L7nrDpvQWfGR17mGXZ\r
+EIRcuOrMPag/WFQCBsqIwYhn0lfQ1OBZv/3kFj+rueV2hWO4PScL3VOlYRJckN9s\r
+xTfO9W3ShT1gVTyBtEjke+cjiZOFxhumdCddL4vrjxHUWFqjQ7Ng2QM5mJpbJv7Y\r
+Ameyp0wGe/UgTsNeMIYsFcYk1lJLqoVl06E0ZZziwsBiBBMBCgAMBQJTcQiaBYMH\r
+hh+AAAoJEIXdISKfK5ylDQQIALSxDss/hRtVRUqjh1yz17UScXNWCAUm04jn3g7u\r
+hpOjM5WX3v7e0D4IT2GvTBE4rgVJfWJkT8VmuEHOsvE8lavPM6dDLO4/ByDKzjO/\r
+iazUYRRfCqonqJ4dJnELhNRHoQ4xYqb3gOYBgRLRO8TIBrYVi/m2LuIZVLMC72te\r
+wVSu3Fg7YKJ/8Z8Z6S/PNLTi4uACwAh0ldNA1/noAyQYm020e3nHPqNur5cydD+d\r
+rHeh9LktJ5904hDhNDjgwRojfNHFqWGf4n38l2hVvmtYfYvioeSZDs5aLxTCvYq8\r
+4wzu2dhLggtbzL9+59FeSCp/TedSxGJ71CWb5DIo/ZkMvi7CwVwEEAECAAYFAlNx\r
+FO4ACgkQNzACNyZ5rjS3OQ/6AkDESs2YHnvxEV2gVaEwgwdz3rGUaFCEkTFa46I0\r
+DSVCEnTSlNuY8QX2R9w1O8YlaYKI9NtBcIodHMb8d3qIi/fg89qFIBLDaIp8hYO5\r
+Gdujyx6l30OoVB0SPcIQ0KSujfdqavS5sB2iCzRzDlWPzw4d7NLEI7EOtlJ/xjH0\r
+n6yXVP+pSkB6Mcn4eo/XgqbJtNXItllmXprzdg45dOGSjf/ICvFT/YM4fJmbYXDj\r
+qqxF8tP+4FxMVOyyTWjCU7Sfs8tbuHtjWpmAj+k5cnPbBf88yghwCOyAtdY4ApiF\r
+tN+0At5WzzwNACXVGS859Mcgf2bWiFgvkfjMqRYFo/LWMm0OzpX6SdbgsPS4xDhZ\r
+Wcsh3cFhheVv/JyLQveaLHPvw0it/5F1tf91yTH+Dqjwjo6sKJIqfiqoT/FJvWlQ\r
+Y7POwY0bWqEO/hq8gOLLmZgdh0EWliphbrrLTYENBXaOyy94Q0yax/AYFIoMHAvU\r
+QBsSaGp7aVoU8EDAsQ9FIXp5yczPyOlf0In6BZ8F1SdUm+NocIbSJWfQm0UKY562\r
+ZfCEtGFDaom4Zn1Waoy+/6LR39BMYc5MNOUnAtImxkravE5WX9KWZvXFMOaabr6U\r
+eDV35Dt717BQVb9+aO6RfsIHS5QPdsVLzwiDQUsjOZc/+3dTA4FBALNMXc1tLP0F\r
+oirCwVwEEwECAAYFAlNxG+QACgkQ5sRt3erDGOkVZg/+JjKYOTiPCczXCQdehiKN\r
+S7BVhmucp37t+JN7h2P5V4wo4Ih50i47AK8em+b6Lc5tBMeFo5rPqzYQXvN6CqNk\r
+Yhjjx5Z8b0Gg2NJFnezmMd3nsZibj+HXZIaaveHFyqdY/uiGIoNX6IV1ytFYvV0c\r
+rFPeNe1E0o1spXAkTrQQxxfRi89/TfGz93f80noBZLYP0bEvijvZxpkYC7LLUO6r\r
+6fxQp3h6KDIgAVGchSkYnmADaU39LyNMhXyvND+lLcpk4AhCLsWDj5VxqLXN7ed2\r
+uf5R5aBOkGOtc9pT52bEqLjbZdNxp8yZQDQumZzKXpSEgUv6D5QcrkF2kPPhlMVv\r
+nfJ/x/eTp/+s6BuhJoKo5p6Ir83GG5nDkKW93yUJP7y4xpSX/2+KDOeRyM5ClXRa\r
+OiXuL1/xLfLzCx9NJzxULAcKcoHUzWfGHLVnFb5VNznDd+cThWbONtQb2d3X9rLa\r
+FwJuAnWdN5ckYjO1FBZOA3vL9rKUeh59YAACWBdUKv1FDWqT9lvB+KVey10oju27\r
+BrKiKM/UCIlnWEUsRQxbYmRTUw8E3mxq7NP1yAtr1ftPmiPwtB9F95n4qI2BJCCn\r
+F/D0zOAVPkQF1hogTDoN/4plTSeh9qvQimnS2JbFldiSpuYLT7U070WIg4a3/Bj+\r
+4beeV6eqkfmvKKqS6OVbXW3CwVwEEAEIAAYFAlOMME4ACgkQx/+TixwaPknwqg//\r
+SpB/+PEa5jCNNljEo71Ebzu3htKvwS2PfVv+FfDQCWdZAdTVd22wwNmVIAeKmXMv\r
+Ev3qTRFwRmZVCPLH3caRwLtlxu4ALhKFA1ipV+EUy/fpqZeuHh+6EX7AlgWxEqkl\r
+o1jh3MbGhC/Uv+I6l0TchLd+VwlH44HnZBmSt1TUotNlc+yhcbPXcL8MzT0s63jR\r
+JEV8VGopscazWEGg+66b4zfbPbIicgOSh+ijTg9E7TnV5yacJvLE057WFwmJONtK\r
+7/6kd7iAWPn1mAuVhoQGcqWjEm28t/jfTv3F5J+8CqjgkecqdAlr6PZTrv+Ot6kx\r
+rbXCL3Z3/NTWw4rTB7ZSyZsv3IofdjAsY6uX+KcJvRcwWySJ9bKQNULeItcYyOT/\r
+Qye2YSK4ok0hk0n/bulvZVDYy44C4JW35IOkXP3ZPwx+fGbQS2QMnKHnoCn6cCmA\r
+YAnHpmt5dfGYXc/4S4Lkd6qWHhI0AA5AbdU/orcQMG2DVPMM8vsIue9z5XQrAdep\r
+vsMteH3dNFn5jauwKDFSHveB84AEt8BzSMNPvsobeWIQXWmBxQoNqm8J0Fz1GVgi\r
+HCO10D489lP5UdJ3/HLiqrhMXcHIjVqRrHA32mQ8SWd7SRExUMMJW6vYwAF6O/H5\r
+5e+kJwIepyVTdAZsQ4gABfLFGLPDv4Opo92YOnKMwlXCRgQQEQgABgUCU4wwdgAK\r
+CRA+oPhreU+dfG+bAJ9VpiS+jRNYAYfYo0WNgyOkcIt2OACdEhtAj3fel5bv8P+/\r
+XS2PbFAanvzCwGIEEwEKAAwFAlNxIYsFgweGH4AACgkQY/sWiK28b0HGWQf7BI9z\r
+hlnkRKqYIV+oUvpPyJsJtZMQmROr6R1CTiLW4H7T7jwJ1ViwtHxDWGIZk2mIZ0Lv\r
+myRPxsI96uDP0bYsqQfdD7dcUX6LdQNMQ4Ic20SeKT45vU3MF7VbWvxsav5M74KP\r
+a7fiLqODk2y9QNSf2mx1wgUHGhKT3v3oQu6yVsGjX1XkAgroTGR2ayHT/w9mr0TC\r
+VUk55HCCRoZ+GVJyTAWI0Ko4RJabz7xLr6yucxawUEBjaPpzkLKFKh0mCf1nGTKU\r
+G1hg2eiSBLJM4Yp0M7kwlBcs+seCKBYSHzByzrhW2mzg7pxC9RGgCBinIY47pRrT\r
+sqvNERmSvD8nfwc0SsLBXAQQAQIABgUCU3EZ7QAKCRAKRWlQy1u1p/vQEACakB6d\r
+ygFgUHlPzEI+m7rxYqJJhTUrjwh3kl4eh7oqsIkRX9JLnUAYUMQlsOyo9dtSrD6r\r
+6ZIWVO5xpgeHpWmDzWUVm9aGLqujoOWZgo8BRv2wkqUHhNuntoejGmhCANR+QPkU\r
+l7L+ViGP4JoE94YSzLvK1ThPley65PsJUs0HY1kHiilhHM94RD6ZdQeklB9+EXC/\r
+K5tk/JTtR9v7uT2KGbopLeKuW7XPVApF+W34JZdvZ9/VIjKh1NeKFbLRIzloZ4MB\r
+07wImlCxoKZ5kl5OtdnTQeSzTuNBEmwccJS7t3canGO+tfZQhJbfDQcvpvyCtqpR\r
+2gOlVhXuhYicxqQUrVIXE03VlteEPQXIHJJHtXLiMQuc5G3pgLSthAIub+HsW7kk\r
+ZZHaxZ4FNX+XnyaMspAtVqlorl2XFXdFXysK/CMmyV7G//l+4P8711zav1pz0uJN\r
+4BeVPO4pYZTKBiT9QMJX2TSjWcXV38YTZ6hFr64TBJxPIx+69Gb35WlqOAQPkw+x\r
+uky839okSwQnw/34brreZ6aEI7gVuBhagNF8dmW/ExvJNNTSqSEy71OOjlNbaQNv\r
+sCUndr2+EoTV/OreSvL550flVd2jDuG3N7UOOQe63EZBYVipPPGKDI+jHGfVYB11\r
+HN6dcsimfFUgq6vdwh8q0WsP32kVZKmlz6v12sLBXAQQAQgABgUCU47K5gAKCRD2\r
+lH2raOe5MY4JEADBh3cw9nAMXgqPux+9QHci44IddLfpL8J9yswGdtBI/UvJjSWW\r
+6/KQjb+uPlBJSmbhGo7jMud/X5fiuIEFeZhONQZqyC0Vc/AIL9FX8EPViRoEDuCH\r
+HAmTjIjeQk7vL8vzdp8JM6kEtUhvU2CdMQqobPoolbr2KfOjocJbTp1x/LoE3Djc\r
+Oi5DIFFTre5c89NSh4g4rKzAghKfgdl58kyEXdIa7nP4Zru/0App4Zfkg3H13M3w\r
+TVNliWre5+Bfx9Y7eDhZqciUAwSFUOZCm07nObdYqL4I1DvKAPQw6r5kjfwKGQxQ\r
+ym1w4NtvIGkebu1A5uom2P50IDtEp0Hb1CbXQnrGx+NdG77j/bKZ8jJT6agG8wud\r
+JjXRWLU45nkWwLh58vZ9nmOSGzBx/rPXEOa/Llau0FWV9BWsyWIjsQPCjHM4zeeL\r
+aQ4sl8kKlWVrXFyC7ZgVbhbtpBQnqcv116boktoX0qb7MSQqTAMTrKY8MnEjVM2i\r
+HPeeK2ZHMIu9paln7IGZqoCun5fOPD1kSMReDeuVjFdEh8rq78pWIA1bzYxP66Nk\r
+jiiTYpcM8AvJC/9OjZVZVNIhCpHWbN7BlhYmO4HGXkFMXpepgJvlohNTXkvk01e1\r
+apSRICacZemBF/vy8BOz4keimplCmNjkmKDAELy/sSV1MYcBuN8N9QQT6cJGBBAR\r
+CAAGBQJToiWuAAoJEPYo65NHQyBsahUAoMtgEr9TcjpScaiWfvfLpWmH6nmyAKDR\r
+Ucms034MsH7vcABZ+DFddS5OpsLBXAQQAQgABgUCU6IlvgAKCRA9kIqz8Pv1H2Us\r
+EACdEzt5YOA+jhvPVgqhSPm/TU0vb+VPQrM0I/9NmdAoMYI1o4OakfrRdOCOVMV8\r
+jKYuDfmufK/f+gaeHPVNzJvRUJa1JGDbprK1iJnEBu7bm9huAeIZc4/39H/PZkXg\r
+Q32jIIv10XKOyGawTfENzz25+IJ9yVbcn0bj1FTXAskyuN3baPnp5p0l1D/x/V4W\r
+cCceS5sNv/Nc0HlLyu2jw6BP10lRusAcVH4QkeBu3Pubaq22ImKG/9eZXinZUyCZ\r
+GOx8NotndXZDrvMElME//LUrf4RLYhq/zhibQjPgDSWwDYFmhKi3d8hwEbZGc1na\r
+9Ft22LQKpRWuiee9xBjSvGas/ISuluOKIw714lnCuvRgaynfd3x0HCUCkh5x20AS\r
+neLGwueScp0uu91mGH/YBy7sqnUbUChBi8si99FERDWjdffbaMvSHKSBclIBJMQr\r
+YA20YYvKracXexYjxZeelg5wTk0AUDwnvFmjBFTLfdzBkwRwK1sw9UknrpDWUcLz\r
+qPoaE1SwC3ZTzL1NpdNsBV+1rVLNOk/Rw3eJxRk0/D1LgrtWiiOQmCI9amAOMyb6\r
+D2RPoF9lgiHtRKdONIxL2FuyMp1hr8Jd0OR15umDCqz8pY2dzFJ6selc4KY/EgKm\r
+A5mQE0Ig/HwCz8lfyBTK903ZW6XUOcLBMr7RmEjzlhgtpMLAYgQTAQoADAUCU6Xu\r
+nQWDB4YfgAAKCRAKrO8XemOPd/sCB/9waNQhW5QUcln0RMxmlWuIhNuKSSe7oTXR\r
+JDiJH6y+o/F2d6pxuhFQAIG0/BqojHBYI726X1HxaWESQoIKKBd9FcOSmZxjA9l7\r
+lnt1gnuLV35qr/lEVTUe83jK0sPxZxQRBlpkOzIzWz/610wmRFsVs/qNj1Mt6VbG\r
+Vz6dD3AFMhr5wSU2RLHSu5nd6LPcG9a/me0Wg8NOCz56pceSyrBZcUzfMX9TV+mc\r
+W6nK7zrFS1yWRVWWsZrcdB2rr4S10DDr0hoz18KfkcxS/Nwc7x3SjsTt50I3hAG9\r
+Pot8hhuSueFI+wmJUaSlLRF5OaaXpxhObfVziMmbpJsjBz+kdNHZwsFcBBABAgAG\r
+BQJTpeLrAAoJEKgl2346K2wAPeoP/iq1qNKiAdXNUdu7YCo2u8hyLy+sVCW9Y1HN\r
+re3CTNTE+CWNMEg3APx5gik3bjHVBq4GDc2/LL/88bfg70HV+KGvyAPdvy0o2I2I\r
+tz7aIM7IcxQzbSgp0BAK3smdtGHeWco2JNcyTBZbOpjP64QxPdzr3P2UbZiXB8Ec\r
+PpQdSDHdQFOVoNBjB0AtdHHRrqLS0VSU2pwhJIO3RoShDSJH2WIXpNIRdziq1org\r
+Ztc76NGVg8Ccwjt7QClS8aUgeWw48WbEIm4xOj1B8T6IcM61jm29bqSUmR6J6EMn\r
+xPaNME5Zrtx8xpyQGKwOEWtlAkeeX+zSPlcA7F9ecxMbzMEWYd08v0sH/sQPdI+y\r
+edkm3O2q7KQNtEfZOGZoNw7JmDpftX0sx1WHpot7wZIK7PR73i3UH0octxgqcSeO\r
+vyUK5YqT0Q15Op7l84BoNHjANW60vMz8sD2Rjjg1b9Sg5Sfi8pkXXEfU9SXDWPdn\r
+2/hsLc0dOI0C062qtnTuschGWdVXhW+KjqjtUlq23L2aS6H/NfuFUyA96Rg2SmP8\r
+RMQIZIjxk/P6Cll0IUYqD/p8cyZ6UxViBQGBwFda1PjfIwAhIpkN2hhfKUiLLuA3\r
+da/YhAPfMWOJclaDvhrZv3BhJn7b/edFL5UPksld6ot2Por8t+UjJOLwPK5zLSBw\r
+V6bL9qM9wsFcBBABAgAGBQJTpe7HAAoJEEIosJ+iVavaSggP/0nG+nkU8Y+ygExx\r
+XWgC8Qt4olgHunqrt65CqiUjL5DwxcDZyDriNSwc2an2r4UozxR6jFShwJmfRYU7\r
+uPAaMy2UBWAD2ikDDXGpoitVoZLL5Rf9XEuwvLtJGmmfla8VkqHRnfsTA9Py7cin\r
+D2PvWADdevHkhTphIpK7chKVMr023ncfbTca/xuZ+2t/p3/o8vdCUXT+UttWAiwh\r
+7uSB0W/E2VJKfsxCoNQZ9WOlFYrT8CrVWzQ3ZpBdPo7WH1lhhVMyunqlCzi+aWp7\r
+jEypg/39NXdYbPeea8bLyD6k3lf4rjc0qjWNL9TCmFbYeW26R1VWml4vGHo6hKo3\r
+rGZMjpSJVyaifCHSGRBUgQ7YYQ31DIjSGh4CrQQDQ+7bF9eNpy2xTpVx6Mcfp8tJ\r
+SvBZZxpP+FNyIspHlqdm9u2EoM6bT4Zda0PQ7AxyNOEBfeYKC2CltcyYCbk1k+uF\r
+QmBYthhf/p9NAr8xcJyn+cd2AbdjhN+LD+QrWdsIi8wRqMQ6LCgV6aEtBc+EWDoS\r
+gJvLPaJbiMS8jT6vadIKKDaSKk1/8LVEz4OF/DmwpCuoexHglBEWP54BWyFiwDaB\r
+lEF6uXVTQ0WVzKdPTZysAN949rRPethprWNAttouvY4IbQlff516vfjlLkFj36sC\r
+m3QWOJPtEsqGhwYEvPC4CBrFSR8lwsFcBBABAgAGBQJTpfPYAAoJEBvzhG3Qq5W4\r
+cqYP/27s1bPEmJ+yMuin9OLeZjFIyolTEBvpLclTkZdizFPi60ZGRPTka8ASlbmG\r
+cdxKGe+a4czYgxexIZk5fnxRczrm4v5M0oqu0R/DeYmLVWw6mt1So9y3WJjvJDO5\r
+UJQxsevRmw5AB/KPYmcFmNoyE2oKXb0cyVfrHYkFz5VnMPZ3mg5O7HfrXA/yisHj\r
+IBj22BbQqcOSVMq0dd+9WSzxxKhxc3THv/Tfq5P4XrJdCuYWdbBQOQ7/hcb1KXq3\r
+phSSIhoSbFspDiiB9YF9BjvMfjPMaa7BG8mHdPsGKd7SmSQl5JI0Py4tcBN0XKJw\r
+FDxG6+5NCIOq7Zg3kXMlEELMyDPbkfs9UY8S4Ua//5PRtZSFziHeV7U8QJqCnex6\r
+S++RFr4Dnk0C656nuNa63qdBIvb2M+Z4IpfEx7Nh85WdsRgYe9gayYR6rDYIIpGk\r
+W053ZtvEqXJhKa01Fbncqx0Q1BZI2g8CmGkTIW6aGwahwdWct2sj7xSZCMc8kSir\r
+6p9dKmq117WBJUMVMJOINkegxPfyBwfxdy9OyHUV7kWHpGo0D5uMmGWRQAN2IK34\r
+sdW8a1q9Dwh/gC1D6g0DZDhKAwzryUI+53vgD2c9CblRxDzzthvbqzuavo8g4fm/\r
+chJ0B3RXikLmq0kRkf7ua+aTjFG+mnV1DOOAtUJwaBumAho6wsFcBBABAgAGBQJT\r
+p8oRAAoJEAopbk1puVOPMU4P/jl7S4AIpzUuvT5IoCprZ39lCi4Z5ZUTB5HetaS8\r
+K+lOHVsdRZrNIrtVAmMKl8hZ+jVUmjBYzkq/5wrAaQuBIfMJAj6dfWe69XXRGQYD\r
+yQq0XNGXQk2VHElTt6IKayh1A5jxfDz+T+RBd26kRgkjWNDGk8okqN3avju1/yAx\r
+c7MHNMdyXME0E63h22CMeTcQrHdPnO7WEVS7TL3HcKPFda4f2kMwo5RFwM+nn0mD\r
+DcgAcomq99/6AQbBmOTL8sCtQYvxgIlyhwlPEdmQulqdqssGLeqbPLyM4wcvk2h+\r
+IWFW2CeK1OxSax3tnPjG0bBRJNZfI/nl9IVWZfZBhPLwDSFEX837GFtbLlvYK7Jd\r
+dm/WhYIkB2gusyH/4foz2T6V9KU7Zw0nTk16KjH5I16TajjQ+8SZu5DLpJcDgQ65\r
+uw+qOXh/VsukvKlaazLC+q/emqVF7CSoetvPSe+VJMOXaZfkGZa0BmqRfT/Hyq5X\r
+COpT4V1xtKpVhGYyS8fX6sT/MyxlkiIkbIcjo2ODkKWz7prlaZfthdfw/LnRNM24\r
+0JNdLMnAw01MaKqv2LR6W4DRy/2ny7M2TOPtGaatDv1/oJDSespuz4gFyMAdpynf\r
+vmyGx+H3C6sJztQm4NIlg3pwcTcL2vrF1i20xRyJDIUJGYy53/NsU+rsQB1mtWea\r
+Vq1FwsFcBBABCAAGBQJTpe3DAAoJEBgMAusJ0L4VYrgP/2pbCyf0uQo9LP+YXAQC\r
+YkMGKPHmb7FNWC38odbX18UKz8m44pHIGrQQ+HuNQRetp0d2f0ZpeeCsJkjMxrgK\r
+4LZbNNueoCi8wLdShp7931iml1w1A3Uxo5zPXr3dbtvUgegECDKsh5IaU7awMcyY\r
+7bMaU9Cp1BbHNrBVa7TMme9dgL0X8dLJ1LSuq3kVbKiyu7iODCAXD0+6jtwvnvbj\r
+UDNls5v0rpsihSQBlDqeAiY3q+OMYVLvmq8DsTpYx6BuZLE1g98d7wL5P0BlhN7r\r
+EIKrA5atPKKBII7z2/AM9lVaPfToj6ohswUl0UiPCnSsuWoHnHelV5BW0xkZR6gc\r
+/FfvODrlS3rvGxQD/c+saofGUvRfoEPMVIoQ6e3U4LRGuqvfZTJb+OfrD39pbxyP\r
+2Vs2NiumK92cDTG1wuuzHtolYNyTSpx9EJDpUm0ZF0L/glarr85tSMATCPVBHT1a\r
+P2UnsTUX7eoVHpLLsyxYeacwt3BSyTo4PyHps4QcrgWEPxlMmf3yzrXw6x/Z+o/S\r
+wgF6n5eULq6KCfj2G071i94FfTdaK4qs7HxuiRgIPjbUZdJqhjRaSPqvUBuC6mjv\r
+aj4lWJ76Q0iernfkD0k+hGbNRFKeUNndCBs6mmXBU5MZVxFvS+D13hyz4pC1Ju0h\r
+iWn28vNKBXhghuJ0EOM3aC4xwsFcBBMBCgAGBQJTpe1zAAoJEMgTKeWEg7cXSd4P\r
+/jXBpe7kApYkTY7qLbmDe802p9/gVCOUHWK7Kj5Y5HEAN9hqa88eb9i6TgU7gl6n\r
+mh0Gm7K7t12Yq+2ILta7UTj8jI1d4WDFd3JDWhu9uP36JpJ/S6F7Q3ONlG3bttU6\r
+3xAt9zINrqlYUPcvocWmdY8HgXLwPtd/Jw9zY+WRoRidYykOtwb/JSE6lLxQGekg\r
+F8iNqXZMw5zIkVxHq0J8RSFEeA1Cw20M1nihzme4sX0ZgSaWuf9FWlt3VpaL7MkK\r
+ww3QsyctDyRYfBwci6+9k1aLSpJKJj2oAlk3QKAO8m27Mym5lIQV8xQUwgPy0Iqt\r
+CwvKd+5VBOyDW+3nJGXIN4Mf0XSOb/+ecmhqFPPwfDrG09QdL//Ogv0CNeqzq4UJ\r
+IDbmReGhKoGCb0esXGagQ2szVy9aWAh224+I6na9YjPZDaxPbNXSL8dIJuF++Wor\r
++WYgziHW/Ha5dNVU0qdwSe/36SIQK2MWq5B/MO7dPEWiuz8H75xN+rEdUDV1Jstf\r
+5MHZ//cg66gwDSkiGFs9Jkf4I+4Q+nrGvRQbMA8NRR93U8ttgVA3DdFMAecHy1QY\r
+uNK69BhGKSIdVElixsw/O+GsEP7UZvK9Ah+oUYOQ1mErRLukshVQT6BdjNv8/Jhj\r
+U9EJIweqYYpJjS36aVxqysTmZYK7ASPkHsheFyzOCIMmwsFcBBABCAAGBQJTvUcM\r
+AAoJENmBfo/facMuajEP/0rbu7aOmQ3ApDlh0lOTmIqmqx+UYrVJu0vJV/S3LAh2\r
+ZSjtAJdGo3h9vsbPGTLmQwudKFrpmPiK8M+mTWCxW/27mFlY4UBI0/rgoKFq331u\r
+am/uh/wRG2QU5ixd4dlUI7kLnAemsSz8z86YFK6VIdmY5Wq2bV9Q2zdj6yLOF+Kl\r
+GcUpXY1otOziHAmOq4GvPCqlCoc8mTHtqUXWz5DfRAt+9Y5B96NLaXFMGzWsFVEu\r
+rBBcUcyST4EHSgO4B7nubUbmlQCF3BOijY2dykimmOPGQvvcT47uZA9wIuEEM+94\r
+89ANnOb2tA6kiDFvzUZtjcSGy/TnToLdBlby4kFoAu00AapzwNxQ/UuP+Pxx4iRW\r
+OfZpRZTXi4TUSbxZTvJBnRUqTZ6vsTy8OXMkjvHTU8GvNOgKGihfmRGGFFXt2Hqj\r
+cd5H9q/7/2ve3ba03B1MAXGJwnGO94MX54bQaAIE06shcrLuvmAOqQCi7DBTrjO5\r
+VCz3m9xVYz46W60ak/Fm0sCr4kK1Wt9LYsevuRMkjafs8hXJQDyjhW/w2NeKSaFG\r
+cir0FYUQ4p2DBjZQ0Xp2bAiPzJRnxiSk+uK1K1iLoJwEZHW8RoO/iQT8Ku0pQ2IY\r
+jTHkPwbp5jTIBC1hKG1PAg/jvPXSKtrQU1I7RgZeoGwMmIzmPLOiCYdEfg4j8cTV\r
+wsFcBBMBCgAGBQJT3QeQAAoJEEHcSEn1UxP7aeYP/02Q86vs9f4VBa/oQH6b73EG\r
+LXq5JFUkH6Xh/qmAnZGn0U0f+UBcPzQFxQBiwhU9ptNbKIbM9NOrnwav0PsTURmv\r
+au7vS+afoHPNMaVLEZiQwVV44UXivrfLW6nfo+oSkJ7h+GDjTIXy4z3obWOSvnFz\r
+YJUEsbGjhzFE2SBU243TzTU9yonrkrxZb9GEYD+XavPSizfMYKwRzUay0yCg7MnO\r
+Q7ywKGCk1KIurTqUwiifE8GDImSJBzqC32u/I6N4PeB0XG5TCVY+HErraccPLPLa\r
+LXn792jiaCWmsf0Ev1qEhI0/kSQM1m9XyPYrTHFl534SumOZucfF/t6wCmBQBG93\r
+OlBgM7T3fwaXtDZ/+GyGz5S4z6i/YU3Z9DT8Out201ikOk7xm/iO8fmr++Znzmmt\r
+s+CfsnR0ABHkNfpME/F3Xq9h5B2u4Y8S5+Ucn4QQw2IUVYFVI84SZhb2ixQmxNt+\r
+WJOBSpweFz4rZkFjmNtYe74w3Mu0ZgWe3wBdyDMewW0tVssrxw0MX07qiPwHiSkU\r
+bMODXb70YvbuawbJ9pt4hesddmnjDCHC8HKmdDqIVK+yF9A4FEUWof9erVyv9fX7\r
+h4r2/JFh7nGeZTRzKofQNrreAnI/QyWoxaiL9e91ZAGlSVB/1/dcKukvsjgqFkHi\r
+pRfQUFTfqHgO9tIscevcwsFiBBMBCgAMBQJT3QgTBYMHhh+AAAoJEPPgD/qcKRt9\r
+pMoP/iaTcvQZCuIBmYVkzffTs62PSW3RItm1aqH35aTIBgN075UplkzxQ9uwP0P6\r
+RUb4g5tAD2frENgOawz14V1XgwRB5be2r4D/zSs7z8pn+f8Y1kHXt9VPtUSbQjoa\r
+beZws7xf580/cWObUEVzj4aOuFwq223FJVMABmqpJj14pSDjawJ8D5i2/s6Z/RNz\r
+ZEuSTTMBnl3ZnXp7xCezccNy+Ufg/vxxz05Ks/adNB90M+ycIQ3d6abha3ZhH7Ms\r
+cNxVs/VifEfUh8qWnSvU2BOwXVqyUGCXxr3qEX6aUbi9dMV3WlqZ8BmWrJca/q/7\r
+jRL+l8TzZjCQ/Yr7cVBHTdleIdpV7XVGFnZWFvXPb2seCAisCLz/SDIh7am4yMIp\r
+WZhi/c1XaJ1TjfN+Eo+sA1jXCZyaUdQL6BQFVsg9xCCEBFBxP3jjq4mNp8msisdV\r
+AocSoGMBJNu8cRXWQ2UsS33xY571Fbt9AxotP+JU4H67zCrvwpDw27ZzGguUOLLS\r
+/wkXpfwZBv4ex21D5CRXhDFPDpqAoje4BaBClA1JieV35KxWKceaYIpMG6o5QVo9\r
+gkVfrGRYI2rFNUnFO16HYYngS7xaAFn4O2Qo4WBiSMUfiamvVYkQ+wE0BSBmPlfH\r
+SlzDXvXZ1jJcaPy3CD0H9xoKvqdsspQLa/VSv9C8CXsWOSGuwsBcBBABAgAGBQJT\r
+3Qk+AAoJECQK/QwSiA+Z1QsH/2F4gtkylsG6+Zq7abO7JcYTs8B+rdCf7J1YzSid\r
+hA+EC+T9ZfSShfKyhL4EjCtsMpyieui7YUxdVVBybN+lXnwRCfm//DQlbFL+YKJR\r
+jj6Lknig2M86rZ73z2rM4VNW8BlddipEjOhP4B4azNfMnJH9mb12dm+BbiLyIqrZ\r
+6YsccR0poxTZ6Yl2+lX+eJRTddfw0VZ4RGv31I3qJSR52IFEM8Q5XFrn1+PhHDZO\r
+GOYp6UVceOzRjOnAJUZza9HRcU5EYpagaAWmrz9iakILx4C5V1AdUHtQGn628+nv\r
+YqdbTVIUL0p7sRu1WZXCrAj9QAazv+9AWlQzP5h59AGuZKPCwFwEEAECAAYFAlPi\r
+a0UACgkQ8edY/ArAPXdhsQgAhav9odIo1YXSNMpSrdfIuP+0wq7VJ3dC1x8m4Gxh\r
+mrSdYC3iU5IV9oMPqYQzYFIFf50lqodV5k1k5zRhqeVnfyqvcU0tCEQ3gH1Wd3tF\r
+ZeC0FPrS6RJuezeR/9wND09e1h/ZceZBcztAG83H5QHX9NYkR9EAthlv09/QyBx+\r
+Rl7Sw0LdBr5g5Cm/JmvVvy8vo2HzBzng3rZMYOL2/btZR8rfAueItf2cb6nj+kf+\r
+pL5z29SnKhHOr6tSn1PTn914rvGjW0AcTIXLMAVFZ1EtzCduiGDsMe8uEV5jt8IC\r
+6n6LOcwFSHXQb6BxO8UtVPLGmd7oUFIlRoo0Csgn0JFGLMLBXAQTAQoABgUCU6Xw\r
+MAAKCRA4bQgWkNvZQNp7D/4uWsGQz2BzHrkM5W2am6VjtBAOS1QfarJNDDLCYjGd\r
+QqrpTIx0eyoF71Y+kogOzhPl/RTp+ZU/ICpJNOJkVTl4OzWcagbPFnGHr36WAmG8\r
+PYliTkziO8F4VGDDT9fc2qhPS9AcrUCyTtD23BY28+YBZDEcByDnC625ejvb3uWa\r
+zzras7wmh3P++y/lLpGxWmwWnMZL+z1Pa/KUPPviZD0KNuza2p6FppW5RC5w7RDw\r
+LfCL2ObHFoCbRCqDAdstv1yJSxcH7JTTEZMtYJN6Q0dDtESlowd/C9dbzginrxzH\r
+VuNF/K8G6KgBwnIhwULX1S78UMWl33gbm9Bwi8vOLalut37TbV1HCtux59T+7EZS\r
+aBUJMGOBPlT4PAlEyShSgsOw26WupjR0N3PkLNnEP+FvM6fZEwj9rD5v6381KyyG\r
+1mOzUtgjTAoIHrxczzhIw6cnM3XH3yc7OxDzGzzSk/PQnFcsHjt5evXjMFIbP3dy\r
+6x0OH23om72i0epZ9bHOJZTvKFfVTBib+KWhW+6dyxfiCKsj22xOwxV4qaEnfDM6\r
+a7Hot/KcENGE3OzL3h0JI88817n0euSk2500SJwGNZ33AkVE9bn1Yxn0bLzTxEI/\r
+PeyXeqFi5k0QwLqg51tLxteWok6LJ+xEoAWEym9UyuVb4g/W4bWEIHemHkdLqqxn\r
+ZsLBYgQTAQoADAUCU90IGgWDAaiDAAAKCRDXXn6koubB1uJBD/9aQcRpNVUK3Ero\r
+RaG8gjXVMbCWG+1IVwmLGA0rmklFJOCzMMQ3lAfmFyi2Rrw0vTzhOdrOCgod5mud\r
+Wnn40XiiufCl7NzSPiOQTxT3b2Ee6X9zlvRW/e+6yC8s43C81b5jcWkRJMUEJJo2\r
+hOzUlyz9EvvGBvLcbw1ncGX9ngq47+tWNDLD1hrgRpYwCNRgXgwu1oySq5Wygqak\r
+3P8K1XRy0+a5xDGMb2iJIkNqLSjOawBhL064ly/yNRGrnC06VPnl+c4R7A/ySbiq\r
+8T5jEvgld5BmKG3FLyS7eOTx18ml1+wmqwTDkOeoL32skBTSJIkDVkw96oTVFHOQ\r
+HLNZ2EZf/ebAn+mXmhwR8dGWf7zLAg8VdasnG0zJlqWDYY1wLy/XLC0xPPONBUSc\r
+ku48+89VBQZEM3Vhr3GTb/y1hqbs+KiFodJrKYAhOlBzK2Slj1Mtdi1mPiJHiqAh\r
+R6QK7G4BHh6R8s/3Qen8q6K5pLhqXaDUwlmnde5umw3uiuH36Cwkdk5vBsmz77nm\r
+1kThZShmhssNm5UbGL3T/++DKXO+C1I9JwbDqeJNW0n9qYIhYBbtTVQ1Q0aPDO+Q\r
+ne0gOBNtlN6U8IhztBoIg1W4Z1LzMfa/BNabRU9f1BEN+LD+jY3hCQIxLaH5qBBW\r
+9TrZUwtdAK1A5bFSAaGpCw8vTsdvk8LBXAQTAQIABgUCVJHpVAAKCRCfGAZoRETN\r
+jz0tEACjCClx5J0wzDLrQwS87ClGigxTAyj8Hd9mK/ZSXtjByk1mTNH06gIPUsAe\r
+isxLNDWpWXJ4uabv8P4gvH7jK121S4rhTvxlUkY948oBYMkhT/toINxaYiYqtAJU\r
+PS0T65A3npNljB4qJk5LmJboiGjcs1SJD93bIOSA/4YVWNt5FoMIb0nVlKt1mbTI\r
+O1hmhvST6c9q4HQ3qBLBe99WXClKCxq3P9+NygeH8L3jy//Lw/cWgWv8VW6J1O9K\r
+XJWr0OMdqJiJzgdZp/bAcTwernNiaJZLxENHpSbwlUMWAm02HnBK+JNQWOMllZ/I\r
+OLGlgvPtbMuuRlvgigCBEWFXO5Ma/j/qlEhicTCX8ytBxLfm8LHRtFXGYIeAvg+t\r
+JI6NnxEAWwegnH+3RQzaYMUk/J5M6cQ1FpOtibFSgxxTavbZEDQe/fRjJGvs5Gzi\r
+7GqssIEtyDxOxuYLZroF4VlnmZKJ3GaVLswPYbil7Wd4vWeDAc+SLWcATf1yiCpT\r
+EMIOI5Ir/ZgsAYHuP9lVdngsM+sAuLVSFe80y4YgYgpXiOk/NzibwTf6BcqM0kty\r
+AO95qYzYgQOtG4MPNJiKQB7Y1PoIRnHh0g3UPHdzIqPqnIvRXevk16Nq5GplNTPP\r
+SoZWQ9R+5gCYB+9yFnqpLMiQvhZN+Z9WcRGDp2C4U8+RCuqzq8JrBBARAgArBQJU\r
+tOGHBYMB4oUAHhpodHRwOi8vd3d3LmNhY2VydC5vcmcvY3BzLnBocAAKCRDSuw0B\r
+ZdD9WDk2AJ9MOQTUUu0TqR4oCksOF+Mzj8d0hwCeMZ1RHrCHmNRuXah6ahyWh/o3\r
+gFHCwVwEEAECAAYFAlTKamQACgkQWn5FD5nTzuedhg/+LS3FwbhW0esyIu+KFUeo\r
+2B0FNH5LLw4C2ydm5QADTQxr8xmWKp07Js3odSlSoDjMi6UFfVEG/sK0RODy7tW4\r
+yJTWndcW+oGeRgVISNXHdmY7eRQsRwevilz7t0wSE9/n5lBn6LRZyVw6bcWl3rU2\r
+iM1DOeSlBwW/G9ORKVxZ85MLiSiguORdh3L3s5jJxFAoECWmuCOlxhPW+mjsZFT0\r
+j3Q/qdvgSGZrBrh6uN2muYjLEePiXQAfKlsUupsCqSx6qXHqAJZpzYXY+TiHzzBL\r
+94NfL2xckwA/E16Jw24OEM/zUl4uBuDXK7nT9uQFmGLiX/hFQrnEOMpr8JozW5nq\r
+zJgmXzKb4S54XL80lcdbClfXdX5VX1BQ0tijuoJpzAFbbvhUypINeLYqpJlDCjRJ\r
+wXgZqF+3VzxmYPYOlbWVSG572kmGkKBC7AHhaIDr8jw7w6wGjuuL2DdSs175S28A\r
+vj+4VgJ5bVcBOlk3yP4aSBJWp00b45f9zeK0kGpAHfP0eftUoEwWXSOxKf1V6x75\r
+SkcmvX7hHBXQ0dBDEEs94x2olbH6Mu1qb8ukJlG8TXyPd3UU0KZhGgaTTbwAcY4S\r
+azdHa/MynybYn5PQiLhxV7RoQek8RVJ4+d8Tty8JYl70tU3A3pfH7OzbQq2Klksr\r
+wPfbPfviqbZI/X6AXCrjOmfCwVwEEAEIAAYFAlTKfUYACgkQ3aqD8vfO1FUB1RAA\r
+tVn5gXQDp2vv3Waj7uWG08J6jd1939hV6NwbWX6cjJ5Fe/67/LHc4wfVHw7DXPN0\r
+swzbtmNBaR4VDTDmymJFtJCm2d0sST6pHj0Jbkf2ZxSNFJf05eeXUULybBN9SD0K\r
+HXpusHHwfqTBnXSRS0myHxHxudwYhIWiGaS/VWk7PheLHe6qsQXN367BBJbLSGCf\r
+dh0NhBWZwfT8Msg61mqmY3VPoQTBzdi9wvwjZeaRjI0I8QfmY3gOmHJcwUraxgSV\r
+7Qujta3+sKcUzm68itztxWPha7j6X2jy0TBo83BsuAdo0drs3fiDEfv63aQGb7Fd\r
+0S5fikJWjB1rlXBCGbej24GGNU6D6CzFoYPdZtISCyHhto01NkybE129Ef/aZ9TU\r
+CwugfDc72pykPkX3DYvgGX3B3td+W/nKyArMBjmhmvOuHktqNBzo22r9TfQyseVu\r
+6pXjtHh7J3xz2R8Vob7TF0E4jg1U1ROCMiNd+BKZDD2LQ4XEpkdRofCoLoswAGLd\r
+Xd2e0rfIIPLplPXmXlgLOCyE0DM23fpl5uOhpq8lw7iVDl03A0su5DZfLVedPtgj\r
+/lqiW5MD3RqT3fECT0B2OlcYPL6GLmNWPFhE5yBOuLy6YD3cF+p1wmgy+cqMJ0j/\r
+06CnZ62KiG2EiWcCmcMJbSjROZtm1dQICK42RAcFTH7CwVwEEAEIAAYFAlTKfWMA\r
+CgkQ5fHPan6KVly1ig//SVVp2Bo5nyJYKzLPx0me4GHqvpNe1nYhnkRV9pjtSlug\r
+z/TDNOYhLPbRXr0Owu4fPeqW8gOiLRcFbWwHbGURzgVLpN2i6Q2zWnTJvXrARIiZ\r
+L17q6C4jXqW6tBNGeCatr6H2zZzcSf0mK3vsyMz16BnFwzwvYCdDzRb+JrvCsUjp\r
+s7G6WKI2iYW/cYCIUF7rmIQWnQ9+mXdUUDoK0NO5fKARYnkAxk8V7EQ3K6fPeR0E\r
+xR2GPlJFF+ids6GtbadoRoczGkIyyx+No81hwNfC/vKp3lP8be2kwblFGOe4AkwX\r
+ySABqfLCMvHYz2YLqMK62J01pVJ+FtWq6PshXe5VVwzs6iLg/cpg94QWXmqUOaF4\r
+8n53NVP7m9R5rHXG93DRXUy04oi5Pt0UOZHO0OjzvsMUOfalLV0dzDCJMHqIXP+l\r
+JqUtIrKM97vGCbBKtTAzXZttGFEnsnn3qYMZJQtq+ttZ+t4cILenU65y5AAkdZ86\r
+gtyZH8qNoZZ7MJkhWERHB/5e8EjdHSL9SnFvA8OhlAJU9ij4c2IMsY1ljzk8g3YZ\r
+D19sX3XEZvUFFbByM0ueC5ATaKJ6ZJcPZNkhRm/9WnOD06rEnk+D7DgKsaYhYC4b\r
+s3aseLtPID5Rhj1R8/ygiMVrJuv2EbFiyCLNlveL73Wspe35+5IPg6un2yVqBs7C\r
+wVwEEwECAAYFAlTKam8ACgkQyC0ga2TV603ohg//aDp86NdeeZovXl2OcQmfK2Ps\r
+RZOkNsOlXTLuFfIHs29Rm5ALXyZ71/7aaCI4ayx3HRaghytnzyBpL/zNH6qD6OAT\r
+Xoc/NyaJAbvPFuaWhgMAdG7kKRsBGtG/woFYOFvXTHZ2o8koHRC/yBqZ82Vua0J5\r
+sG7UDqazyqCz4NxDWxIWhUj0upS84arxnuM5gsvQLAsldRu+UOg1JKPhjNbRAK/X\r
+4OCZhYD3BFs7ISRB7bK580zslOLvSZi4h5gI0wc/tdLi/LJsoEK6Yeq8R3GP9ew7\r
+e2atKQD/+8YWJs1L9y1uWW/TlLuAt9lWhFrtuKloHOkpIKE18R03tTu0Yyc+jRy7\r
+3Ahwf8hpKZrQVgWXFSukHQha29g96aqVJDz7KT1KCiAdDmotYie/FEqD7EChuDEr\r
+4pkGlvtb1NvIrY/CiwRs5jjFPDhRAo23q3A4aNVtqX6fHloTJu9I4lO9377GS0v5\r
+g8+iqzcOcRpwq3fPqyBF7ZJZm0RYME2YurxECYJes6mby/hWx7yL5iwcej0I+dYn\r
+3cskHoRJgzcpXijJUslf+dp/gxfvhA01KL5RjmPfxhXm/+XMvS6yTpCU50739K2i\r
+7jqj+NeJkchpD1ii6MI0lXDEbuPuw3MThcNtqoMe9eBpwegxa1FNfAOHF+YVLt1h\r
+M2lqPJbvTOFIjEDcaDXCwVwEEwECAAYFAlTKb+0ACgkQDu55BMaeGhYzWw//Snze\r
+7VMz0dScigrWwfQX68a/YG7dBHnzBmpGV+5u6vnSK7tvDMfgyn/6V4vX/M6jS/dv\r
+Kg5O71nsEAaM63A873RiYSZ6EjqsCr7M66MOgkIG35ioc/dFcQURpPZjgdpk6GcW\r
+srBldGmupUX3ulaMJf1wEl8J0pMPE1WfoOSINXr0forQWyUVv6zaoGpyzhEvlxcY\r
+QSUfaqQEJ5vW1noOkWrVOXWaUrck2oyPUSu6QhW0RGLcOu4oSA5rFms7gOMRZ49m\r
+cA4m7F8K5qITP7VvROzQl8oVE1zMyQIcS3yv/1jKdtbbO9vZcPuP+Mgwq+e/ikgc\r
+94UC1IpZZ6lZrPgub4vQOlrQkLVGdeJKd9QdA8vY5taAicHvrytCh6cZAlvWAPQy\r
+YADuRoEkM5ZY26gQj8nCQ090IW7J8uz5sakfUm3dyv5d8dRRr26/7Qc6e2hZpjYD\r
+3FJkm/i/VuLy8N0f6KwtFXBO0zbGsM/TmtHXV2VsGqBuhfRjCP3/kdzvhy2IF5xW\r
+FYMD9eUJ1fc6R1gK5b+pnvBjdD4NQ5bN00IxMG7HdwJanPPAeFsYnD9BqhFbcjst\r
+7c8EGk4owQVHJes/JM3qFV2Znme2A3ssavaGwLkbSIvtiuAX+k6Vf4ELie8V4Jo7\r
+2PHS1CUjDisRhomIAnMERJnZ6gAHrdUStOT8JyzCwVwEEwECAAYFAlTKfUsACgkQ\r
+8px9x7dEYvpx/w/+P2nOdcuFJJ3NMrK32FdF93IgUqtKzuzjTQTKcW9WTCSBxGcB\r
+n2A0SqlewB143KYVLqs14uR7RDNLZoPhYYJ9WloW+1kSc6D6ImQ8ZSB70Oe6dftB\r
+guP+mvNRVqqfR5rOfmRGShQxwlDX8KddLcvuzhM+i5gnNt1oFfbos44sbd779fpU\r
+CieNbGQIm8OOIbqtn4oY0y1XD5D0YV6mok6PT7LrNwm1lNvpfCI+LYAQX2n01zz3\r
+CL67DjfA+bbLI606VxIJZBtOyTAta9+BHcBYeRy8bq6yZUg6tSVCU4a7Q4HyGjQ0\r
+w3ua3fNWIi1IqbAqvJezTgxNW+yhV1RaEeRZksJ3Zm8ueJrKV8yJJS2I4/QECwgj\r
+n6XvrQSs98vRW+mRyHzFof0/BY3avnh5BbPsaaoABfXwwLMXVKVvd0aXGV4ytHs4\r
+s55EScu9BHwAYzmImSghHgat9wQxy7QCjhR26DDKMULzxcBFSud7Lx8QvQ9p0J9e\r
+IuMxugIS2mcAaPDHXg87qJMkbT1Mg47iNkvg+sBxhpPiQwRBbJX3rK4vGBCayAXi\r
++erI1rI9NYd+xcglJLWraEgD1BfZb1fQIrrjMBvSEwI0isk5pMYWpmW4Rj7ToUGN\r
+X59T1Is1XvkGBPEENdiULtlVwG+UNRNOL1ikU5/rwfu5aVy2wyoojd0msYTCwFwE\r
+EwECAAYFAlTKfQUACgkQ7QEeGNFBqo1UJQf/Wfxky6C9fBrpJDcBoOtEKy3oQTs/\r
+zDrjq6eYyM2r0ri5vRk9g4ZqylTtCEZ5xmRjnBBZeDANwXOGtEsuQdD14So0qF8G\r
++11ouW88l1O3nPLLf9rYSN7ZcbnVzj1xtslgLC2POOr6QG4r7jP0vq/oId4C73Kq\r
+i4hO3SOHFVg41Df1LaGKgASzcpNcv96yYRKRF51IepWoD/HWhxYxfb5PYCqk1Vtf\r
+gfkLUlV/+5XLV6AjkT3DgyZfUuZ2BJHi1LCb08RAbG58WL9bu1rt9MddW0Gxoys0\r
+179wXl4qurPD3f3WT4ds6WLYexUMby0zj3rf4Z/KAdz5mhCKWVLYjWk7DsLBXAQQ\r
+AQIABgUCVbe5UgAKCRBvKQit4kLuI8pJEACT2vRghf/Q/Hs9+SOjnWzc7ZeS02+7\r
+zCx97lLs6FRy95NkIWmvunrduwcNgcuGeC05/yZlT0N9LbDhQWWhlF9Te5bAZFFV\r
+CK1meTHKPKckIlnoxzagtL97AMCnhSL3FtfrnZuQ0BXdWt0cvZvIrq35W8gO7uHa\r
+EMTVEN/9eSg+ueraZR/5LthCv2Fm3gIdOTbrymrGBqfodVh8ak4aEp9YECxmwFmD\r
+A0xL81V3vS3LKjMSzYeOCoSDRd9soaN0bjbHqZch1iXZcW4LeO3aaB8LVhsyktI5\r
+RBlQwuDAtF87aEDCQVWcaP5VVj1frQ8d9ZfiAg+8qvlJAgLyNN5HK+6bfVhrbAPJ\r
+k4DEGahXT0MI8crTOUpkDFT0cSDCraIFITbnYZwxoWdHCRRwoGU07RuMxDBKLgv7\r
+EHkNXc5V01w0QEs5cdJ5e5Ystg9SAeeDnFkF19o+UW4luEE2iZH+ABXpEcxCEOOB\r
+1nYjEWMbpyWaySeObCC2H75vraUhXFz5klFivaHrsj2yLrE3/5zL0tg9ach0atHK\r
+2lpwUeXhDHJ/TVZLY5DKOZExMzxsMl8iNqVwo61aR8POeZH2+bJ/7RFDmQw9ib3p\r
+XBb6aJ7Vc7tw8Sa2Hx8UMwoCkiaLS+JrkYxCAeBKQczJLpmC5Xia/fuZ+Xh7IxiS\r
+Tov2ygfzQbdRo8LBXAQTAQIABgUCVQsDjAAKCRDqFtvwnq3ghj5OD/oCNxtUYWC2\r
+3rDp3L/TZzUjwDINiG1WpQXcR6rAuIX61VNiIKssEtDqScAP6Ja0NNatNXn8jkLM\r
+N/FlEEN16awDj0Szf/TXvJG0Ph5q4K2BYVoDESD9EGRpyTINKipOkWMv1AaLAu+T\r
+uO4riMaLuhL1/+pI62ugbwVkl4I9VD9pOPl36KriLlQKOyNk2shjWn6l91l9Cio6\r
+r8EDQo4ujB7mze42Prz6ldkth/45FBvaqD2qy1qB8sxQ7PkBNtZ0fwuflIxzOV6l\r
+L43KmXG3kbxYUJEaecEPh75tj6paRdm486+PARdDY2GT9O96iR3je7ws+JCfhZoL\r
+C0wtCxqfyZpfBjhTDm3pzwgYsmexovANDGD+IMWmc7kPj3SpdPlEbA7rLRqrABuF\r
+bfApNuazcyoX03H8xREAsBhITTVww/rXS3ZTMITam40x+eY48UigMXLLxSG7O0ip\r
+J8POw5w9s7J/bfb8fuJUkqRIGxo2XPP+1zcYxebvDQ8kppl03lTT7j+qJIJUYj7b\r
+mm1sWeMmS83xMWrcSZ4qOCx7pKIj5WH4PBmmSlFKvtmDeNLaA+3cDpugI74XSDiH\r
+d3ICsEwfqL/7/mgyUF82A1WIdHbEKpDlJqHFRb0IIHKT9YymHj0rxhabAHtaHtlW\r
+TjWZTHv3L58ig3bD20s5dbQMXWjgtULrmcJGBBARCAAGBQJXWA6PAAoJEO5JHD4B\r
+I/Ly/NYAoLGrG8JKZ3L+EWpKJBXpL/3g4BgTAJ9/9O/lUUhY3GFaImtMMBz+Tu2o\r
+FMLBXAQQAQgABgUCV1gOrAAKCRAlBTPMopt2T5upEACc3T7PjNL1jgg/1p0y9wGH\r
+dVZUEAq506Fgvx2bBEFnH/SeZcDlbBtrvXp5oUCwSgQW/2SVzI71lPZ8lLVrvRqP\r
+g/YLSkNqROMTq+iZdOO4FkPRHCXr7Ee2E12Q9HZN5x0qotD5aJqc8B73lK+JNqXV\r
+VCmL5nsmCEVnLMEHdJvN1ZUzDSWFa0QJC88oYdiNoTpb/ozfhzDwqsx0WJL1Yk4Q\r
+kWAIWd5x0segtpzlU+SJvdqftm7MWh0MiVnZIO4xXkDWe6iYoEFxJYp4ApT46LXE\r
++5SDS210Dfh+XrD3s2JDhz6j/upbTt08uvrLYSVqXfWmHK3CugONLQDhyVkS4Pdl\r
+FEZMzk+m+mR09JZasSWk/ngvGbnkln7TZdCxW7gqfHDEhGYm/sRr166vqjjPMPV1\r
+ZqJ41//xk+t/ksXVeuAOLwkr8k6XjDMDPy2tQqVcxmXedSe+PTJmkSTRw1sREBNq\r
+cThsmUej0dAKZHuPYUV9HmQ0cIicXprAXpxJNLzoTTcBimxrZ6XXuwDmwvnEH/l5\r
+yS1LPMlorejVlLIu7N54IO1HkIK7OB0BTiwgE6G9Ajhl2hY7yp7QL96oM5/p/WoT\r
+UXcUlijWut5gRdjlh5v3r4Qid9X45vld3fiQSf6pd7lDzVBzoXXhwnqqtScum7fM\r
+WdC92nS4Ln7YjoIHHadFLMLBXAQQAQIABgUCVp/u1AAKCRDGNe2TiilRSuZiD/9f\r
+syTDxyC8X9RpotcKhBVvihCe2cJjyPoW+d0tZzmrD5HKfefB1jpy5q4XoZHdj7sF\r
+w0HM/SnGp53XtEziTdS3kk7zHGOHK3DX2IT7kb+ZtVvrFppIo3Z0QuTPLncNBK9a\r
+CU4c8I10ifaouWzshKUxZGnJZMCwf4JXulCgbib1EkcX3/2Tlb8wdzVKfsJSvshx\r
+U/9deRWSCYjDrlNrBO9WTlKa3LhTFzz+C4Q7sS5Gge5ohDPSmPG5GK4RF8/FaEbz\r
+9lqcyBRyeqvH0gNXMtu9CDPVHY5Lb149U4XshvnlA3Ov8wmFeMkKryup6f0J45BF\r
+hXOCLZt6S+Pc6bfzZUCKhzHBKFWe3tTgYX/hXp5Nz+WU2PVzMgUq+oyONO3CE7OO\r
+jwW10JlZ2sRx/k4TOIuZnjpjdbzv20usJCZCGq6xc0Nh/ALSAklU+UjM2PMrI7d2\r
+2ppPB2tcRsnBOn3FA1tMskdcRMpfKI9EpB5sSk/2qwlCdGUwTTT6ZGxZdAw4P085\r
+D/Z82S+Jhz2sGDSf4v1PGoooo5nkr/goYwI2TpbhrLQpUTfqqaipHFUQzQlzEApW\r
+BGtMyNlPLZsOdH4AH8bnF2pst5zzE7qe4W1mZLmDVURlu+14s7U6kyFU3pCx36GA\r
+lZoTGxT6t52JQMpRfr/FaHqY0dveuuap3CetrPSe0cLBcwQQAQgAHRYhBIjiUT0Y\r
+BEBYGdB5P/gO3m1Cor3TBQJbM8cwAAoJEPgO3m1Cor3TJBYP/3OH4S2e/x9AGhLG\r
+narc1fXZ+W/cDZ8ik/kY5NrRL+IIov7evNR+5Tu3Dp133yqQZiIrOmmIrmvuYaLe\r
+XG+tKefe7NRg5d7djOM1pdxI//CyS7ozBDV8SDQoXzX++VUZoCAfFGJRaZAAN60k\r
+Ps4sF8kVyqSYS5beVjGYevB06FIvA3+WPptTvPZnuxczO9ARPA5OQfK43F86MKYv\r
+CAzQfYJ6sNYlkmLxEY5eqWjWlkDsdDhqTrrlnk6hVb8JNq0EIVyaa45fu5tfywZ0\r
+HR7d+cZLWAb61YQ/2XdxZaXR50VG7WTgB06tHjycvVpollFAyfs4JejZX5uQ8Q4i\r
+xhQwmCCMgzMqRXB4jjCCBV7KFM0YS7r+c4nQeXIVcAkyyr0h/M6D032tzHDZHx7m\r
+3GWr61+MhmXQ4YwYEIAR0NtDI7ZWDebgOS9ESf/vFMMhKPoNxhvUMqllDJ3aGDdK\r
+7H3uqRBw9186G5VmyHe7zqIVT/baTZgV0kOcPztQtnoiSCU59bo0iDBhru6YO604\r
+mmmVfdcxmxr1/E19Hb8L/3kM2sZY9t3TynCk4ucZRae0mOD9nLZOLPHtA+VumlF1\r
+6BLP5rSshEsw8I9ffBtV5dDM93F+mbsULqDPqkIuDiXJNdpUDoogAX5CmGTI55FS\r
+3bBu890uIY3JqrAE20UswbQnYu9dwsGlBBMBCgBPFiEEVZIclGjjarIqk30HDiuE\r
+AZp9t7AFAlrp8cgxGmh0dHBzOi8vd3d3LmZyZXlzdGVpbm4uY29tL3N0YXRpYy9z\r
+ZWMvcG9saWN5LnR4dAAKCRAOK4QBmn23sPYPEACjvoxBRmpU89zxEH0/h7RIQBF4\r
+IfMU8VohfvmWsNQHUoClG3olU6JjTLO29/TmU2UxnlNePXLP1ZewHsD5NsshuYWB\r
+d3bkW92UNF2FZR5L7234wbt6eVKEtNgDAaWD4IYF22WVk1DsCCnF8drISPMNMIlL\r
+CbRVgruzX3jauT8CoqAlytQoOPDPkFLvPPIMhcE2Nlb/62uJL6yPzHGxX44+Gv2p\r
+8RLyUqp+UMmTQzf7SMeHvWLDalpNyIstZwuNKEUgTX4PWCB/7RiFhsgOn7JS1Jk9\r
+KTTfYM0z3ULenC6bda6qzM+qGhr4yqJefLcfCz2YoLgxTXkAyVSUL0o3jLUjcgpt\r
+rylihTFANDjJa1xmac3QK2M+Ptfx0sBOSlJJyBxzmSwq+bHY4yPhKb6tFrMUQc1x\r
+vKLYU7yQYprMkfhKXftb+UD328e7j2E7SFq1eOLt6QFsyQuC2bnzsuSEeJCAm/s1\r
+fnBQZ2q+K4cfEp7W1mhFMaaxFt5h/jeffvJokQv35sOAHz7Kn688gswna6mtWfgS\r
+iP8JgtfaUN9rSKPYEG7PQ57cdp/6+tjKEWVDZCyYPzqk7tzKAHffynrG79zKdwS2\r
+9LY1NfhooGi7U+2y+LkbyfbcV84SY1U7PXeLs8fI2RYBIiPQh2sAzUAw+u/Wvij/\r
+vgYIO+gsHVHBZzYa7cLBfAQTAQoAJgUCU0Ak7QIbAwUJCWYBgAQLCQgHBRUKCQgL\r
+BRYCAwEAAh4BAheAAAoJEAyxjlIbJPP/yWwP/1jSywTOWjoHVhUhPVz25cmW72/P\r
+ujWankAvFFx2/HYb3QxTw3HC1huaE2Ubg+pW+EAmCL9DB26cNG8pXa3W2+6Hgc8G\r
+vfFcJ+DV2/DTr0ikYB5ZMm0BX5j2VI13B34XbH/HTummJljby9XXPaL3dMR3NW7P\r
+d26nbwu3BbUvyw2FG2Rfjfo+u7LenjkJpfdMhk6YD/rMYhz2pJMSXzLf6Bsiq+zD\r
+vjnl0whtHrrY5ZcgBtFzhta2EJ5Mj68cyUVzGiarvYbpqxoPxd3rND02ci2m9tcc\r
+tnP5b+pKiv5Q/3cvXSzrPkEU/0fsEFIFPlHjzB1CR5x+b1gy+bbrff4jZ9fuGWzi\r
+y+MSFqJpxGWwxuCtdg+7KyabljNGX3FgUVwLYgkEkOHVRANk5nCKsfsHoP/V+S7a\r
+V1GbqJu5hHkPrsWarufih8hoVJ/JjWRBKxYzo12c3G5soPkzaRTORwAfMB7UUfrd\r
+0wwDPGFR5rZZR0L0b/FmIJlZA02PoHo0TZfpvRHZ5BsepoaARus6bc+h6kx64WHr\r
+vo3THo9AnMWEl++jZJshCGguYt1WuWMF/6oO+2AOvfIErTq7GDGtVxrUoygfVNSy\r
+men4fBj8QMoStvi8DKNzdB7GLE730Ham+Nht7Z2KZSW4t9Y3qWT26sdi8bM6pyQC\r
+Q6YKiJgKDFoKomtgwsFcBBABAgAGBQJTpfE+AAoJEDYeVbUY1WqdbmEP/ij3s/pS\r
+3JbH20stlW8ryHbxxNUX2DDAO2r1kQyoeDiuFscxr2PvKVxk8b+/onnDk/egkooI\r
+bDrmr6PCw4MuIwI6U+irLuEMDULd+a1cMZFK2FAJ6TiTi2uhgf8j0DOfD5FVV2ia\r
+VD32QqL2USuwIjNUfIXylrSVEfl39c0C9YFWW6Lg7tZJ8kt/ls1XB5Ci2FSw0naW\r
+CvIiiXNBYx/9Yc6gXYV/7TyOjZU+T+p5RL1PyqAum/MzeUt7S1+IX1omqdZ1QL3J\r
+b3n+S8GX9G2aaAw+TGyLr27JgtlniMBTz7LqP6lkJGZxG0Su5t1uqkHb0bYSGVV/\r
+HE23oO8sZnfkIjLj2j7pnyk9N0bSV2VLxIfGSL47jOqgsjtG589b5cVemXLWV64N\r
+0K3odZ0byXiGkr3xMeb5LeSARUUCTjaNEktnI5Wu6UJhBxeNXLM6CxEGmAnDU4RG\r
+lOP4GOJznG3NkbsUYG/AwUCI5IywjcB5EkaIDNnFANYMYXQ7XhLFmJyL6iqLFv/N\r
+m+n+F131hDDKGulraPYGcjgdNa9zbKpB/tWfP9Mf7WiPM4TvJIoXvfye5s9NnOd1\r
+9axsuO661T2Pvwhw4S7bAyq5TNoDvKWqrVg9AKnffLko0l4pWB7VUwjx7SbA5RpR\r
+PnTFV0j3WUHfeTv9B5zzqIRx0OBGnIZ32KF1wsF8BBMBCgAmAhsDBAsJCAcFFQoJ\r
+CAsFFgIDAQACHgECF4AFAlNBrpYFCRLNjKkACgkQDLGOUhsk8/84EhAAjynS6UbJ\r
+Gold3ia5p2boG1GC7sOFVT+yvh4iiY9/WlrUvd1yUIreyPGzvVag1RCPyNSURlbF\r
+m0OuD0xSIGFPDhRJK8ljq3ynJxgDVQJP1WC+tYCJrMEOu8+uLAkUgH3T2l1H9o4x\r
+LcjhJxQu5H4Re/ZDM7fCn1JYwFdc/JEMvUBnHzF9qKmUQQSgXUTJNXnTUh2+Byin\r
+J4LKJyMT5NkwyLhlPT+hcVvg29Ck62z0LEiZc3XkY/LiN0G/Pyq8toLES03mMEvk\r
+oxtgTIYFPcdJK3azQ2iAtBK35QGLyxavMo8vgiAYC8jvhW+PUvQf/JeIGC0Hk6L+\r
+BQKDh09Cl8Y5sJPJA4Vm0lej2bc013/U3rwNV7dNWQJor+0rdJb7vlQzYqqYGIss\r
+Gqa/YmJpimFifhX2++NUbcZeXJ+Unl6ABKAeSO0PdwEnSXa/BI+ALf7JOumiy2ol\r
+83qXniYDxDc2wsMy/HxnDk/+NoUKSfVTPtybu9y/DGVfWl0ttBM/2QPiJngfwhjg\r
+RDGEMzs2x8cQl4AEKa0jHOixCmQlOXph7cdhTbP2r18RT4KWBHQ8yM6yRN+lkeD9\r
+UEYofCn8Ce3CgztI2FvBs13y7bhJAaTWiB0UXf6bAlTrUccORxlmSYOHF/XwI8HH\r
+kA7q1ojdfOrzNVkU2Mm2CLdLq04Rml/Y9rXCwXwEEwEKACYCGwMECwkIBwUVCgkI\r
+CwUWAgMBAAIeAQIXgAUCVAGlqAUJAqK0OwAKCRAMsY5SGyTz/5ruD/0S65O3M8B2\r
+J8QSic1KyB4+C0ALSj5IDxd9uATAdj6ebZ7jtlV7jZCCkovIQEoIJYR/4B/W7WzV\r
+2cbMetlaPN5ZabgKKSflpzPy4q62WN/1qVZPgncFUGkP1u5PSFdug5JEZF0Y9Afq\r
+juxGkSG1Tt9dTDPVH03aUUeDWKqQsDh+CFthziBdW0hhHElJ2umpLNuC4JNLZmk6\r
+09ealD0/bqJcPQN5JO7y5RZg1Ic5RlFPClVafEU5zy/pr1QhjBhGzkgxkvqXFrB1\r
+3FVYsyKB7xz4MZQWIg+UeyxFCn+SbjpFYi7Ro1swv96NrUfYRu4vUkkKm7wXqRXl\r
+kr0rzCfXUKAjLdJT63j0FDJwDtCwIr144jeyzh8Q7IwH2CU9Pm425H1IWcQ+VZBK\r
+6oNyAwcdx8LSiGzzHI8unv6d86G+aJYYI0eUgm7dAJ98daesVM6gSW1+5hQ866dG\r
+mBwF6h4GNdiQkiFeuVLyUrNUU6BpC9OXbjaJtyP5c3HGa0fvfuDZ8i+S3FXYQenU\r
+RP4BnNNauNpEINZdOx72EWXDQvgobWGUJSB752akwVXGmtZCMWF2vGilI+p0sXs/\r
+UhCkQKuKGQ7/V0ufX4DT24w4SaR9UCxBh7r/m8cf/JkOqfhU/bS9J6zj/tj2uUS6\r
+iKw+hU9+p6SxPl0wGhS77mp7XEsJJbpnWsLBXAQQAQIABgUCVMpgzQAKCRAob8y+\r
+k1Uc13ANEACJy//0JTEr6uBkO4PRVRWr3NT84ct/bh1KwirpTHKMUkPdLG/7MpBC\r
+TLdiG8rtHo91grNwy61l0FrT8VE+lxS4Rx4jmUZVm0Xtqtd7mEVMw9sXKlGriu8v\r
+jFPDUsM9MDTAhlXc6zqNM84i+jTKaz3zLvCuU4CHHUFaXxhHiol8PBR2O/Obd5vm\r
+8PP5Dt08pw4TQTA9MzCpS46PuYdjLyF8m/85b1/m2Ts4FNdvu1BcoiOhFTieFNNN\r
+1RYuvvOZobu00kNmqvAa4oIjSGaGVL/kkd3LuuSwsfTZcN9RcQMEoGIDjhHb9E9n\r
+RlB4q9XDXHY33ZlKwIdSTDO3mAgDGliayhXhs4hh67vce6s2+4MGilKN4JdWFLpi\r
+uEhgVnA+R/EAN6xsjWHkC9UCpfKZzr4kFftSNNlJF4eSgb+j74r5GdB1/HwirkoQ\r
+93bjJwDfxVsIn8oISEJC54frqbB0mujtkClm+SXrWCTfUWRCVwQbheLW1uM+458h\r
+rXEvf2h3TMhxaRX6xJvCqaSfDbq7wLrpj030algJRLT1iqzjnEorpnXXRzmTpyJn\r
+b3HnDj/u1xiSYbU2NL83yk9sFg2WXBlcMN7+Cqh4ObThD8/YOCCqzZQ9yM+ykGZx\r
+MdLX+mmvFkeM/aAC+D0Zd42q0pMCan7/mY+carjY3SxuoRMn22lTc8LBXAQQAQIA\r
+BgUCVMpicQAKCRDNh5PIjN4vIgxKD/9HU5Em5Y5jmEn8hRigytxxVyEFBz6FRPGC\r
+Iz8SWtJe5J1FvFDP0x9aX9pAAxOOyj1G2wvKcOW2abqVP+MsEJpISVw7vwZV0mvJ\r
+PUDS4KiFmh3gDJQA6IGJ0jeyq0O1j/p2Zm2djaTEsdWNr4ei0awDI/oj6lRO+bJ1\r
+vz+7nrLX20RJvn4Hv7TixxAxo1GnitYKuSjHZrsMk3N33UCyfv0f4G4njwDOGN2q\r
+3SwCkE3G8X8x5sXbm5zeYOKl0kSwtduDD0REOwIs223kLUh0wGjWW8O5DAK0jeSq\r
+eJfHyEzSeE6txgLSuxmFIT9kokdzzPMSGZuNi8UxlrkdpfOUHQE7fD13rt1GaTow\r
+A8HdT6uGReCSLMOEZw2gD6n/8v94NvQtjyl9eUxerZJ7vaUjmE37U9GwUOEobR2g\r
+/bxOAMMz8ELP2TWASoHyVKQIbGKlWhswfDILGIfAOY2jNHNk6Na3ibmpYm4JUFbg\r
+HqH/4/bGuXwOtiO0BYCMshLM5NjwXjD/8GkBMp3CuyW5FHwWy41C3CPM6F5tdoY9\r
+N/go6fzMfZ3VELdB/UcjIgdE+651kl6pFMvMulZT9vk7nm4nYf+s/z6GWKv6hZ5k\r
+4JoWpZH+aVW9w+bM90D8u+v5sdSet1CvlqFBhASxdniXS9wJvb4g2/8jUph59o8y\r
+7NKZOPXnlMLBXAQSAQIABgUCVMp83gAKCRCcxk5KKUWsN3ZbD/9wlNnOLurIIotx\r
+oEkI+FrhAGcv/3gPIJZqMgX3GNpecUJpgvSu6F+PIPKUnKCCKcdPI0ix9tmoXqLS\r
+okZ/VP7XaY9CZdDOuIf2Np7aiJGAzjxJYZnc882Oc0dsEC8AC8SYkA4bi1qxPIDb\r
+/LGVBDxAtZ/fiCQuFNEYHiAD9RivsMPODpRWyr7vLTai7DsArFaEZUPD4Rej0vxr\r
+Q7EmKgbLgY4h0DSmkvv2bB26gERyT6PsUXfVAEkv5JZAd0AC9STM6GiosALTBZH7\r
+E3+4gXIe9Hgt6AZ/wVfAr7bQ3yVh3PG6im2fqZi5+ruW+RVBs+/+NycMOfbUmG8Z\r
+U40fobuyTQkPbdH3+47aGGRdbMhUTvBUCsvQ7qhfW1iJyS9YHGTH0GfN9gkDSoTF\r
+bU2tP07RKAa6C10uvAeMjjWiolOzwK/dNz4iBH74+M9dx+FpSksnWg49rRD621tA\r
+7VE8YmdmMT1S18Wz+QpI0ZsSIUT0HwxfnwDX8T44KsMPsWMQjdn0mPXmn1kZPb3b\r
+lGlexGH3xZF2de2sYfOyet4OYOp93iVO+d+Mc/VDPGC++Nca6JMJWW+fz58giy3V\r
+xKwUMwTYn8y1kyH76bfK3O9RYXmywsWNgtW/+MKk/MLkiZSC9fX5nn/8qsmXYiME\r
+gtFqiSid4/iLo7EWaw6FZdFrFgeVdMLBXAQSAQoABgUCVMpqaQAKCRBgjc9jYM3Y\r
+6dHBD/kBjCLUJqgs3L6DU8TIonaZkGisAepbk4+8q0pLHnKiUqk09BGIyAmHJiaO\r
+tnlkXCEPF9WH854saMj/YL2tXTfG3fDBx0IwpKoUeiRGQCqySEiHXquJwCNJycHB\r
+qnc2A+mYemIidPOA7pDpywGWSbPb4TlT2wPSGBVfoSI4FQAJ56G0T1Dm+xbe8F11\r
+emnESM+fPcN6kptL54fD85ZxWUpL929l9pp4eW2N+et54yp1T56g7mYcdedF3e/v\r
+CsvUErdfWx/GOvvYVYq+NpCdqHHtIoM3pQLDpz+J/dxUfvOL0w/q0es5yGFuzVbm\r
+2yuRXXFZwYj8L6tSx+UChWFcKceqV88UuTQXg/G6iE1gmvCMrT0fldzaswBV0i4P\r
+Txer9A+KqBTOW3LjBGK8Ji2MUEhivvuu4fCeXzWg/do0cMYnlizBlDK2DGJoOKtI\r
+9KUy4UxfM+RYZzBpALoC5MCLDv9lim22PgU0mzU3AdeBSpkegk3gXoXaOs4D1HqF\r
+Npo9AaMnh6gGEb8SBHCZO2rWtcEJwYwZLcrac7mdC5iZeNAW+SUpn9DnPjmDEnrV\r
+MYXp8tWf9/Efpj6mPbFCOsjBeaRnObGVMGXebBGX/AihXqtzKIjjVEPez7PzuXIj\r
++4YFxek68jRrd8BjDZ6jhtv+g4AuFYEbaj5D3ex8ObE8HgyrGcLBYgQTAQoADAUC\r
+VMpqaQWDB4YfgAAKCRBEDMFF/bC4PxZNEACLvj1SWK4TSjYaiegDdsvcfei9p0ND\r
++4u61ssGe3CRzjdVJQBj1FBbgsBEm8gj3Z9Y1zf4A5siwtmDyb7cUp4eS/J8B+N8\r
+nYVhknozryoUmogBt7xN+2ILtQxwUjRKTvocuUMe2E6sUuTy/t6g3ePSSakRoFyF\r
+wMRld9NuhhIr17+YI5OfBe2TiwQWG4Q8PfF5ZwIowAGVurnu0SuVT7JngYM2CbO8\r
+gIBkCQh1i4YUwD2wH+2EQFm/TOusDn/62d+RAaIVHReuxBe0PATyqEhgM5GT6UsC\r
+8BzSI98he+r6KREwjKK4s8otdFyENriZkUkWajQSZnfl4eyQg7eTSO5y8hJv0CQc\r
+6RuPaeyYy9UYbUSxI+GYTrZH/d+UfxOvUI81KxaR5mvF/gew3mgurT9AQELBuD8z\r
+RGu97a12B+YTilOx6MvNvDCI0UGUDqEv2NHvoKeVBujCOJvxQsYaR46nR2B1pPEI\r
+5335EJZPVhS/FOjvJoWGAwqiNoRCF+IocWJg8hsDgZVgREoiiQXJ20mFT98stuyP\r
+abTq+GaFU9r0Oh/NE9J98QudzX1CFl8nsb+FHr0sprpENAxtZwKKQmaMRX5djeBY\r
+Xjos54+/FL2Nfzjpzde20JVuPxgSgyfe5GGaAyGkxqakdW29xedT0KmLNFITwQa+\r
+RgwtFKGIbhphw8JeBBMRCAAGBQJT3QW6AAoJEM6MEvMqLPEbk7QA/iPmsaIwXaOU\r
+kEMmZNRPDeeilh1Fm8SZauQK5ZE2CZ8qAP9wPRUPedwloUz6/exeeY+krXc+tpVx\r
+hhBxzgxgSTHJ88LBfAQTAQoAJgIbAwQLCQgHBRUKCQgLBRYCAwEAAh4BAheABQJU\r
+AaX5BQkGZRuMAAoJEAyxjlIbJPP/U3AP/iozxCVQ9HwtDH07v/C57uoUDRfHYKV1\r
+KUwGShZnREX7ruk8Bj0mTTrJqNH156Hbv/KGq+VmtbOwGY3OqlqCQE7NoSBpql3a\r
+3l0rHwkoGIyNcZ+2HcDHqL+GJXJV++gNQQTDixokCf0heobuoYzFIsLZp74m0dTi\r
+WcfnLszvrj0hhkd8C0IhHJxdsfR+Uf5f2JglXn4CvI80L+LGb6DYiCnUVkpqKVtv\r
+2PY6qRgdNnvR/DZ39rKwqyuIQ0RaDmZe7KK/SbkhGhK1W3xlkIPtrgL202rJYbP0\r
+XdgGP5wOEfNRG6gXkdNE4jFq1uvrF0mmD+DEdtVLWaiSYOhpn7rvduuHqMB91Q+9\r
+LktpT5wOvGjDrm9gmVx7AFqUDFb/vppztTAs3HU8863e/HYKZYxmi+fHsnWO+MR8\r
+iJ/BBd6dxWSdgItx/8tFhj+GRjvqgqfMq2fdOhVifnRLUi794oorKeUPyHOgy2yp\r
+nkvi27hiiuKYgtg/Pzg3liJrKsnuS5pKq1VbO2vQ9DaOuB2o13vOsBeUPclXFPhG\r
+2GjfcPSO1uydWOcQTWWG19za7iZMCTTItgyUI9zWD4iz5PdKZP/sY/QmK+eAG1R9\r
+d20WRd+pG+t8w9qFfzR73GvVBChBj1wpIhjZ4nVnW69w/ggrAsFcUi39PQcKsQtn\r
+7fD+4ytXG8VcwsGTBBMBCgA9AhsDBAsJCAcFFQoJCAsFFgIDAQACHgECF4AWIQS9\r
+P1r22V13xIqFr2gMsY5SGyTz/wUCWYzz0gUJCC4CZQAKCRAMsY5SGyTz/y79EACt\r
+leQmnhhjlEZg4PuEUAWPZK9AnFVkJAdwQbZccOQxoYNEotyQO1BKULSDnZRIzsjC\r
+KKdnXBUqKnMPwg5oIKsox0uSqsYoYtN1b3Vfq4Q+LAyCso8sgSMPHZELIswSIk/v\r
+Oq8ucisCsB1Wh0CyER0HlaWa33SrVOzlVQtZiiOFyLnLXB+dC6GJvdnEgjD8aPNB\r
+XYG5cBZdPPiCnaEOK5KIbcBcdpcWpxOznQRCK2GCVitkyeb5AJ6iNC/aw/boPg+t\r
+EZtAKU8FMZt76HKjS1BWva3iJx8fgqS+XqC7UczwjeTKGnsfJMq0qanPo0im6+mm\r
+v4lm6Y0A5zLcQe4j5DRzy26J+d8kLmH/0U6pysZC5MJxqAiekBaTxRPiFf8rBlyB\r
+8X4EATdcdEtpw3rESzVoKCMF//y6OlmMl4a+DRP4pD492jv9dafgq4vL2c50LjwE\r
+1JxmfdM2OGe2KaZC4nkvVBBrGRUp5gWscHaT+1Vl5wZoGOZJKpzbB7kUvF9hD4Ze\r
+8PYgxobgMBNeVtvYET+m3Z8zdRPDUjj3P5NF9t0zP+M4t4yH5YbtpXh4c1atptZq\r
+8f+YwMm+TqlVhhCIDcY8hRq69EuIX3Zhld8M0y39Bo/gz9Rh6bIz4RWGZYfzyve1\r
+qL0fz40uGlqfsv+lOHudegLIRcnv1rxTguBK10DhS8LBkwQTAQoAPQIbAwQLCQgH\r
+BRUKCQgLBRYCAwEAAh4BAheAFiEEvT9a9tldd8SKha9oDLGOUhsk8/8FAl0uD8cF\r
+CQvPHloACgkQDLGOUhsk8//9HQ//SATgLp/nTLwWVFO1NCbE8r+Vje16fSUu87cU\r
+lK35xQoR8tR+jj9Mbzi+3wgKGuqcovD+9j87GehP6YgJGibCt3ZNIDBR59Pru1a4\r
+W8V04dZFRAUcApjb+9xt4d9yXx+/33rfkSgLFgvgD5irZpEJe8iYa9lEsYievd7D\r
+IO40+5LLgo89fMZzwaVk2Q2uc6LUMNPeH71cjkoQ0l8G2CCq4a8SQDfLfKQ4jw+q\r
+D/s1jqmHuazsutlvqr+hZ7WPNdKMSkwBTbF4Ek/PgGPY7zf9cG9/AtIIBzYGTmUU\r
+42GBsF/+P7fdtJ2FA3yChUZR7Y68hAAvWJhdLAUS6aWZmmjPWlnbvUIgRcpHa1CX\r
+BpMlGa7Dr+YDpo5kYVp4txSynnL18kf+FXgxzgHFfpug5vT/eokjv5UXu69VNLfe\r
+ODYJpLAIgoa8ovAoZLhJuzhgLsJ06Tw+rpcFAX1by495L8ma2kzU+ul2KSxFY1Az\r
+kFbIe96ucQCWYYY9d7TKmB5XZpmnpv1XtpKrddbiQi4zLOgjzk82bWu8GBr13Mn+\r
+2Wv+Aa9HxagVusFzB+pNvZ/I1L3Kq1suavPHgMZMxxMlLrh8bjqAuBLx7N6U7CgT\r
+MsDLsaze0w2d7401aS6yKy+whnl0Gws5zVa20HauHNhtEgwKbRvywx/p8FpFTc/w\r
+RgaTwXjCwZMEEwEKAD0CGwMECwkIBwUVCgkICwUWAgMBAAIeAQIXgBYhBL0/WvbZ\r
+XXfEioWvaAyxjlIbJPP/BQJbTg6yBQkJ7x1FAAoJEAyxjlIbJPP/t50QAK1mxW6J\r
+V7Wsf/xpISXVjfuwuXHpU+lqToU815ECMp2oMDRhmZTulcEC3X2D4/PSv3yEv3EQ\r
+Ic26QMxPCzwMjGkdq31iXk2sc5ktJYocXZ7s+h2oF7A975vhLiJS3UtQ+hwP6zeL\r
+Nk3JBOlxc0WGzLqwvvsHPyF5RZKRG1SnI1cbNBdgEaa4kZY1rmYz5as2s1egQOI/\r
+ehBSUz70w49BptTjBPox7m3QPTBzL7VVkV3/T4Y0cw1irMlT6KvPVlTm8+3truc+\r
+nhQAqU7TD8cD7ee4q67PWFkTu2xZqZKIDaDTzGwOGRzNqTbwH71XuXixk+NLTzAP\r
+vPSE4VP0ZS1dzyrKOiMb8n7Sl5x6nm7ib29xHqFITboqyluhOGyplrltRXov6va6\r
+lJMvWugqtKIRRZy/2OLCxa/xyeqi2Q4uedavSjCwLxbFPQ3rfdJsbThrdmU436zv\r
+IXalew8zJSzUPKdn2Qw7r5Hw8zIVvaHxNFxGkURfYCdwTb0O4zxqWfOAU1N2Ilfh\r
+mAsCoF9qOVYbt3uSbpzWsnImvhxp0MXq79Cs86kccel/S8oSpctFdGXKvby7gt2D\r
+IG8IfNt6W4limqQwJjISULms49LjVPaopYHd/k4/kE8nxg2ZAE93MCocHxbUQSzE\r
+cKxbFc7Zc25mUwhJWLdaxCArKyHRzC79mRMGzsFNBFNAJO0BEADXkqJ9khCYra4T\r
+dIfeSQtFFcS+mSsfavGmVonuFC5105tcm4XEcIItNyXlIfpyt5Vpw3kLq6qaT6is\r
+zKXrLCCYDOtegsL7TkxwgCpqWNtwr/ZysjyvLLNdvWuEDGQsCCnlY/2aiCLsGY8v\r
+/qBFj3RRZgtUhoTHr9UMprrPnMdXfqIcSy5rzeo35EE7hiKjQLCwSo9Ss2EJqEeC\r
+ZEx/X0Gw0wVmwIBkrmgDWX6Euf46wF24L84V2DK6wPspNwXtekyPRdmCyANP9bi5\r
+gdMlr3tQgi+ZiMZC6XJwZECoS4zqFDKzogWC+ohazGCG7c0MqBeMMkEMxqZlLj+3\r
+M48MpWiPUS5rEVsBbUskuns+DstCr+R4XFRHsrCIbzubZmeCxJFhYirJKlB/MIf7\r
+U4qOjAWZgzCsE5qnWBB4zRb2amK1tDtFFa8DWYMAAXRl9xN+6px3BLL1ILMJmPzB\r
+Kr/DrV4H8jfnoC0RxEk6YKZUAbE6s89MUl0mINVqlU0DFrboNTSEvF3hip36yV0j\r
+CuFoJ0Sq7BT9UPIBE0dd/mt/phdxylnvcLVNoA0VNnDPvBBng7LwJ88WHB71fvBN\r
+CdN8VPC+orG23lgKj5ORUfdBVNVUw7xenvZe8ySzyyDvFCmXwqdYBFJoCLO6m1wN\r
+907hZaWLxWBjvxFOoV7yef1Lk2jeNwARAQABwsGGBBgBCgAmAhsMFiEEvT9a9tld\r
+d8SKha9oDLGOUhsk8/8FAmDQblgFCQ9q5WoAFAkQDLGOUhsk8/8JEAyxjlIbJPP/\r
+BjoQAJKs3rKUvL6sorVxJKYlcHRKtXqA4tkAADd86WfdJbYYjKf6YKUlKRfkExUN\r
+6m3O7mw68VnoH1k6ZOYS8XEasr6U+OO2bYdMZpG9lO5OqvaFMK0vxX7kmQy4Wpl5\r
+Ui3T05PEpA/1PYE777kbbZofq3NAJseZqQWOrnipFEXq/ZxYq3nmPWtOOpfCmKPl\r
+M7K2mw7rKOPmSJy6LKdwvG1srLaUeHeQDXVLDPjHEabW8AMELHuWjBWav+1DjrW3\r
+kZ2frYTU/YTuZOu5ho3lewm+rPQXUxTn/6u9Py6alp8AeLOnTCsZ28Z52FYaal0U\r
+eTG58VWRv1P3UNj5fdQ1ctGKHLbVQUVOEomijXMiybJJ65kQ7fTIKgKRXioopbmg\r
+cFeUg/rdEhIwezttwqYqivBXYnKzhSE7BsnPbrPesuLbedKI3WrQC29oOgHymm9o\r
+kciFzUteuD/OxC0ionItQat1bT1S06/R9QEJWe+PbMGAXHKmMD105AxJvkxunSYS\r
+XF3l1KzRTMvqHSKB2zu3Cc6VmaGHKqcIbUbIbrHcKBBJaJt1wANWqu+TGeEB/7ig\r
+vJ1FmkZOSu5tAaWXM7F4eglx3RhYW6k4SRm/JK+j6ZUDeYQBX3w+660IRamqWDNV\r
+1V8T51tlNkTm5Bz1rPAXT514y3LhIzYssQazOCBMifNFcTEf\r
+=44lh\r
+-----END PGP PUBLIC KEY BLOCK-----\r
src_base: "/home/r/src"
out_base: "projects"
projects:
- - name: "lilass"
+ - name: "bubblebox"
- name: "dyn-nsupdate"
- - name: "zonemaker"
- - name: "schsh"
+ - name: "git-mirror"
+ - name: "lilass"
- name: "rust-101"
src: "rust/rust-101"
- - name: "git-mirror"
+ - name: "schsh"
+ - name: "zonemaker"
defaults:
- scope:
{% if include.item.author %} • {{ include.item.author }}{% endif %}
{% if include.item.meta %} • {{ include.item.meta }}{% endif %}
{% if include.item.categories %} • {{ include.item.categories | category_links }}{% endif %}
- • <a href="https://git.ralfj.de/web.git/history/refs/heads/master:/ralf/{{include.item.path}}">Edits</a>
+ • <a href="https://git.ralfj.de/web.git/history/refs/heads/master:/personal/{{include.item.path}}">Edits</a>
• <a href="{{ include.item.url }}">Permalink</a>
{{ content }}
<p class="comment">
- Posted on <a href="{{ site.baseurl}}/blog">{{ site.blog.title }}</a> on {{ page.date | date: "%b %-d, %Y" }}. <br>
+ Posted on <a href="{{ site.baseurl}}/blog">{{ site.blog.title }}</a> on {{ page.date | date: "%b %-d, %Y" }}<!--
+ -->{% if page.license %} and licensed under <a href="{{ page.license-url }}" target="_blank">{{ page.license }}</a>{% endif %}. <br>
Comments? <a href="mailto:post+blog-AT-ralfj-DOT-de?subject=Blog post comment: {{ page.title | uri_escape }}">Drop me a mail</a><!--
-->{% if page.reddit %} or <a href="https://www.reddit.com/r{{ page.reddit }}" target="_blank">leave a note on reddit</a>{% endif %}<!--
-->{% if page.forum %} or <a href="{{ page.forum }}" target="_blank">leave a note in the forum</a>{% endif %}!
self.process(@name)
self.data ||= {}
- content = File.read(src, Utils.merged_file_read_opts(site, {})).each_line.to_a
+ content = File.read(src, **Utils.merged_file_read_opts(site, {})).each_line.to_a
self.data['layout'] = 'page'
self.data['title'] = content[0].match(/^#* ?(.*)\n$/)[1]
title: The Scope of Unsafe
categories: research rust
reddit: /rust/comments/4065l2/the_scope_of_unsafe/
+license: CC BY-SA 4.0
+license-url: https://creativecommons.org/licenses/by-sa/4.0/
---
I'd like to talk about an important aspect of dealing with unsafe code, that still regularly seems to catch people on the wrong foot:
It turns out that the underlying reason for this observation is also a nice illustration for the concept of *semantic types* that comes up in my [work on formalizing Rust]({% post_url 2015-10-12-formalizing-rust %}) (or rather, its type system).
Finally, this discussion will once again lead us to realize that we rely on our type systems to provide much more than just type safety.
-**Update (Jan 11th):** Clarified the role of privacy; argued why `evil` is the problem.
+**Update (2016-01-11):** Clarified the role of privacy; argued why `evil` is the problem.
<!-- MORE -->
There will be back and forth between various alternatives, and in the end, hopefully there will be a compromise that everybody can accept.
Happy safe hacking!
+
+#### Footnotes
title: How MutexGuard was Sync When It Should Not Have Been
categories: rust research
reddit: /rust/comments/6gavfe/how_mutexguard_was_sync_when_it_should_not_have/
+license: CC BY-SA 4.0
+license-url: https://creativecommons.org/licenses/by-sa/4.0/
---
A couple of weeks ago, our ongoing effort to [formalize Rust's type system]({% post_url 2015-10-12-formalizing-rust %}) lead to us actually discovering a bug in the Rust standard library:
That said, such decisions are of course going to go through the usual RFC process.
It's certainly possible that someone comes up with a compromise that preserves some of the usefulness of auto traits, without putting safety at risk.
+#### Footnotes
If you have any comments or thoughts on this, please join the [discussion in the Rust forums](https://internals.rust-lang.org/t/sharing-for-a-lifetime/6675)!
I'd also be interested in feedback on how understandable this post is; this is my first attempt at translating research results into a blog post.
+
+#### Footnotes
Crucially, **pinning does not provide immovable types**!
Data is only pinned after a `Pin<T>` pointing to it has been created; it can be moved freely before that happens.
-The [corresponding RFC](https://github.com/rust-lang/rfcs/blob/master/text/2349-pin.md) explains the entirey new API surface in quite some detail: [`Pin`](https://doc.rust-lang.org/nightly/std/mem/struct.Pin.html), [`PinBox`](https://doc.rust-lang.org/nightly/std/boxed/struct.PinBox.html) and the [`Unpin`](https://doc.rust-lang.org/nightly/std/marker/trait.Unpin.html) marker trait.
+The [corresponding RFC](https://github.com/rust-lang/rfcs/blob/master/text/2349-pin.md) explains the entirey new API surface in quite some detail: [`Pin`](https://doc.rust-lang.org/1.27.0/std/mem/struct.Pin.html), [`PinBox`](https://doc.rust-lang.org/1.27.0/std/boxed/struct.PinBox.html) and the [`Unpin`](https://doc.rust-lang.org/1.27.0/std/marker/trait.Unpin.html) marker trait.
I will not repeat that here but only show one example of how to use `Pin` references and exploit their guarantees:
{% highlight rust %}
#![feature(pin, arbitrary_self_types, optin_builtin_traits)]
fn init(mut self: Pin<SelfReferential>) {
let this : &mut SelfReferential = unsafe { Pin::get_mut(&mut self) };
// Set up self_ref to point to this.data.
- this.self_ref = &mut this.data as *const i32;
+ this.self_ref = &this.data as *const i32;
}
- fn read_ref(mut self: Pin<SelfReferential>) -> Option<i32> {
- let this : &mut SelfReferential = unsafe { Pin::get_mut(&mut self) };
+ fn read_ref(self: Pin<SelfReferential>) -> Option<i32> {
+ let this : &SelfReferential = &*self;
// Dereference self_ref if it is non-NULL.
if this.self_ref == ptr::null() {
None
I spelled out the details [in the RFC issue](https://github.com/rust-lang/rfcs/pull/2349#issuecomment-372109981).
So, if we want to declare that shared pinning is a typestate in its own right---which overall seems desirable---do we want it to be restricted like this due to an implementation detail of arbitrary self types?
-**Update:** @Diggsey [points out](https://github.com/rust-lang/rfcs/pull/2349#issuecomment-379230538) that we can still have a `PinRefCell` with a method like `fn get_pin(self: Pin<PinRefCell<T>>) -> Pin<T>`, as long as the `PinRefCell` never gives out mutable references. So it turns out that combining interior mutability and pinning should work fine. Later, @glaebhoerl suggested we can even [combine `RefCell` and `PinRefCell` into one type if we dynamically track the pinning state](https://internals.rust-lang.org/t/safe-intrusive-collections-with-pinning/7281/11?u=ralfjung). **/Update**
+**Update:** @Diggsey [points out](https://github.com/rust-lang/rfcs/pull/2349#issuecomment-379230538) that we can still have a `PinRefCell` with a method like `fn get_pin(self: Pin<PinRefCell<T>>) -> Pin<T>`, as long as the `PinRefCell` never gives out mutable references. So it turns out that combining interior mutability and pinning should work fine. Later, @glaebhoerl suggested we can even [combine `RefCell` and `PinRefCell` into one type if we dynamically track the pinning state](https://internals.rust-lang.org/t/safe-intrusive-collections-with-pinning/7281/11?u=ralfjung). **/Update**
## 4 Conclusion
Anyway, as usual, please [let me know what you think](https://internals.rust-lang.org/t/safe-intrusive-collections-with-pinning/7281)!
+**Update:** Years later, I finally realized that there still is a major problem with intrusive collections -- it is basically impossible to have a by-reference iterator!
+With the collection not actually owning the memory the elements are stored in, it is basically impossible to guarantee that during the iteration, elements do not get removed from the collection, which would invalidate the reference.
+However, for many use-cases (like an intrusive queue) it is sufficient to just support by-value operations such as "enqueue" and "dequeue" over such a collection, and those can be given an entirely safe interface.
+If iteration is required, then more advanced [techniques to control sharing](https://plv.mpi-sws.org/rustbelt/ghostcell/) seem to be needed. **/Update**
+
#### Footnotes
(`/var/log/mailman/subscribe` on Debian) to see if any illegitimate requests
still make it through, but unless you site is really big I'd be rather surprised
to see bots being able to answer site-specific questions.
+
+**Update:** With Mailman 2.1.30, this patch is now included upstream.
+The `CAPTCHAS` format is slightly different than above to support multiple languages; consult the Mailman documentation for further details. **/Update**
However, `Arc` is one of those cases where the overhead induced by an exclusive lock is just way too big, so it is worth using a more optimized, unsafe implementation.
As such, you are going to find plenty of atomic accesses in [the source code of `Arc`](https://github.com/rust-lang/rust/blob/c0955a34bcb17f0b31d7b86522a520ebe7fa93ac/src/liballoc/sync.rs#L201).
-And it turns out, as Hai and Jacques-Henri noticed when attempting to prove correctness of [`Arc::get_mut`](https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#method.get_mut), that there is one place where `Relaxed` as used as an ordering, [but it really should have been `Acquire`](https://github.com/rust-lang/rust/pull/52031).
+And it turns out, as Hai and Jacques-Henri noticed when attempting to prove correctness of [`Arc::get_mut`](https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#method.get_mut), that there is one place where `Relaxed` was used as an ordering, [but it really should have been `Acquire`](https://github.com/rust-lang/rust/pull/52031).
Discussing the exact details of the bug would probably fill another blog post (`Arc` is *really* subtle), but the high-level story is exactly like in our example above: Thanks to `Acquire`, an ordering is induced between the code that follows the `get_mut` and the code in another thread that dropped the last other `Arc`, decrementing the reference count to 1.
The PR that fixed the problem contains [some more details in the comments](https://github.com/rust-lang/rust/pull/52031/files).
With `Relaxed`, no such ordering is induced, so we have a data race.
Hai and Jacques-Henri are currently working on remedying this particular simplification by extending the first RustBelt paper to also cover weak memory, and that's when they ran into this problem.
**Update:** Turns out Servo has a [copy of `Arc`](https://doc.servo.org/servo_arc/index.html) that [has the same problem](https://github.com/servo/servo/issues/21186). So we got two bugs for the price of one. :) **/Update**
+
+#### Footnotes
---
title: "Pointers Are Complicated, or: What's in a Byte?"
-categories: internship rust
+categories: internship rust programming
forum: https://internals.rust-lang.org/t/pointers-are-complicated-or-whats-in-a-byte/8045
---
}
{% endhighlight %}
It would be beneficial to be able to optimize the final read of `y[0]` to just return `42`.
+C++ compilers regularly perform such optimizations as they are crucial for generating high-quality assembly.[^perf]
The justification for this optimization is that writing to `x_ptr`, which points into `x`, cannot change `y`.
+[^perf]: To be fair, the are *claimed* to be crucial for generating high-quality assembly. The claim sounds plausible to me, but unfortunately, I do not know of a systematic study exploring the performance benefits of such optimizations.
+
However, given how low-level a language C++ is, we can actually break this assumption by setting `i` to `y-x`.
Since `&x[i]` is the same as `x+i`, this means we are actually writing `23` to `&y[0]`.
Of course, that does not stop C++ compilers from doing these optimizations.
-To allow this, the standard declares our code to have [undefined behavior]({% post_url 2017-07-14-undefined-behavior %}).
+To allow this, the standard declares our code to have [undefined behavior]({% post_url 2017-07-14-undefined-behavior %}).[^0]
+
+[^0]: An argument could be made that compilers should just not do such optimizations to make the programming model simpler. This is a discussion worth having, but the point of this post is not to explore this trade-off, it is to explore the consequences of the choices made in C++.
First of all, it is not allowed to perform pointer arithmetic (like `&x[i]` does) that goes [beyond either end of the array it started in](https://timsong-cpp.github.io/cppwp/n4140/expr.add#5).
Our program violates this rule: `x[i]` is outside of `x`, so this is undefined behavior.
Such interpreters have a hard time dealing with operations of the form "just choose any of these values" (i.e., non-deterministic operations), because if they want to fully explore all possible program executions, that means they have to try every possible value.
Using `Uninit` instead of an arbitrary bit pattern means miri can, in a single execution, reliably tell you if your programs uses uninitialized values incorrectly.
+**Update:** Since writing this section, I have written an entire [post dedicated to uninitialized memory and "real hardware"]({% post_url 2019-07-14-uninit %}) with more details, examples and references. **/Update**
+
## Conclusion
We have seen that in languages like C++ and Rust (unlike on real hardware), pointers can be different even when they point to the same address, and that a byte is more than just a number in `0..256`.
**Update:**
Since publishing this post, I have written [another]({% post_url 2018-11-16-stacked-borrows-implementation %}) blog post about a slightly adjusted version of Stacked Borrows (the first version that actually got implemented).
That other post is self-contained, so if you are just interested in the current state of Stacked Borrows, I suggest you go there.
+Only go on reading here if you want some additional historic context.
**/Update**
## 1 Validity-based vs. Access-based
This is an invariant that all the unsafe code implementing `Vec` maintains, and it is the invariant that the safe API surface of `Vec` expects the outside world to uphold.
The reason this works is that the fields mentioned in this invariant are all private, so safe code cannot break the invariant and use that broken invariant to cause havoc.
Again, the safety invariant is about ensuring safe execution of safe code.
-Unsafe code can of course break this invariant, but then it us just Doing It Wrong (TM).
+Unsafe code can of course break this invariant, but then it is just Doing It Wrong (TM).
Thanks to privacy and an abstraction barrier, types in Rust can define *their own safety invariant*, and they can then expect the outside world to respect that invariant.
As we have seen with `Vec`, when generic types are involved, these custom safety invariants will often have a "structural" element in that being safe at `Vec<T>` is defined in terms of being safe at `T`.
forum: https://internals.rust-lang.org/t/stacked-borrows-implemented/8847
---
-Three months ago, I proposed [Stacked Borrows]({% post_url
-2018-08-07-stacked-borrows %}) as a model for defining what kinds of aliasing
-are allowed in Rust, and the idea of a [validity invariant]({% post_url
-2018-08-22-two-kinds-of-invariants %}) that has to be maintained by all code at
-all times. Since then I have been busy implementing both of these, and
+Three months ago, I proposed Stacked Borrows as a model for defining what kinds
+of aliasing are allowed in Rust, and the idea of a [validity invariant]({%
+post_url 2018-08-22-two-kinds-of-invariants %}) that has to be maintained by all
+code at all times. Since then I have been busy implementing both of these, and
developed Stacked Borrows further in doing so. This post describes the latest
version of Stacked Borrows, and reports my findings from the implementation
phase: What worked, what did not, and what remains to be done. There will also
be an opportunity for you to help the effort!
+This post is a self-contained introduction to Stacked Borrows. Other than
+historical curiosity and some comparison with my earlier work on
+[Types as Contracts]({% post_url 2017-07-17-types-as-contracts %}) there is no
+reason to read the [original post]({% post_url 2018-08-07-stacked-borrows %}) at
+this point.
+
<!-- MORE -->
What Stacked Borrows does is that it defines a semantics for Rust programs such
code.
I will explain these rules again in this post. The explanation is not going to
-be the same as last time, not only because it changed a bit, but also because I
-think I understand the model better myself now so I can do a better job
-explaining it.
+be the same as [last time]({% post_url 2018-08-07-stacked-borrows %}), not only
+because it changed a bit, but also because I think I understand the model better
+myself now so I can do a better job explaining it.
Ready? Let's get started. I hope you brought some time, because this is a
rather lengthy post. If you are not interested in a detailed description of
--- /dev/null
+---
+title: "Rust 2019: Solid Foundations for Unsafe Code"
+categories: rust
+---
+
+It is that time of the year again where our feedreaders are filled with people's visions for what Rust should become in the next year.
+Coming up with a vision is not exactly my strong suit, but it is probably something I should learn, so here's my episode of the "Rust 2019" blog post series.
+
+I think in 2019, we should make a coordinated effort to improving the *foundations for writing unsafe code*.
+Of course my particular Rust bubble is mostly about unsafe code, as you will know if you have read some of my previous posts -- but safety is a core value proposition of Rust, and the entire ecosystem rests on a foundation of crates that make heavy use of unsafe code, so I believe that caring about unsafe code is an important piece to Rust's overall success.
+
+<!-- MORE -->
+
+## Foundations for unsafe code
+
+What do I mean by "improving the foundations" of unsafe code?
+There are many questions that frequently come up when writing unsafe code, like:
+
+* What are the rules around handling uninitialized or otherwise invalid data?
+* How can I create pointers to fields of packed structs?
+* When can I turn raw pointers into references, and what can I do with these references?
+* When do I need `UnsafeCell`?
+* What can I do with a `union`?
+
+I am sure there are more, these are just some questions that regularly show up in my inbox because someone @mentions me.
+What all of these questions have in common is that *we don't know the full answer*, and even the parts of the answer that we do know are hard to figure out for someone who's new to this.
+
+So a key ingredient to providing better foundations for writing unsafe code is to start answering some of these questions, and to figure out which other questions need answering.
+One issue that any attempt to answer any one of these questions precisely will run into is that we do not have a proper specification for what a Rust program *does* when it is executed -- the semantics of Rust are basically defined by "whatever the generated LLVM IR does", and that's not a very solid foundation at all.
+As a consequence, we often even lack the terminology to make precise statements about what unsafe code can and cannot do.
+What we need is a specification of at least a large enough fragment of Rust to serve as the framework in which these other discussions can take place.
+
+But that's just the beginning: we can't just dump a bunch of rules onto programmers and let them deal with the rest.
+We need to help them write code that follows the rules.
+To this end, we need more documentation and teaching material -- things like the [Rustonomicon](https://doc.rust-lang.org/nightly/nomicon/).
+
+But I strongly believe that we can't leave it at that.
+We should also provide *tooling* that helps programmers check their code for rule violations.
+I believe that this is extremely important: it does not just help programmers increase confidence in their unsafe code, it also helps them learn what the rules even are.
+We all learn by making mistakes, but that only works if you *know* that you made a mistake -- and with unsafe code, that's very hard to figure out.
+And, last but not least, creating such tooling will uncover ambiguities in the rules and raise new questions about corner cases that we forgot to consider, and thus help flesh out the rules themselves.
+
+## Where are we at?
+
+Some of the things I described above are already happening, but things are moving slowly and more hands are always needed!
+
+The [unsafe code guidelines effort](https://github.com/rust-rfcs/unsafe-code-guidelines/) (UCG) is slowly but steadily chipping away at the kind of questions I raised, determining whether we can have a consensus among the small group of people that is actively participating in these discussions.
+This is an interesting mix of descriptive and normative work: we are trying to cater patterns that already existing unsafe code follows and cast them into rules that future code will have to follow.
+Of course, anything we do still has to go through the RFC process before it becomes normative, and I am sure new interesting questions will come up then.
+Right now, we are discussing ["validity invariants"](https://github.com/rust-rfcs/unsafe-code-guidelines/blob/master/active_discussion/validity.md), which describe the assumptions the compiler is allowed to make about the contents of a variable based on its type.
+This just started, so now is a perfect time to join the effort!
+
+A [new approach for handling uninitialized data](https://github.com/rust-lang/rust/issues/53491) is available in nightly, replacing the old `mem::uninitialized` that turned out to be impossible to design reasonable rules for.
+At this point, this is mostly blocked on [bikeshedding the API](https://github.com/rust-lang/rust/pull/56138) and figuring out whether we want to allow or forbid unsafe code to create references to uninitialized data (without reading from them) -- the latter being one of the most interesting open questions around "validity invariants" that the UCG is discussing.
+
+There is an [accepted RFC](https://github.com/rust-lang/rfcs/blob/master/text/2514-union-initialization-and-drop.md) for resolving the open questions around the interaction of `union` and `Drop`, and a [work-in-progress partial implementation](https://github.com/rust-lang/rust/pull/56440) of that RFC.
+
+There is an [RFC under discussion](https://github.com/rust-lang/rfcs/pull/2582) for resolving the situation around references to fields of packed structs, that also helps with other questions around references in unsafe code.
+This RFC could need some pushing over the finish line, and then it needs implementing.
+
+In terms of tooling, there is [Miri](https://github.com/solson/miri/) (also available on the [Rust Playground](https://play.rust-lang.org/)), which could need quite some love to improve the error messages, to track more information during execution for better diagnostics, to make it run faster and to support more things that test suites often do (like panics, or seeding RNGs from the OS).
+
+Rust also, in principle, supports some of the LLVM sanitizers, namely asan, tsan, msan and lsan.
+However, using them [still seems to run into some issues](https://github.com/google/mundane/issues/15), and the [rust-san project](https://github.com/japaric/rust-san) seems to lay dormant.
+We should aim at making it standard practice for crates to run these sanitizers as part of their CI.
+Even if the crate doesn't use unsafe code itself, it might have a dependency that does, so using the sanitizer here helps improve the test coverage of that dependency.
+This won't catch violations of Rust-specific rules, and to my knowledge these sanitizers all have false negatives (meaning they miss bugs in their domain), but they *do* find many issues and that goes a long way.
+
+Of course, it would be awesome to have a Rust-specific sanitizer as well.
+There is some low-hanging fruit here, like having a way to compile your program with assertions guarding unchecked slice accesses, or verifying that pointers are sufficiently aligned -- and then there is a long list of interesting checks that are increasingly hard to implement.
+
+And then there are all the things I did not think of that could help unsafe code authors do their work with confidence, and reduce the number of mistakes that people inevitably make when writing unsafe code.
+I am sure there is a lot of possibility here for API design to improve the overall reliability of unsafe code.
+
+So, as you can see, many things are happening, so I think there is some real chance that we can make serious progress on this topic in 2019.
+Let's make it happen, together!
--- /dev/null
+---
+title: "All-Hands 2019 Recap"
+categories: rust
+reddit: /rust/comments/apreqi/ucgmiri_allhands_2019_recap/
+---
+
+Last week, I was in Berlin at the Rust All-Hands 2019.
+It was great!
+I will miss nerding out in discussions about type theory and having every question answered by just going to the person who's the expert in that area, and asking them.
+In this post, I am summarizing the progress we made in my main areas of interest and the discussions I was involved in---this is obviously just a small slice of all the things that happened.
+
+<!-- MORE -->
+
+## Validity Invariants and `MaybeUninit`
+
+We had a session to talk about [validity invariants](https://github.com/rust-rfcs/unsafe-code-guidelines/blob/master/active_discussion/validity.md) ([meeting notes here](https://paper.dropbox.com/doc/Topic-Validity-Invariants--AXWuzG6GFwiTyUDukuPV7vDmAg-Wpl6mhrqNBStyrCHwhpYI#:uid=357396828564014720965680&h2=NOTES)).
+No firm conclusions were reached, but we got input from people that haven't been part of the discussions inside the UCG (unsafe code guidelines) WG yet and that was very interesting.
+It seems that deciding about the [validity invariant for references](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/77) and (to a lesser extend) the [validity invariant for unions](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/73) will be a slow process---there are lots of different options, and some of the trade-offs come down to prioritizing one value over another (like prioritizing checkable UB over optimizations, or vice versa).
+Probably the closest to a conclusion we reached was that [uninitialized integers should probably not be UB](https://github.com/rust-rfcs/unsafe-code-guidelines/issues/71#issuecomment-460599057) until you perform any operation on them.
+
+That said, it also got clear that the sooner we can stabilize (parts of) [`MaybeUninit`](https://github.com/rust-lang/rust/issues/53491), the better.
+And as @japaric pointed out, we do not have to wait until all of these questions get answered!
+We can stabilize a subset of the API, and leave (in particular) `get_ref` and `get_mut` out of that.
+I think the biggest blocker for that is to get [RFC 2582](https://github.com/rust-lang/rfcs/pull/2582) accepted (which is very close to beginning its FCP).
+Once that is done, I think I'll just beef up the documentation a bit and submit a stabilization PR.
+
+What worries me a bit is that we did not get much feedback from people using the API.
+It seems like everyone is waiting for it to become stable, but of course then it will be too late to fix API mistakes that we made!
+Given the importance of the API for certain classes of unsafe code, I think it would be great to get some more feedback before we set things in stone.
+
+To finish up the API, I went ahead and renamed the method that you call when a `MaybeUninit` is fully initialized to [`into_initialized`](https://doc.rust-lang.org/nightly/std/mem/union.MaybeUninit.html#method.into_initialized), which seems, uh, good enough?
+I also like `assume_initialized` but that does not sound like it would return anything.
+One thing I wonder about is whether some of these methods should be "downgraded" to functions, forcing callers to write `MaybeUninit::into_initialized` or so.
+If you have an opinion on this, please [join us in the tracking issue](https://github.com/rust-lang/rust/issues/53491)!
+
+## Uninitialized Memory
+
+@stjepang, @Amanieu and me talked a bit about [uninitialized data in atomic operations](https://github.com/crossbeam-rs/crossbeam/issues/315), and how to make sense of the [C++ spec](https://en.cppreference.com/w/cpp/atomic/atomic/compare_exchange), in particular in the presence of mixed atomic and non-atomic accesses to the same object using [`atomic_ref`](https://en.cppreference.com/w/cpp/atomic/atomic_ref).
+We did not make any notable progress, other than the possibility of a sound `AtomicCell` *without* `get_mut`.
+
+Another discussion was around using [`Read::read`](https://doc.rust-lang.org/nightly/std/io/trait.Read.html#tymethod.read) to initialize memory.
+The problem here is that with an *unknown* `Read` implementation, we cannot just pass in an uninitialized buffer.
+Uninitialized data is strictly different from any particular bit pattern, and using it in any non-trivial way is UB.
+Since there is no way to know what the unknown `Read::read` function does, we cannot pass such bad data to it.
+In [my terminology](({% post_url 2018-08-22-two-kinds-of-invariants%})), even though integers with uninitialized data are (likely) *valid*, uninitialized data violates the *safety invariant* of our integer types.
+To solve this, the proposal is to add a [`freeze` method](https://github.com/rust-lang/rust/pull/58363) that turns all uninitialized memory into arbitrary but fixed bits.
+Such frozen memory admits fewer optimizations (because it must observably have a consistent value when accessed multiple times), but it is also safe to use at any integer type (for the same reason).<br>
+The main reason I like this proposal is that it officially acknowledges the subtle but important distinction between "arbitrary bit pattern" and "uninitialized", and that should help a lot with teaching these concepts to unsafe code authors.
+
+## Stacked Borrows
+
+Finally, there was some discussion about [Stacked Borrows](https://github.com/RalfJung/unsafe-code-guidelines/blob/stacked-borrows/wip/stacked-borrows.md) ([meeting notes here](https://paper.dropbox.com/doc/Topic-Stacked-borrows--AXUlwiiCbkUsrhfLWPy43~yiAg-2q57v4UM7cIkxCq9PQc22#:uid=304192866787878821395738&h2=NOTES)).
+The feedback was very positive, and (to my surprise) everybody agreed that the things I had to change in the standard library to make it conform with the model were appropriate bugfixes.
+@arielb1 and @matthewjesper helped a lot with detailed discussions about some of the [remaining issues](https://github.com/rust-rfcs/unsafe-code-guidelines/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-stacked-borrows) in the model, as well as figuring out a plan to fix a problem [found by @Amanieu](https://github.com/solson/miri/issues/615).
+However, [two-phase borrows remain a problem](https://github.com/rust-lang/rust/issues/56254).
+
+## Miri now tests libcore and liballoc
+
+I am particularly happy about the progress I made on [Miri](https://github.com/solson/miri/) during this week.
+With help from @eddyb and @alexcrichton, I got Miri to run the libcore and liballoc unit test suites.
+This has already helped to [uncover a bug](https://github.com/rust-lang/rust/pull/58200) and some [subtle undocumented invariants](https://github.com/rust-lang/rust/issues/58320) in the standard library, as well as [several](https://github.com/solson/miri/pull/625) bugs [in Miri](https://github.com/rust-lang/rust/pull/58324).
+I have [set things up](https://github.com/RalfJung/miri-test-libstd) such that Miri will run these tests every night, giving us higher confidence that the standard library is free of undefined behavior---or rather, that the parts of the standard library that are covered by unit tests are free of undefined behavior that Miri can detect.
+I've been partially working towards this goal for several months now, so it is really satisfying to see it all come together. :-)
+
+Moreover, @Amanieu ran Miri on hashbrown, not discovering a bug in his crate but running into [several](https://github.com/solson/miri/issues/612) bugs [in Miri](https://github.com/solson/miri/issues/614) which I then fixed.
+
+And last but not least, Miri can now [pass arguments to the interpreted program](https://github.com/solson/miri/pull/624), which is particularly useful when running test suites to run only the test one is currently debugging.
+
+I think that's it!
+Lots of exciting progress as well as lots of grounds for further discussion.
+This won't get boring any time soon. :D
--- /dev/null
+---
+title: "Sandboxing All The Things with Firejail"
+categories: sysadmin
+---
+
+Sometimes, I run software that I trust less.
+All software I use on a daily basis is open-source, but there is still closed-source software I run occasionally, like video games.
+Unfortunately, the usual Unix security model does not protect against such software misbehaving, as illustrated by [this XKCD](https://www.xkcd.com/1200/).
+There is no good reason that these applications should have access to all my personal information and secret keys that I have stored on my system, but without proper application isolation (like we are used to on mobile devices nowadays) nothing stops them from leaking all this data.
+I didn't want to wait until such technology becomes the default on Linux desktops, so I did some research for ways to sandbox existing applications and ended up with [Firejail](https://github.com/netblue30/firejail).
+
+<!-- MORE -->
+
+Firejail is packaged for Debian and other distributions, so you should be able to install it with your distribution's package manager.
+It runs an application in a sandbox, the details of which are configured in a *profile*.
+Firejail comes with profiles for many applications.
+For example, `firejail firefox` will start Firefox inside the sandbox.
+This adds an extra layer of security around Firefox' security mechanisms.
+
+However, I have found that my setup is sufficiently non-standard that I often have to customize the profiles.
+The most complex piece of the profile is to configure which parts of the file system the application will be able to access.
+Unfortunately, the configuration language can be quite hard to use---it is often difficult to find out why access to a particular file does not work.
+Also, the whitelisting mechanism is fairly inflexible: you cannot control which directory gets whitelisted; instead, when you whitelist something in `/home/user/foo`, whitelisting gets turned on for all of `/home` (and similar for a bunch of other hard-coded "whitelisting roots").
+But with some experimentation, you'll get things to work eventually.
+
+For example, if there are extra directories that no sandboxed application should be able to access, you can add them in `/etc/firejail/disable-common.local`.
+I am blacklisting all my personal files:
+```
+# Personal files
+blacklist ${HOME}/Documents
+```
+This will affect all sandboxes, because `disable-common.inc` is included pretty much everywhere, and that includes `disable-common.local`.
+
+## Custom Profiles
+
+Firejail cannot come with profiles for all applications, so sometimes you have to write your own.
+For example, I use one profile for all video games as well as TeamSpeak, which is based on the profile for Steam and looks like this (I store it in `/etc/firejail/gaming.profile`):
+```
+# Steam profile (applies to games/apps launched from Steam as well)
+noblacklist ${HOME}/.steam
+noblacklist ${HOME}/.steampath
+noblacklist ${HOME}/.steampid
+noblacklist ${HOME}/.local/share/steam
+noblacklist ${HOME}/.wine
+noblacklist ${HOME}/.ts3client
+include /etc/firejail/disable-common.inc
+include /etc/firejail/disable-programs.inc
+include /etc/firejail/disable-passwdmgr.inc
+
+caps.drop all
+netfilter
+nonewprivs
+noroot
+protocol unix,inet,inet6,netlink
+# wine/games don't work properly with seccomp
+#seccomp
+
+# give access to some directories
+read-write ${HOME}/.cache/winetricks
+read-write ${HOME}/.ts3client
+read-write ${HOME}/.steam
+read-write ${HOME}/bin/teamspeak3
+whitelist /mnt/store/r/games
+# -> implicitly, the rest of /mnt is blocked
+```
+Obviously, you will have to adjust the paths.
+If one particular application needs something that got globally blacklisted, you can use `noblacklist`.
+However, you might also have to `whitelist` it if something else sets up a whitelist for this directory.
+Also, there seems to be no way to have something on the `blacklist` globally but then make it `read-only` for a particular sandbox only.
+(This is what I mean by the configuration language being hard to use.)
+
+Next, it is a good idea to test the profile.
+You can start a shell inside the sandbox with:
+```
+firejail --profile=/etc/firejail/gaming.profile bash
+```
+Now you can do things like `ls ~/.ssh` to make sure your secret keys are not accessible in the sandbox.
+Also test accessing the directories where the games are stored.
+Use `exit` to leave the sandbox.
+
+I have created a file `~/bin/gamejail` to conveniently start an application inside the sandbox:
+```
+#!/bin/bash
+exec firejail --profile=/etc/firejail/gaming.profile "$@"
+```
+Now you just have to wrap the call to run the application inside `gamejail`, for example `gamejail steam`.
+
+That's all I got.
+I hope this little tutorial helps you to make your system a bit more secure by restricting application privileges.
+Firejail is far from perfect, but it works quite well and it is extremely customizable.
+In case you have trouble configuring the sandbox properly, I have found it very helpful to look at all the examples that are shipped in `/etc/firejail`.
--- /dev/null
+---
+title: "Miri available as rustup component"
+categories: rust
+reddit: /rust/comments/b5um0v/miri_available_as_rustup_component/
+---
+
+Running your unsafe code test suite in Miri has just gotten even easier: Miri is now available as a `rustup` component!
+Huge thanks to @oli-obk and @mati865 who made this happen.
+
+Miri can run your test suite in a very slow, interpreted mode that enables it to test for undefined behavior: if an out-of-bounds array access happens, uninitialized memory gets used the wrong way or a dangling raw pointer gets dereferenced, Miri will notice and tell you.
+However, Miri cannot execute all programs, and it also cannot detect all forms of misbehavior.
+For further information, please check out [Miri's README](https://github.com/rust-lang/miri#readme).
+
+<!-- MORE -->
+
+Installation (only on nightly toolchains) is now as simple as
+```
+rustup component add miri
+```
+After installing Miri, you can run your crate's test suite in it with `cargo miri test`.
+I suggest you do `cargo clean` first because Miri needs to build its own standard library, and rustc can get confused when crates built with different standard libraries get mixed.
+If you have `#[should_panic]` tests, try `cargo miri test -- -- -Zunstable-options --exclude-should-panic` because Miri currently aborts execution on a panic.
+
+There's a lot of work left to be done, in particular to enable more programs to execute in Miri.
+Still, slowly but steadily, my [vision]({% post_url 2017-05-23-internship-starting %}) of Miri as a practical tool to test for undefined behavior is actually becoming reality: [the standard library](https://github.com/RalfJung/miri-test-libstd) and [hashbrown](https://github.com/Amanieu/hashbrown/) have their test suites running in Miri under CI.
+I cannot express how glad it makes me to be able to contribute to the Rust ecosystem becoming a bit safer.
+
+Maybe your crate is next?
--- /dev/null
+---
+title: "Stacked Borrows 2"
+categories: rust research
+forum: https://internals.rust-lang.org/t/stacked-borrows-2/9951
+---
+
+Recently, I have [significantly updated](https://github.com/rust-lang/miri/pull/695) Stacked Borrows in order to fix some issues with the handling of shared references that were uncovered in the previous version.
+In this post, I will describe what the new version looks like and how it differs from [Stacked Borrows 1]({% post_url 2018-11-16-stacked-borrows-implementation %}).
+I assume some familiarity with the prior version and will not explain everything from scratch.
+
+<!-- MORE -->
+
+## Stacked Borrows: Table of Contents
+
+Before we start, let me give a brief overview over the posts on Stacked Borrows that I have written so far.
+I didn't plan this out in advance, so things are a bit more messy than I would like.
+
+* [Stacked Borrows: An Aliasing Model For Rust]({% post_url 2018-08-07-stacked-borrows %}) is the first post of the series and describes my initial ideas of what Stacked Borrows would look like before I started implementing them. It might be interesting for some of the context it gives, but is largely superseded by the next post.
+* [Stacked Borrows Implemented]({% post_url 2018-11-16-stacked-borrows-implementation %}) describes Stacked Borrows 1 and my experience implementing it. It is self-contained; I was not happy with some of my explanations in the original post so I decided to give it another shot. **This is the post to read if you are catching up.**
+* [Barriers and Two-phase Borrows in Stacked Borrows]({% post_url 2018-12-26-stacked-borrows-barriers %}) describes how I extended Stacked Borrows 1 with partial support for two-phase borrows and explains the idea of "barriers". You do not have to have read that post to understand Stacked Borrows 2, except for the parts that specifically refer to barriers and two-phase borrows.
+
+## The problem
+
+The problem I wanted to solve with Stacked Borrows 2 was that the first version of Stacked Borrows only performed very little tracking of shared references.
+My thinking was, if the location is read-only anyway, then it is not harmful to grant anyone read access.
+However, [as @arielby noted](https://github.com/rust-lang/unsafe-code-guidelines/issues/87), this leads to loss of optimization potential in cases where a function receives a mutable reference (which is supposed to have no aliases) and then creates a shared reference from it:
+{% highlight rust %}
+fn main() {
+ let x = &mut 0u32;
+ let p = x as *mut u32;
+ foo(x, p);
+}
+
+fn foo(a: &mut u32, y: *mut u32) -> u32 {
+ *a = 1;
+ let _b = &*a; // This freezes `*a`. Frozen locations can be read by any raw pointer.
+ let _val = unsafe { *y; }; // Hence, this legal in Stacked Borrows.
+ *a = 2; // But we might want to drop the earlier `*a = 1` because it gets overwritten!
+ _val
+}
+{% endhighlight %}
+Stacked Borrows 1 allowed any raw pointer to read a frozen location.
+Basically, once a location is frozen we do not keep track of which pointer is allowed to read; we just allow all pointers to read.
+However, that means that `&*a` above (reborrowing a mutable reference as a shared reference) has the unintended side-effect of permitting raw pointers to read `*a`!
+This violates the idea that `a` is a unique pointer: we'd like to remove the first write (`*a = 1`) because it gets overwritten later and there is no read *through `a`* in the mean time.
+The issue is, there *is* an aliasing read through `y`, so removing the seemingly redundant write would change the return value of `foo` from `1` to `0`.
+Hence calling `foo` like we do here with `a` and `y` pointing to the same thing must be undefined behavior---and yet, Stacked Borrows 1 considered the example above to be legal.
+
+## The solution
+
+To fix this, I had to replace the mechanism of "freezing" by something else.
+Remember that in Stacked Borrows 1, when a shared reference was created, we stored the timestamp at which that happened and also recorded in memory that the location this reference points to is frozen since now.
+Whenever a shared reference gets used, we check that the location is frozen *at least* since our reference got created.
+This is in contrast to mutable references, where we require the exact unique ID of that reference to be present in the borrow stack.
+
+To rule out cases like the example above, instead of freezing a location and allowing *all* shared reference created henceforth to access this location, we keep track precisely of *which* shared references are allowed access.
+The per-location borrow stack now consists of items that grant a particular *permission* to a pointer identified by a particular *tag*:
+{% highlight rust %}
+pub struct Item {
+ /// The pointers the permission is granted to.
+ tag: Tag,
+ /// The permission this item grants.
+ perm: Permission,
+}
+
+pub enum Permission {
+ /// Grants unique mutable access.
+ Unique,
+ /// Grants shared mutable access.
+ SharedReadWrite,
+ /// Greants shared read-only access.
+ SharedReadOnly,
+}
+{% endhighlight %}
+There is no longer a separate "frozen" state; keeping shared references read-only is now handled by the borrow stack itself:
+{% highlight rust %}
+pub struct Stack {
+ borrows: Vec<Item>
+}
+{% endhighlight %}
+The *tag* is also simpler than it was before: there are no longer separate tags for mutable and shared references.
+{% highlight rust %}
+pub type PtrId = u64;
+pub enum Tag {
+ Tagged(PtrId),
+ Untagged,
+}
+{% endhighlight %}
+Just like before, a new `Tag` gets picked on every retagging; in particular whenever a reference gets created with `&mut <expr>`/`& <expr>` and when a reference gets cast to a raw pointer.
+Other than that (like on pointer arithmetic), the tag just gets propagated to keep track of where this pointer comes from.
+However, unlike before, the only difference between mutable and shared references is the permissions that are associated with that tag.
+
+### Memory accesses
+
+But before we go into creation of new references, let us look at how permissions in the borrow stack affect what happens at a memory access is legal.
+So assume that some pointer tagged `tag` is used to either read from or write to memory.
+For each affected location, we go through two steps: first we try to find the *granting item*, then we remove *incompatible items*.
+
+**Finding the granting item.**
+To find the granting item, we traverse the borrow stack top-to-bottom and search for an item that has the same tag.
+If this is a write access, we go on looking until we find an item with the right tag whose permission is `SharedReadWrite` or `Unique`---we do not consider items with `SharedReadOnly` permission to grant a write access.
+This is the item that grants pointers tagged `tag` the permission to perform the given access, hence the name (the same concept used to be called "matching item" in Stacked Borrows 1).
+Once we found the granting item, we remember its position in the stack; that will be important for the second step.
+If no granting item can be found, the access causes undefined behavior.
+
+For example, if the stack is
+```
+[ Item { tag: Tagged(0), perm: Unique },
+ Item { tag: Tagged(1), perm: SharedReadOnly} ]
+```
+then a read access with tag `Tagged(1)` is granted by the item at index 1; a read/write access with tag `Tagged(0)` is granted by the item at index 0, but a write access with tag `Tagged(1)` is not granted by any item (and thus not allowed).
+Just to keep things shorter and easier to read, in the following I will use a short-hand syntax for writing down items, so the above stack would look as follows:
+```
+[ (0: Unique), (1: SharedReadOnly) ]
+```
+
+To consider a second example, if the stack is
+```
+[ (0: Unique), (Untagged: SharedReadWrite), (Untagged: SharedReadOnly) ]
+```
+then a read access with an `Untagged` pointer is granted by the item at index 2, but a write access with an `Untagged` pointer is only granted by the item at index 1.
+
+**Removing incompatible items.**
+In the second step, we traverse all the items *above* the granting item in the stack, and see if they are compatible with this access.
+This realizes the idea (already present in the original Stacked Borrows) that using one pointer disallows future uses of another pointer.
+For example, if the current stack is
+```
+[ (0: Unique), (1: Unique) ]
+```
+then doing any kind of access with a `Tagged(0)` pointer should remove the `1: Unique` item from the stack.
+This matches the part of Stacked Borrows 1 where items got popped off the stack until the granting item is at the top.
+We say that the granting `Unique` permission is *incompatible* with the `Unique` permission of the item higher up the stack, and hence the latter must be removed.
+
+However, in the new model, we don't always remove *all* items that are above the granting item:
+```
+[ (0: Unique), (1: SharedReadOnly), (2: SharedReadOnly) ]
+```
+In a situation like this, there are two pointers that may be used for reading, `Tagged(1)` and `Tagged(2)`.
+Using either of them should not have an impact on the other one, and the fact that their items are in a particular order on the stack has no impact.
+In other words, a granting `SharedReadOnly` permission is *compatible* with other `SharedReadOnly` permissions, and hence when using a `SharedReadOnly` permission to grant an access, other `SharedReadOnly` permissions above it are maintained.
+
+Sometimes, being compatible depends on the kind of access that was performed:
+in our last example stack, if we *write* with a `Tagged(0)` pointer, that should remove the `SharedReadOnly` tags because writing to a unique pointer should make sure that that pointer is at the top of the stack.
+In other words, on a write, a `Unique` permission is incompatible with everything.
+However, if we just *read* with a `Tagged(0)` pointer, that's fine!
+This is to allow [safe code like this](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=d9aa504b1be72a0f55fb5a744cba69ff) that interleaves reads from aliasing mutable and shared references.
+To enable this, on a read, a `Unique` permission is only incompatible with other `Unique` permissions.
+
+The following table fully documents which permissions are compatible with which other permissions on a read or write access, respectively:
+
+|↓ **Granting item** \ **Compatible with** → | `Unique` | `SharedReadWrite` | `SharedReadOnly` |
+|---:|---|---|---|
+|`Unique` | no | only reads | only reads |
+|`SharedReadWrite` | no | yes | only reads |
+|`SharedReadOnly` | no | no | only reads |
+
+So, for example, if the granting item has `SharedReadWrite` permissions, then all `Unique` above it in the stack will be removed, and moreover on a write, all `SharedReadOnly` above it will be removed.
+This makes sure that the pointers for which those `SharedReadOnly` permission grants read access cannot be used any more: those pointers assumed the location was read-only, so we must make sure that after a write, they become invalid.
+
+The name "stack" is a bit of a misnomer at this point, because we do *not* follow a stack discipline when removing items: just because some item got removed due to incompatibility, does not mean all items above it will also get removed.
+I will discuss [later][no-stack] why a strict stack discipline does not work.
+But I felt it would be even more confusing to rename the entire thing at this point, and the order of elements in the "stack" still *does* matter, so I kept the name.
+
+In Stacked Borrows 2, there is no more action occurring when a pointer gets dereferenced: the tags only matter for the actual access.
+This simplification is possible because the rules for accesses are now more complicated than before, so there is no need for extra checks on the dereference operation.
+(In particular, previously the dereference checks relied on knowing the Rust type at which the access happens for proper handling of interior mutability; but now all that information is encoded using `SharedReadWrite`/`SharedReadOnly` tags as we will see. Not relying on types any more here also [fixed an annoying issue around `UnsafeCell`](https://github.com/rust-lang/miri/issues/615).)
+
+### Retagging: when permissions get added
+
+The framework of permissions in a stack with a notion of (in)compatibility that we have seen so far allows us to express some ideas like:
+
+* "A `Unique` pointer will get invalid when any of its parent pointers get used."<br> That is realized by having "no" in the entire first column of our table: every access removes all `Unique` permissions above the granting item.
+* "A `SharedReadOnly` pointer will get invalid when the location gets written to."<br> This gets realized by having "only reads" in the entire last column of the table and not granting write accesses with a `SharedReadOnly` permission. In addition to that, however, we *also* need to ensure that in the stack we never have a `Unique` or `SharedReadWrite` *on top* of a `SharedReadOnly`! In a nonsense stack like `[ (0: SharedReadOnly), (1: SharedReadWrite) ]` we could write with a `Tagged(1)` pointer without invalidating the `Tagged(0)` pointers.
+
+Now the question is, how to we *use* this "language"?
+We have to define which items and permissions get added to the borrow stack.
+Just like before, this happens during retagging.
+Before we discuss retagging in general, let us look at our motivating example and see how retagging works there in Stacked Borrows 2, and how this program is defined to cause UB:
+{% highlight rust %}
+fn main() {
+ let x = &mut 0u32;
+ Retag(x); // Tagged(0)
+ // Stack: [ (0: Unique) ]
+
+ let p = x as *mut u32;
+ Retag([raw] p); // Untagged
+ // Stack: [ (0: Unique), (Untagged: SharedReadWrite) ]
+
+ foo(x, p);
+}
+
+fn foo(a: &mut u32, y: *mut u32) -> u32 {
+ Retag(a); // Tagged(1)
+ // Stack: [ (0: Unique), (1: Unique) ]
+
+ *a = 1;
+
+ let _b = &*a;
+ Retag(_b); // Tagged(2)
+ // Stack: [ (0: Unique), (1: Unique), (2: SharedReadOnly) ]
+
+ // UB: y is Untagged, and there is no granting item in the stack!
+ let _val = unsafe { *y };
+
+ *a = 2;
+
+ _val
+}
+{% endhighlight %}
+Initially, `x` with tag `Tagged(0)` is the only reference, and the stack says that this is the only pointer with any kind of permission.
+Next, we cast `x` to a raw pointer.
+The raw retagging of `p` turns `p` into an `Untagged` pointer, and adds a new item granting `Untagged` pointers `SharedReadWrite` permission.
+(Really, in the MIR it will say `&mut *x as *mut u32`, so there will be an additional `Unique` permission for the temporary mutable reference, but that makes no difference and I hope [we will change that eventually](https://github.com/rust-lang/rfcs/pull/2582).)
+
+Then `foo` gets called, which starts with the usual retagging of all reference arguments.
+`a` is originally `Tagged(0)`, and retagging a mutable reference acts like an access (just like in Stacked Borrows 1), so the first thing that happens is that all items above the granting item that are incompatible with a write access for a `Unique` permission get removed from the stack.
+In our case, this means that the `Untagged: SharedReadWrite` gets removed.
+Then, `a` gets the new tag `Tagged(1)` and `1: Unique` gets pushed on top of the stack.
+Nothing interesting happens when writing to `a`.
+When `_b` gets created, it gets assigned the new tag `Tagged(2)`, and a new item `2: SharedReadOnly` is pushed to the stack.
+As we can see, shared and mutable references no longer differ in the tag they carry; the only difference is what kind of permission they get granted.
+(I think this is a nice improvement from Stacked Borrows 1, where shared references had a different kind of tag. In particular, transmutes between mutable references and shared pointers no longer need any kind of special treatment.)
+Finally, we come to the interesting point: the program reads from `y`.
+That pointer is `Untagged`, and there is no item granting any access to such pointers, and thus the access is UB!
+This is in contrast to Stacked Borrows 1, where instead of `2: SharedReadOnly` we set a special "frozen" marker on this location that would, as a side-effect, also grant untagged pointers like `y` read-only access.
+
+In general, during retagging, we start with some original tag used by our "parent" pointer, and we have a fresh tag `N` that will be used for the new pointer.
+We have to decide which permission to grant so this tag, and where in the "stack" to insert it.
+(We will not always insert new items at the top---again, as we will see following a strict stack discipline does not work.)
+All of this happens for every location that the reference covers (as determined by its type).
+
+**Retagging a mutable reference.**
+The simplest case of retagging is handling mutable references:
+just like in Stacked Borrows 1, this starts by performing the actions of a write access with the parent tag.
+`Unique` permissions are incompatible with anything on a write, so all items above the granting item get removed.
+Then we add `N: Unique` on top of the stack to grant the new tag unique access to this location.
+
+This encodes the idea that mutable references must be used in a well-nested way---if you just consider mutable references, Stacked Borrows 2 still follows a stack discipline.
+
+**Retagging a shared reference.**
+When retagging a shared reference, we have to be mindful of `UnsafeCell`.
+
+Outside of `UnsafeCell`, we start by performing the actions of a read access with the parent tag.
+Then we push `N: SharedReadOnly` on top of the borrow stack.
+This way, we grant the new pointer read-only access but make sure that it gets invalidated on the next write through any aliasing pointer (because all write-granting items are below us, and thus we test for compatibility when they get used).
+There might be items left between the granting item and the one we just added, but that's okay: if any of them gets used for reading, that will be compatible with the `SharedReadOnly` according to our table above; and if any of them gets used for writing then it is important that our `SharedReadOnly` gets removed.
+
+When we are inside an `UnsafeCell`, we will *not* perform the actions of a memory access!
+Interior mutability allows all sorts of crazy aliasing, and in particular, one can call a function with signature
+{% highlight rust %}
+fn aliasing(refcell: &RefCell<i32>, inner: &mut i32)
+{% endhighlight %}
+such that `inner` points *into* `refcell`, i.e., the two pointers overlap!
+Retagging `refcell` must not remove the `Unique` item associated with `inner`.
+
+So, when retagging inside an `UnsafeCell`, we *find* the write-granting item for the parent's tag, and then we add `N: SharedReadWrite` just above it.
+This grants write access, as is clearly needed for interior mutability.
+We cannot add the new item at the top of the stack because that would add it *on top of* the item granting `inner`.
+Any write to `inner` would remove our item from the stack!
+Instead, we make our item sit just on top of its parent, reflecting the way one pointer got derived from the other.
+
+**Retagging a raw pointer.**
+Retagging for raw pointers happens only immediately after a reference-to-raw-pointer cast.
+Unlike in Stacked Borrows 1, retagging depends on whether this is a `*const T` or `*mut T` pointer---this is a departure from the principle that these two types are basically the same (except for variance).
+I have recently learned that the borrow checker actually handles `*mut` and `*const*` casts differently; also see [this long comment](https://github.com/rust-lang/rust/issues/56604#issuecomment-477954315).
+Given that the idea of Stacked Borrows is to start with what the borrow checker does and extrapolate to a dynamic model that also encompasses raw pointers, I felt that it makes sense for now to mirror this behavior in Stacked Borrows.
+This is certainly not a final decision though, and I feel we should eventually have a discussion whether we should make the borrow checker and Stacked Borrows both treat `*const T` and `*mut T` the same.
+
+When casting to a `*mut T`, we basically behave like in the above case for inside an `UnsafeCell` behind a shared reference: we find the write-granting item for our parent's tag, and we add `Untagged: SharedReadWrite` just on top of it.
+The way compatibility is defined for `SharedReadWrite`, there can be many such items next to each other on the stack, and using any one of them will not affect the others.
+However, writing with a `Unique` permission further up the stack *will* invalidate all of them, reflecting the idea that when writing to a mutable reference, all raw pointers previously created from this reference become invalid.
+
+When casting to a `*const T`, we behave just like retagging for a shared reference `&T` (including being mindful of `UnsafeCell`).
+There isn't really anything that a `*const T` can do that a shared reference cannot (both in terms of aliasing and mutation), so modeling them the same way makes sense.
+
+### Two-phase borrows
+
+Stacked Borrows 1 [had some support for two-phase borrows]({% post_url 2018-12-26-stacked-borrows-barriers %}), but some advanced forms of two-phase borrows that used to be allowed by the borrow checker [could not be handled](https://github.com/rust-lang/rust/issues/56254).
+With the additional flexibility of Stacked Borrows 2 and its departure from a strict stack discipline, it is possible to accept at least the known examples of this pattern that were previously rejected:
+{% highlight rust %}
+fn two_phase_overlapping1() {
+ let mut x = vec![];
+ let p = &x;
+ x.push(p.len());
+}
+{% endhighlight %}
+Previously, when the implicit reborrow of `x` in `x.push` got executed, that would remove the item for `p` from the stack.
+This happens as part of the implicit write access that occurs when a mutable reference gets retagged.
+With Stacked Borrows 2, we can do something else:
+when retagging a two-phase mutable borrow, we do *not* perform the actions of a write access.
+Instead, we just find the write-granting item for our parent's tag, and then add `N: Unique` just above it.
+This has the consequence that on the first write access to this new pointer, everything on top of it will be removed, but until then the existing item for `p` remains on the stack and can be used just as before.
+
+Just like with Stacked Borrows 1, we then proceed by doing a *shared* reborrow of the parent's tag from `N`.
+This way, the parent pointer can be used in the ways a shared reference can (including writes, if there is interior mutability) without invalidating `N`.
+This is somewhat strange because we then have "parent tag - new tag - parent tag" on the stack in that order, so we no longer properly reflect the way pointers got derived from each other.
+More analysis will be needed to figure out the consequences of this.
+
+We should also try to fully understand the effect this "weak" form of a mutable reborrow (without a virtual write access) has on the optimizations that can be performed.
+Most of the cases of Stacked Borrows violations I found in the standard library are caused by the fact that even just *creating* a mutable reference asserts that it is unique and invalidates aliasing pointers, so if we could weaken that we would remove one of the most common causes of UB caused by Stacked Borrows.
+On the other hand, this means that the compiler will have a harder time reordering uses of mutable references with function calls, because there are fewer cases where a mutable reference is actually assumed to be unique.
+
+### Barriers are dead, long live protectors
+
+With the departure from a strict stack discipline, I also had to re-think the concept of [barriers]({% post_url 2018-12-26-stacked-borrows-barriers %}).
+The name was anyway terrible, so I replaced barriers by *protectors*: an `Item` actually consists not only of a `Tag` and a `Permission`, but also optionally of a "protector" represented as a `CallId` referencing some function call (i.e., some stack frame).
+As long as that function call is running, the item with the protector may not be removed from the stack (or else we have UB).
+This has pretty much the same effects as barriers did in Stacked Borrows 1.
+
+## Why not a strict stack discipline?
+[no-stack]: #why-not-a-strict-stack-discipline
+
+The biggest surprise for me in designing Stacked Borrows 2 was that I was not able to enforce a strict stack discipline any more.
+For retagging, that is only partially surprising; really in Stacked Borrows 1 we already added barriers below the "frozen" marker sitting next to a stack, and the `aliasing` example with the `RefCell` mentioned above only worked due to a hack that relied on reusing existing items in the middle of the stack instead of pushing a new one.
+However, I had initially hoped that the rules for memory accesses would be slightly different:
+my plan was that after finding the granting item, we would seek upwards through the stack and find the first incompatible item, and then remove everything starting there.
+That would be a stack discipline; we would basically pop items until all items above the granting item are compatible.
+
+Unfortunately, that would reject many reasonable programs, such as:
+{% highlight rust %}
+fn test(x: &mut [i32]) -> i32 { unsafe {
+ let raw = x.as_mut_ptr(); // implicitly: as_mut_ptr(&mut *x)
+ let _val = x[1];
+ return *raw;
+} }
+{% endhighlight %}
+The issue with this example is that when calling `as_mut_ptr`, we reborrow `x`.
+So after the first line the stack would look something like (using `x` as notation for `x`'s tag)
+```
+[ ..., (x: Unique), (_: Unique), (Untagged: SharedReadWrite) ]
+```
+In other words, there would be some `Unique` item *between* the items for `x` and `raw`.
+When reading from `x` in the second line, we determine that this `Unique` tag is not compatible.
+This is important; we have to ensure that any access from a parent pointer invalidates `Unique` pointers---that's what makes them unique!
+However, if we follow a stack discipline, that means we have to pop the `Untagged: SharedReadWrite` *and* the `_: Unique` off the stack, making the third line UB because the raw pointer got invalidated.
+
+`test` seems like a perfectly reasonable function, and in fact this pattern is used in the standard library.
+So in order to allow such code, accesses in Stacked Borrows 2 do *not* follow a stack discipline:
+we remove all incompatible items above the granting item and ignore any interleaved compatible items.
+As a consequence, line 2 in the above program removes `_: Unique` but keeps `Untagged: SharedReadWrite`, so line 3 is okay.
+
+However, this means we also accept the following, even if we eventually do fully precise tracking of raw pointers:
+{% highlight rust %}
+fn main() { unsafe {
+ let x = &mut 0;
+ let raw1 = x as *mut _;
+ // Stack: [ (x: Unique), (raw1: SharedReadWrite) ]
+
+ let tmp = &mut *raw1;
+ let raw2 = tmp as *mut _;
+ // Stack: [ (x: Unique), (raw1: SharedReadWrite), (tmp: Unique), (raw2: SharedReadWrite) ]
+
+ *raw1 = 1; // This will invalidate tmp, but not raw2.
+ // Stack: [ (x: Unique), (raw1: SharedReadWrite), (raw2: SharedReadWrite) ]
+
+ let _val = *raw2;
+} }
+{% endhighlight %}
+Naively I would assume that if we tell LLVM that `tmp` is a unique pointer, it will conclude that `raw2` cannot alias with `raw1` as that was not derived from `tmp`, and hence LLVM might conclude that `_val` must be 0.
+This means the program above would have to have UB.
+However, in the current framework I do not see a way to make it UB without also making the reasonable example from the beginning of this section UB.
+So maybe we can find a way to express to LLVM precisely what we mean, or maybe we can only exploit some of these properties in home-grown optimizations, or maybe we find a way to have UB in the second example but not in the first.
+(Like, maybe we do the stack-like behavior on writes but keep the more lenient current behavior on reads? That seems annoying to implement though. Maybe a stack is just the wrong data structure, and we should use something more tree-like.)
+
+## Conclusion
+
+Stacked Borrows 2 shares with Stacked Borrows 1 just the general structure: pointers are tagged, and there is a per-location stack indicating which tags are allowed to perform which kinds of operations on this location.
+In the new version, shared references are tagged the same way as mutable references (just with an ID to distinguish multiple references pointing to the same location); the stack keeps track of which IDs are read-only (shared) pointers and which are unique (mutable) pointers.
+The only operations affected by Stacked Borrows 2 are memory accesses and the `Retag` instructions explicitly represented in the MIR; there is no longer any action on a pointer dereference.
+This balances the extra complexity of the new access rules (the new implementation is actually a dozen lines shorter than the old, despite long comments).
+The "stack" is unfortunately not used in a completely stack-like fashion though.
+
+I leave it as an exercise to the reader to convince yourself that the [key properties of Stacked Borrows 1]({% post_url 2018-11-16-stacked-borrows-implementation %}#5-key-properties) still hold, and that uniqueness of mutable references is maintained even if we do a shared reborrow, as long as we keep that shared reference to ourselves.
+
+The new model gives some wiggle-room in both the notion of which permissions are compatible with others and where exactly which permissions are added in the "stack" in a reborrow.
+We could use that wiggle-room to experiment with a bunch of closely related models to see how they affect which code gets accepted and analyze their impact on optimizations.
+
+I am quite excited by this new model!
+It puts us into a good position to do more precise tracking for raw pointers, similar to what already happens for shared references (and something I was worried about in the original model).
+That will be needed for compatibility with LLVM.
+However, [there are still some known issues](https://github.com/rust-lang/unsafe-code-guidelines/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-stacked-borrows), and also the fact that we do not actually use the "stack" as a stack indicates that maybe we should use a different structure there (potentially something more like a tree).
+This is definitely not the final word, but I think it is a step in the right direction, and I am curious to see how it works out.
+As usual, if you have any questions or comments, please join the [discussion in the forums](https://internals.rust-lang.org/t/stacked-borrows-2/9951)!
--- /dev/null
+---
+title: "Exponential blowup when using unbundled typeclasses to model algebraic hierarchies"
+categories: research coq
+forum: https://coq.discourse.group/t/blog-post-exponential-blowup-when-using-unbundled-typeclasses-to-model-algebraic-hierarchies/289
+---
+
+When formalizing a proof in an interactive theorem prover like [Coq](https://coq.inria.fr/), one reoccurring issue is the handling of algebraic hierarchies.
+Such hierarchies are everywhere: some operations are associative, while others commute; some types have an equivalence relation, while others also have a (pre-)order or maybe even a well-ordering; and so on.
+So the question arises: What is the best way to actually encode these hierarchies in Coq?
+Coq offers two mechanisms that are suited to solve this task: typeclasses and canonical structures.
+Both can be instrumented in different ways to obtain a (more or less) convenient-to-use algebraic hierarchy.
+A common approach using typeclasses is the ["unbundled" approach by Bas Spitters and Eelis van der Weegen](http://www.eelis.net/research/math-classes/mscs.pdf).
+However as we learned the hard way in the Coq formalization of the [original Iris paper](https://iris-project.org/pdfs/2015-popl-iris1-final.pdf), this approach quickly leads to terms that seem to be exponential in size.
+In this post, I will explain the cause of this exponential blowup.
+
+<!-- MORE -->
+
+I should note that this observation is not new, it already occurs in [François Garillot's PhD thesis](https://pastel.archives-ouvertes.fr/pastel-00649586), a [paper by the same author](https://hal.inria.fr/inria-00368403v2/document) and even the authors of the "unbundled" paper already note having performance problems when scaling up.
+The goal of this post is to provide a more self-contained presentation, not requiring all the context set up in that thesis and going into more details than the brief remarks in the two papers.
+
+## Unbundled groups
+
+A formalization of the algebraic hierarchy for groups using the unbundled approach could look as follows:
+{% highlight coq %}
+Require Import Lia ZArith.
+
+(** The operations as typeclasses for overloading, without any axioms. *)
+Class Op (A: Type) := op : A -> A -> A.
+Infix "+" := op.
+
+Class Unit (A: Type) := unit : A.
+
+Class Inverse (A: Type) := inv : A -> A.
+
+(** The algebraic classes adding the axioms, taking the operations as indices. *)
+Class Semigroup (A: Type) `{Op A} := {
+ assoc a b c : (a + b) + c = a + (b + c);
+}.
+
+Class Monoid (A: Type) `{Semigroup A, Unit A} := {
+ id_r a : a + unit = a;
+ id_l a : unit + a = a;
+}.
+
+Class Group (A: Type) `{Monoid A, Inverse A} := {
+ inv_r a : a + inv a = unit;
+ inv_l a : inv a + a = unit;
+}.
+{% endhighlight %}
+
+As usual for the unbundled approach, we end up with lots of tiny typeclasses.
+`Op`, `Unit` and `Inverse` serve as operators that can be overloaded.
+They each just take a type `A` as argument (we say they are *indexed* by the type: we can do typeclass search that, given the type, finds the instance).
+`Semigroup`, `Monoid` and `Group` supply the laws for these operators; they are indexed by the operators they involve and the typeclasses from which they "inherit".
+Taking the superclasses as indices instead of embedding them as fields simplifies typeclass search and is necessary to support diamonds in the hierarchy; see [the original paper](http://www.eelis.net/research/math-classes/mscs.pdf) for more details.
+
+We use Coq's feature for anonymous implicit generalization: `` `{Semigroup A} `` adds not only a parameter of type `Semigroup A` that we don't have to name; furthermore all typeclass arguments of `Semigroup` (in this case, `Op A`) are also implicitly added as parameter.
+This avoids lots of repetition that the unbundled style would usually incur, [but the semantics of implicit generalization are quirky and poorly documented](https://coq.discourse.group/t/documentation-of-foo/129).
+
+Next, we can show that the integers `Z` form a group:
+{% highlight coq %}
+Instance Z_op: Op Z := Z.add.
+Instance Z_unit: Unit Z := 0%Z.
+Instance Z_inv: Inverse Z := Z.opp.
+
+Instance Z_semigroup: Semigroup Z.
+Proof. split. intros. unfold op, Z_op. lia. Qed.
+Instance Z_monoid: Monoid Z.
+Proof. split; intros; unfold op, Z_op, unit, Z_unit; lia. Qed.
+Instance Z_group: Group Z.
+Proof. split; intros; unfold op, Z_op, unit, Z_unit, inv, Z_inv; lia. Qed.
+{% endhighlight %}
+We let `lia` do most of the proving, but we have to unfold some definitions first.
+
+We can also show that a product of two {semigroups, monoids, groups} is itself a {semigroup, monoid, group}:
+{% highlight coq %}
+Section prod.
+ Context {A B: Type}.
+
+ Global Instance prod_op `{Op A, Op B}: Op (A * B) :=
+ fun '(a1, b1) '(a2, b2) => (a1 + a2, b1 + b2).
+ Global Instance prod_unit `{Unit A, Unit B}: Unit (A * B) :=
+ (unit, unit).
+ Global Instance prod_inv `{Inverse A, Inverse B}: Inverse (A * B) :=
+ fun '(a, b) => (inv a, inv b).
+
+ Global Instance prod_semigroup `{Semigroup A, Semigroup B}: Semigroup (A * B).
+ Proof.
+ split. intros [a1 b1] [a2 b2] [a3 b3]. unfold op, prod_op.
+ rewrite !assoc. reflexivity.
+ Qed.
+ Global Instance prod_monoid `{Monoid A, Monoid B}: Monoid (A * B).
+ Proof.
+ split; intros [a b]; unfold op, prod_op, unit, prod_unit;
+ rewrite ?id_l, ?id_r; reflexivity.
+ Qed.
+ Global Instance prod_group `{Group A, Group B}: Group (A * B).
+ Proof.
+ split; intros [a b]; unfold op, prod_op, unit, prod_unit, inv, prod_inv;
+ rewrite ?inv_l, ?inv_r; reflexivity.
+ Qed.
+End prod.
+{% endhighlight %}
+
+## Unexpected complexities
+
+Seems reasonable?
+Unfortunately, we have set ourselves up for some unexpectedly large terms:
+the size of an instance of something like `Group (Z*Z*Z*Z*Z*Z)` grows with the fifth power as the number of `Z` increase!
+
+{% highlight coq %}
+Definition test: Group (Z*Z*Z*Z*Z*Z) := _.
+Set Printing All.
+Print test.
+(* Lots of output. *)
+{% endhighlight %}
+
+What is happening here?
+We can see that when we try to construct the proof terms more explicitly.
+We start by defining `Zn`, which is basically `Z^(n+1)`: a product of `n+1` times `Z`.
+{% highlight coq %}
+Fixpoint Zn (n: nat): Type :=
+ match n with
+ | O => Z
+ | S n => @prod (Zn n) Z
+ end.
+{% endhighlight %}
+Unsurprisingly, this term has size linear in `n`: the recursive case adds `@prod • Z` around the term produced for `n-1` (represented by the `•`).
+This means the term grows by a constant when `n` increases by 1, leading to linear complexity overall.
+
+Next, we define `Op (Zn n)`:
+{% highlight coq %}
+Fixpoint Zn_op (n: nat): Op (Zn n) :=
+ match n with
+ | O => Z_op
+ | S n => @prod_op (Zn n) Z (Zn_op n) Z_op
+ end.
+{% endhighlight %}
+In the recursive case, we use `prod_op` as one might expect.
+Coq could infer all the arguments here, but let us take a closer look at what it would infer: what is the size of `Zn_op` as `n` grows?
+In the recursive case, we add `@prod_op (Zn n) Z • Z_op` around the term produced for `n-1`.
+This term includes `Zn n`, which we established is linear in size.
+When `n` increases by 1, the term grows by a linear amount.
+Thus, the overall term size is quadratic in `n`!
+Ouch.
+I don't know about you, but I found this surprising when I realized it for the first time.
+
+The index basically repeats all the information that is already present in the recursive call:
+to extend a `Zn_op` by one more element, we have to repeat the full type for which we already constructed an `Op`.
+
+But, you might say, quadratic complexity is not so bad.
+Computers are fast, we can handle that.
+Unfortunately, I got bad news: things get worse as our indices start nesting.
+Let us look at the instance for `Semigroup (Zn n)`:
+{% highlight coq %}
+Fixpoint Zn_semigroup (n: nat): Semigroup (Zn n) :=
+ match n return @Semigroup (Zn n) (Zn_op n) with
+ | O => Z_semigroup
+ | S n => @prod_semigroup (Zn n) Z (Zn_op n) (Zn_semigroup n) Z_op Z_semigroup
+ end.
+{% endhighlight %}
+How big is this term?
+Well, the term added as `n` increases by 1 is `@prod_semigroup (Zn n) Z (Zn_op n) • Z_op Z_semigroup`.
+This includes `Zn_op n`, which is quadratic -- so `Zn_semigroup` is cubic!
+
+`Zn_semigroup` has an `Op (Zn n)` as index, and again the full nested term describing the operator of both semigroups that we take the product of is getting repeated each time we add just one more `Z`.
+So, while `Op (Zn n)` has an index that makes it grow quadratically, `Semigroup (Zn n)` *has an index that has an index*, and grows cubically.
+
+In general, to compute the complexity of our terms, we have to figure out *how nested* our indices are: with no indices (such as `Zn n` itself), the term is linear; for each extra nested index the power increases by 1.[^1]
+If we look at `Group`, we see that it has a `Monoid` index which has a `Semigroup` index which has an `Op` index which has a `Type` index.
+That's 4 nested indices, and hence the terms have size `O(n^5)`.
+More generally speaking, the term size is *exponential* in the height of our hierarchy.
+That hurts.
+
+[^1]: Note that the *number* of indices does not matter, like the fact that `Monoid` has both an `Op` and a `Unit` index. That just affects the constant factor. It's the nesting of indices that gets us into trouble here.
+
+## Conclusion
+
+We have seen that the number of nested indices of a typeclass has exponential impact on the size of proof terms that arise when deriving typeclass instances for compound types.
+This effect has grinded the original Iris coq development back in 2015 to a halt: compiling the correctness proof of the logic itself took around 3.5 hours and needed more than the 4GB of RAM that my laptop had back then.
+
+In my view, this basically rules out the unbundled approach for any serious formalization effort.
+But what are the alternatives?
+
+Instead of making superclasses indices, we could make them fields, as in:
+{% highlight coq %}
+Class Monoid (A: Type) `{Op A, Unit A} := {
+ monoid_semigroup :> Semigroup A;
+ id_r a : a + unit = a;
+ id_l a : unit + a = a;
+}.
+{% endhighlight %}
+
+This limits the "index depth" to 2, and thus limits term complexity to `O(n^3)`.
+Still not great, but at least it doesn't grow further as we add more subclasses.
+However, this scheme does not support diamonds in the hierarchy as instances will be duplicated.
+Also, the instance we implicitly added by writing `:>`, namely that `Monoid A -> Semigroup A`, means we'll check all `Monoid` instances any time we just want a `Semigroup`.
+This leads to exponential complexity when performing backtracking typeclass search: a `Semigroup (A*A)` can be obtained either by searching for a `Monoid (A*A)` or via `prod_semigroup`, so in case of backtracking Coq will go through a combinatorial explosion when trying all possible ways to derive `Semigroup (A*A*A*...)`.[^2]
+
+[^2]: In genaral, such combinatorial blowups happen very easily with backtracking, and they are hard to detect in advance. This is one reason why I believe that backtracking-per-default is a bad choice for proof search.
+
+Maybe there is a way to make the unbundled approach viable with some help from the proof assistant.
+Some way to eliminate indices would help (similar to primitive projections, but also covering operations such as `prod_op`).
+Or maybe the proof assistant could exploit the huge amounts of sharing that are present in these large terms.
+If all subterms are properly shared, I think the overall term size remains linear.
+Coq deduplicates term representation in memory, but (to my knowledge) does not exploit sharing when traversing the term (for unification or type-checking or any other purpose).
+Defeating the exponential complexity here would require exploiting the sharing in *every* pass over the term.
+An alternative to opportunistically discovering and exploiting sharing might be to [bake it more deeply into metavariable inference](https://eutypes.cs.ru.nl/eutypes_pmwiki/uploads/Meetings/Kovacs_slides.pdf).
+
+In Iris, we instead went with an adaptation of [this approach based on canonical structures](https://hal.inria.fr/hal-00816703v1/document).
+This fully bundled approach avoids indices entirely and thus maintains a linear term size.
+On the other hand, it comes with a lot of boilerplate code, and mixing canonical structures with typeclasses can be tricky to debug. (Unfortunately we cannot avoid typeclasses entirely as we need to have `Proper` instances in our algebraic classes.)
+When composing types, Iris users have to write things like `prodC (listC natC) (optionC fracC)` instead of `(list nat) * (option frac)`, which is an additional barrier of entry to an already complicated development.
+Also, I have to say I don't really have an intuitive grasp of canonical structure resolution, so when something does not work I don't even know how to start debugging.
+On the other hand, the correctness proof of the logic *and* the general proof mode infrastructure *and* a few examples verified in the logic take less than five minutes to compile in our latest Coq formalization.
+That's quite some speedup!
+
+All in all, I think the question of how to best represent algebraic hierarchies in a proof assistant is still wide open.
+I am curious which ideas people will come up with!
+
+#### Footnotes
--- /dev/null
+---
+title: "Putting the stack back into Stacked Borrows"
+categories: rust research
+forum: https://internals.rust-lang.org/t/stacked-borrows-2/9951/8
+---
+
+Less than a month ago, [I announced Stacked Borrows 2]({% post_url 2019-04-30-stacked-borrows-2 %}).
+In particular, I hoped that that version would bring us closer to proper support for two-phase borrows.
+Turns out I was a bit too optimistic!
+Last week, @Manishearth [asked on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/safety.20of.20stable.20containers) why Miri rejected a certain program, and it turned out that the issue was related to two-phase borrows: in combination with interior mutability, behavior wasn't always what we wanted it to be.
+So, I went back to the drawing board and tried to adjust Stacked Borrows.
+
+In the end, I decided to give up on "proper" support for two-phase borrows for now, which I [explained here](https://github.com/rust-lang/rust/issues/56254#issuecomment-493756649).
+But I also made some tweaks to Stacked Borrows that affect all accesses (not just two-phase borrows), and that's what this post is about.
+I am referring to this as "Stacked Borrows 2.1".
+
+<!-- MORE -->
+
+## Stacked Borrows: Table of Contents
+
+Before we start, here's a list of all posts about Stacked Borrows so far.
+I am assigning them each a version number to hopefully make it a bit less confusing (I did not exactly foresee an entire series of posts on this subject).
+
+* [Stacked Borrows 0.1]({% post_url 2018-08-07-stacked-borrows %}) is my initial idea of what Stacked Borrows might look like before I started implementing it. This post is interesting for some of the historical context it gives, but is largely superseded by the next post.
+* [Stacked Borrows 1.0]({% post_url 2018-11-16-stacked-borrows-implementation %}) is the first version that got implemented. This post is self-contained; I was not happy with some of my explanations in the original post so I decided to give it another shot.
+* [Stacked Borrows 1.1]({% post_url 2018-12-26-stacked-borrows-barriers %}) extends Stacked Borrows 1 with partial support for two-phase borrows and explains the idea of "barriers".
+* [Stacked Borrows 2.0]({% post_url 2019-04-30-stacked-borrows-2 %}) is a re-design of Stacked Borrows 1 that maintains the original core ideas, but changes the mechanism to support more precise tracking of shared references. In the following, I will assume you have read at least this post.
+
+## Is there a stack?
+
+As I [explained the last time]({% post_url 2019-04-30-stacked-borrows-2 %}#why-not-a-strict-stack-discipline), there's not really a "stack" any more in Stacked Borrows.
+I showed a widely-used pattern that would be ruled out by a strict stack discipline, and at least the basic optimizations that motivated Stacked Borrows are fine with this weaker model.
+But one thing that I realized is that that pattern just shows that *read accesses* cannot strictly maintain a stack.
+So maybe write accesses could?
+
+And indeed, that's one of the things that changed with Stacked Borrows 2.1: on a write access, instead of removing just the incompatible items above the granting item, we remove *everything* above (and including) the first incompatible item above the granting item.
+Or, if we inline the definition of "compatible":
+* When writing to a `Unique` item, we remove everything above (this is unchanged to Stacked Borrows 2.0).
+* When writing to a `SharedReadWrite`, we keep the *adjacent* `SharedReadWrite` above it, but then remove *everything* above the first "other" item.
+
+The last point is a change in behavior: the following code used to be legal, and is now UB.
+{% highlight rust %}
+fn main() { unsafe {
+ let c = &UnsafeCell::new(UnsafeCell::new(0));
+ let inner_uniq = &mut *c.get();
+ // stack: [c: SharedReadWrite, inner_uniq: Unique]
+
+ let inner_shr = &*inner_uniq; // adds a SharedReadWrite
+ // stack: [c: SharedReadWrite, inner_uniq: Unique, inner_shr: SharedReadWrite]
+
+ *c.get() = UnsafeCell::new(1); // invalidates inner_shr
+ // stack: [c: SharedReadWrite]
+
+ let _val = *inner_shr.get(); // error because the tag is not on the stack any more
+} }
+{% endhighlight %}
+(I am writing variable names in the stack to refer to those variable's tag.)
+In stacked Borrows 2.0, the stack at the end would instead have been `[c: SharedReadWrite, inner_shr: SharedReadWrite]`, thus permitting the final access.
+
+## Let there be structure!
+
+I am not deeply invested in allowing or forbidding that last example.
+However, this change has the nice side-effect that it gives rise to some structure in how the stack looks like and how it can be interpreted, and I think that is helpful when understanding Stacked Borrows.
+
+Remember that the stack consists of items that grant `Unique`, `SharedReadWrite` or `SharedReadOnly` permission.
+
+However, `SharedReadOnly` only occurs at the top of the stack (there can be multiple of these at the top, but there will never be another permission above a `SharedReadOnly`).
+For example, the stack might end in `[..., a: Unique, x: SharedReadOnly, y: SharedReadOnly]`.
+Here, `x` and `y` are read-only shared references that are currently allowed to access this memory.
+They are either derived from `a`, or derived from another one of the read-only shared references.
+(I am using the non-transitive version of "derived from" here, i.e., I am talking about the *direct* predecessor.)
+
+Further down in the stack, if we have two neighboring `Unique`, that says that the one further up the stack was "derived from" the one further down.
+
+But what about `SharedReadWrite`?
+If we think of an `&mut UnsafeCell<T>` (which will have a `Unique` permission), then each time we create a shared reference from it, that will add a `SharedReadWrite` above the `Unique`.
+We might end up with `[..., a: Unique, x: SharedReadWrite, y: SharedReadWrite]`.
+Here, `x` and `y` can all be used interchangeably: reading from or writing to any of these `SharedReadWrite` references will not invalidate any of the others!
+The way I think about it is that a bunch of adjacent `SharedReadWrite` permissions together form one "block" on the stack.
+Writing to the `Unique` from which they all come (`a` in our example) invalidates the entire block.
+
+This is a lot like the "block" of `SharedReadOnly` that may exist at the top of the stack.
+Except that, from an `&UnsafeCell<T>` with `SharedReadWrite` permission, we can derive a `&mut T` with `Unique` permission!
+So we might have a stack like `[..., a: Unique, x: SharedReadWrite, y: SharedReadWrite, b: Unique]` where `b` is derived from `x` or `y` (we don't know which, but that has no effect on what programs are allowed).
+In case of nested `UnsafeCell`, we might even have another (block of) `SharedReadWrite` further up the stack!
+
+So, in general, the stack looks as follows (starting at the bottom):
+* First a "stem" consisting of a sequence of "blocks". A "block" in the stem is either a single `Unique` permissions and or a bunch of consecutive `SharedReadWrite` permissions.
+* Finally, optionally, a "cap" consisting of a "block" of consecutive `SharedReadOnly` permissions.
+
+For all of these blocks, we have that each pointer in a block is derived either from a pointer in the next block down the stack, or another pointer in the same block.
+
+In this framework, we can re-cast the new rule for write access as: when writing to a pointer in some block, remove all blocks further up the stack.
+
+## But what about reads?
+
+However, it turns out that the old rule for read accesses wrecks havoc with this block-based intuition that I just described.
+Remember that on a read, we removed all read-incompatible items above the granting item---which, when inlining the definition of "compatibility", means removing all `Unique` above the granting item.
+If this `Unique` item sits between two (blocks of) `SharedReadWrite` items, that has the unfortunate side-effect of *merging* these two blocks!
+
+For example, prior to the read access, we might have had a stack like `[x: SharedReadWrite, a: Unique, y: SharedReadWrite]`. In this situation, writing to `x` would (under the new rules) invalidate `y` because it is in a block further up the stack.
+However, after reading from `x`, the stack changes to `[x: SharedReadWrite, y: SharedReadWrite]`.
+The two blocks got merged into one, and now `x` and `y` can be used interchangeably!
+That seems like a strange effect.
+Furthermore, this breaks the invariant that in a block of `SharedReadWrite`, each pointer is derived from another pointer in the same block or in the next block down the stack: `y` is derived from `a`, not from `x`!
+
+To fix this and make sure that the structure I described in the previous section actually holds, I changed read accesses to avoid merging two blocks of `SharedReadWrite`.
+Instead of just removing `Unique`, we replace them by a `Disabled` "permission" that grants no access, but just serves to separate two blocks.
+With that, the stack in our example becomes `[x: SharedReadWrite, a: Disabled, y: SharedReadWrite]`.
+This keeps the two (singleton) blocks of `SharedReadWrite` apart, so even after reading from `x`, a subsequent write to `x` will still invalidate `y`.
+
+## Conclusion
+
+That's it already.
+You can find a complete description (but without explanation) of Stacked Borrows 2.1 [in the UCG repository](https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md).
+
+I mostly wanted to talk about this structure of "blocks" that still looks very much like a stack, just a more refined version of what we had in Stacked Borrows 1.
+Back then, such a block of `SharedReadWrite` was basically represented by a single `Raw` item, not differentiating *which* pointers are allowed to access the memory in this way.
+Similarly, the block of `SharedReadOnly` was represented by the "freeze" flag, which excluded some shared references but was still not very precise in encoding which pointers are allowed to read.
+The key contribution of Stacked Borrows 2, then, was to refine these rather coarse notions and keep track exactly of which shared references are part of which block.
+
+For the future, it will be interesting to figure out if we can track raw pointers the same way we are now tracking shared references with interior mutability, or if that will rule out too much existing code.
+I don't really have a timeline for this problem though, the next immediate steps involve writing a paper about Stacked Borrows and then writing a PhD thesis about... lots of stuff. ;)
--- /dev/null
+---
+title: '"What The Hardware Does" is not What Your Program Does: Uninitialized Memory'
+categories: rust research programming
+forum: https://internals.rust-lang.org/t/what-the-hardware-does-is-not-what-your-program-does-uninitialized-memory/10561
+---
+
+This post is about uninitialized memory, but also about the semantics of highly optimized "low-level" languages in general.
+I will try to convince you that reasoning by "what the hardware does" is inherently flawed when talking about languages such as Rust, C or C++.
+These [are not low-level languages](https://queue.acm.org/detail.cfm?id=3212479).
+I have made this point before [in the context of pointers]({% post_url 2018-07-24-pointers-and-bytes %}); this time it is going to be about uninitialized memory.
+
+The trigger for this post is the [deprecation of `mem::uninitialized()`](https://blog.rust-lang.org/2019/07/04/Rust-1.36.0.html#maybeuninitt%3E-instead-of-mem::uninitialized) with Rust 1.36, but the post is just as relevant for C/C++ as it is for Rust.[^deprecate]
+
+[^deprecate]: This deprecation has been in the works for [more than two years](https://github.com/rust-lang/rfcs/pull/1892), and it has been almost a year since [I took over pushing for this](https://github.com/rust-lang/rfcs/pull/1892#issuecomment-410430449). I am very happy that we are finally there!
+
+<!-- MORE -->
+
+## The pitfalls of uninitialized memory
+
+When you allocate some memory (whether it be on the stack or heap), and you do not initialize it, what are its contents?
+We call this "uninitialized memory", but what exactly does this mean and what happens when it gets read?
+For many languages, this question is inconsequential: in Java, Haskell, OCaml and generally in all safe languages, uninitialized memory cannot be read, this is prevented by the type system.
+The same is true in safe Rust, actually.
+However, in unsafe Rust as well as in inherently unsafe languages such as C and C++, not having to initialize memory can be an important optimization, so this is a very relevant question.
+
+The C and C++ specifications (without going into all the detail here) say that uninitialized memory is "indeterminate", but the details of what exactly that means [are unclear](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1793.pdf).
+Many people will tell you that "uninitialized memory contains a random bit pattern".
+This is wrong.
+They might also talk about the system allocator or how the OS kernel allocates pages for the program to use.
+That is just irrelevant information.[^blame]
+
+[^blame]: To be clear, I am not blaming anyone for getting this wrong. There is tons of misinformation out there, and the standard is awfully ambiguous. That is why I am writing this post.
+
+Here is an example to demonstrate why "random bit pattern" cannot describe uninitialized memory:
+{% highlight rust %}
+use std::mem;
+
+fn always_returns_true(x: u8) -> bool {
+ x < 120 || x == 120 || x > 120
+}
+
+fn main() {
+ let x: u8 = unsafe { mem::MaybeUninit::uninit().assume_init() };
+ assert!(always_returns_true(x));
+}
+{% endhighlight %}
+**Update (2022-11-17):** Switched to `MaybeUninit` to keep the example working in newer versions of Rust.
+
+`always_returns_true` is a function that, clearly, will return `true` for any possible 8-bit unsigned integer.
+After all, *every* possible value for `x` will be either less than 120, equal to 120, or bigger than 120.
+A quick loop [confirms this](https://play.rust-lang.org/?version=stable&mode=release&edition=2018&gist=65b690fa3c1691e11d4d45955358cdbe).
+However, if you [run the example](https://play.rust-lang.org/?version=stable&mode=release&edition=2018&gist=c17d299cacd626c572def0c4262aed69), you can see the assertion fail.[^godbolt]
+
+[^godbolt]: In case this changes with future Rust versions, [here is the same example on godbolt](https://godbolt.org/z/9G67hP); the `xor eax, eax` indicates that the function returns 0, aka `false`. And [here is a version for C++](https://godbolt.org/z/TWrvcq).
+
+## What *is* uninitialized memory?
+
+How is this possible?
+The answer is that, in the "abstract machine" that is used to specify the behavior of our program, every byte in memory cannot just have a value in `0..256` (this is Rust syntax for a left-inclusive right-exclusive range), it can also be "uninitialized".
+Memory *remembers* if you initialized it.
+The `x` that is passed to `always_return_true` is *not* the 8-bit representation of some number, it is an uninitialized byte.
+Performing operations such as comparison on uninitialized bytes is [undefined behavior]({% post_url 2017-07-14-undefined-behavior %}).
+As a consequence, our program has undefined behavior, so we should not be surprised that it acts "weirdly".
+
+Of course, there is a reason for this undefined behavior; there is a reason the "abstract machine" is defined the way it is.
+Compilers don't just want to annoy programmers.
+Ruling out operations such as comparison on uninitialized data is useful, because it means the compiler does not have to "remember" which exact bit pattern an uninitialized variable has!
+A well-behaved (UB-free) program cannot observe that bit pattern anyway.
+So each time an uninitialized variable gets used, we can just use *any* machine register---and for different uses, those can be different registers!
+In the case of our example, the program actually compares such an "unobservable" bit pattern with a constant, so the compiler constant-folds the result to whatever it pleases.
+Because the value is allowed to be "unstable", the compiler does not have to make a "consistent choice" for the two comparisons, which would make such optimizations much less applicable.
+So, one time we "look" at `x` the compiler can pretend it is at least 150, and then when we look at it again it is at most 120, even though `x` did not change.
+That explains why our compiled example program behaves the way it does.
+[This LLVM document](http://nondot.org/sabre/LLVMNotes/UndefinedValue.txt) gives some more motivation for "unstable" uninitialized memory.
+
+When thinking about Rust (or C, or C++), you have to think in terms of an "abstract machine", not the real hardware you are using.
+Imagine that every byte in memory is either initialized to some value in `0..256`, or *uninitialized*.
+You can think of memory as storing an `Option<u8>` at every location.[^pointers]
+When new memory gets allocated for a local variable (on the stack) or on the heap, there is actually nothing random happening, everything is completely deterministic: every single byte of this memory is marked as *uninitialized*.
+Every location stores a `None`.
+(In LLVM, this `None` corresponds to `poison`, which [has the potential to replace `undef` entirely](http://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf).)
+
+When writing safe Rust, you do not have to worry about this, but this is the model that is good to have in your head when dealing with uninitialized memory in unsafe code.
+Alexis wrote a [great post](https://gankro.github.io/blah/initialize-me-maybe/) on which APIs to use for that in Rust; there is no need for me to repeat all that here.
+(In that post, Alexis says that every *bit* can be either 0, 1 or uninitialized, as opposed to every *byte* being initialized or not. Given that memory accesses happen at byte granularity, these two models are actually equivalent, at least in Rust which does not have C-style bitfields.)
+
+[^pointers]: In fact, [bytes are even more complicated than that]({% post_url 2018-07-24-pointers-and-bytes %}), but that is another topic.
+
+## What can you do with uninitialized memory?
+
+Now that we have a concrete way to talk about uninitialized memory, we can talk about which operations are allowed on values involving uninitialized bytes.
+My interpretation of the rules in C/C++, and my proposal for the rules in Rust, is that *any* operation working on the "value" of an integer (arithmetic and logical operations, comparisons, conditional jumps) is undefined behavior if any input is uninitialized.
+In particular, `x + 0` is UB if `x` is not initialized.
+However, this still leaves many questions open, such as whether even just *creating* an uninitialized `u8` is undefined behavior in Rust (which is the subject of [active discussion](https://github.com/rust-lang/unsafe-code-guidelines/issues/71)), or what happens when some but not all bytes of the input are uninitialized.
+Over time, we will come to some kind of compromise here.
+The important part (for both Rust and C/C++) however is that we have this discussion with a clear mental model in our minds for *what uninitialized memory is*.
+I see Rust on a good path here; I hope the C/C++ committees will eventually follow suit.
+
+Ruling out any operation on uninitialized values also makes it impossible to implement [this cute data structure](https://research.swtch.com/sparse).
+The `is-member` function there relies on the assumption that "observing" an uninitialized value (`sparse[i]`) twice gives the same result, which as we have seen above is not the case.
+This could be fixed by providing a "freeze" operation that, given any data, replaces the uninitialized bytes by *some* non-deterministically chosen *initialized* bytes.
+It is called "freeze" because its effect is that the value "stops changing each time you observe it".
+`is-member` would freeze `sparse[i]` once and then know for sure that "looking at it" twice will give consistent results.
+Unfortunately, since C/C++ do not acknowledge that their memory model is what it is, we do not have crucial operations such as "freeze" officially supported in compilers.
+At least for LLVM, that [might change though](http://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf).
+
+## "What the hardware does" considered harmful
+
+Maybe the most important lesson to take away from this post is that "what the hardware does" is most of the time *irrelevant* when discussing what a Rust/C/C++ program does, unless you *already established that there is no undefined behavior*.
+Sure, hardware (well, [most hardware](https://devblogs.microsoft.com/oldnewthing/20040119-00/?p=41003)) does not have a notion of "uninitialized memory".
+But *the Rust program you wrote does not run on your hardware*.
+It runs on the Rust abstract machine, and that machine (which only exists in our minds) *does* have a notion of "uninitialized memory".
+The real, physical hardware that we end up running the compiled program on is a very efficient *but imprecise* implementation of this abstract machine, and all the rules that Rust has for undefined behavior work together to make sure that this imprecision is not visible for *well-behaved* (UB-free) programs.
+But for programs that do have UB, this "illusion" breaks down, and [anything is possible](https://raphlinus.github.io/programming/rust/2018/08/17/undefined-behavior.html).
+
+*Only* UB-free programs can be made sense of by looking at their assembly, but *whether* a program has UB is impossible to tell on that level.
+For that, you need to think in terms of the abstract machine.[^sanitizer]
+
+[^sanitizer]: This does imply that tools like valgrind, that work on the final assembly, can never reliably detect *all* UB.
+
+This does not just apply to uninitialized memory: for example, in x86 assembly, there is no difference between "relaxed" and "release"/"acquire"-style atomic memory accesses.
+But when writing Rust programs, even when writing Rust programs that you only intend to compile to x86, "what the hardware does" just does not matter if your program has UB.
+The Rust abstract machine *does* make a distinction between "relaxed" and "release"/"acquire", and your program will go wrong if you ignore that fact.
+After all, x86 does not have "uninitialized bytes" either, and still our example program above went wrong.
+
+Of course, to explain *why* the abstract machine is defined the way it is, we have to look at optimizations and hardware-level concerns.
+But without an abstract machine, it is very hard to ensure that all the optimizations a compiler performs are consistent---in fact, both [LLVM](https://bugs.llvm.org/show_bug.cgi?id=35229) and [GCC](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65752) suffer from miscompilations caused by combining optimizations that all seem [fine in isolation, but together cause incorrect code generation]({% post_url 2020-12-14-provenance %}).
+The abstract machine is needed as an ultimate arbiter that determines which optimizations can be safely combined with each other.
+I also think that when writing unsafe code, it is much easier to keep in your head a fixed abstract machine as opposed to a set of optimizations that might change any time, and might or might not be applied in any order.
+
+Unfortunately, in my opinion not enough of the discussion around undefined behavior in Rust/C/C++ is focused on what concretely the "abstract machine" of these languages looks like.
+Instead, people often talk about hardware behavior and how that can be altered by a set of allowed optimizations---but the optimizations performed by compilers change as new tricks are discovered, and it's the abstract machines that define if these tricks are allowed.
+C/C++ have extensive standards that describe many cases of undefined behavior in great detail, but nowhere does it say that memory of the C/C++ abstract machine stores `Option<u8>` instead of the `u8` one might naively expect.
+In Rust, I am very happy that we have [Miri](https://github.com/rust-lang/miri/), which is meant to be (very close to) a direct implementation of the Rust abstract machine, and I am strongly advocating for us to write the Rust specification (once that gets going) in a style that makes this machine very explicit.
+I hope C/C++ will come around to do the same, and there is some [great work in that direction](https://www.cl.cam.ac.uk/~pes20/cerberus/), but only time will tell to what extend that can affect the standard itself.
+
+If you want to do me a favor, please spread the word!
+I am trying hard to combat the myth of "what the hardware does" in Rust discussions whenever I see it, but I obviously don't see all the discussions---so the next time you see such an argument around and undefined behavior is involved, no matter whether it is about uninitialized memory or [concurrency](http://hboehm.info/boehm-hotpar11.pdf) or [out-of-bounds memory accesses](https://github.com/rust-lang/rust/issues/32976#issuecomment-446775360) or anything else, please help by steering the discussion towards "what the Rust abstract machine does", and how we can design and adjust the Rust abstract machine in a way that it is most useful for programmers and optimizing compilers alike.
+
+As usual, if you have any comments, suggestions or questions, [let me know in the forums](https://internals.rust-lang.org/t/what-the-hardware-does-is-not-what-your-program-does-uninitialized-memory/10561).
+
+#### Footnotes
--- /dev/null
+---
+title: 'What Type Soundness Theorem Do You Really Want to Prove?'
+categories: rust research
+---
+
+My advisor Derek and some of my coauthors Amin, Robbert and Lars just put out a blog post on the [SIGPLAN blog](https://blog.sigplan.org/) on the idea of "semantic typing".
+This is the methodology behind [RustBelt](https://plv.mpi-sws.org/rustbelt/popl18/), so the post is a great starting point for understanding the context of that paper and the key bits of prior research that it rests on.
+In fact they used Rust as their main example language for that post, and I helped them out a bit with some of the technical details there so they kindly added me to the author list.
+
+The approach they describe is much more widely applicable than Rust though; it provides a whole new perspective on type systems that I think deserves way more attention than it gets.
+If you are interested in formal methods for type systems and in particular for Rust, you should check it out!
+It's a great read:
+
+> ["What Type Soundness Theorem Do You Really Want to Prove?" on *PL Perspectives*](https://blog.sigplan.org/2019/10/17/what-type-soundness-theorem-do-you-really-want-to-prove/)
--- /dev/null
+---
+title: "Stacked Borrows: An Aliasing Model for Rust (the paper)"
+categories: research rust
+reddit: /rust/comments/dy8avz/stacked_borrows_an_aliasing_model_for_rust_the/
+---
+
+I have [blogged]({% post_url 2019-05-21-stacked-borrows-2.1 %}) a [few]({% post_url 2019-04-30-stacked-borrows-2 %}) times before about [Stacked Borrows](https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md), my proposed aliasing model for Rust.
+But I am a researcher, and what researchers (are expected to) do is write papers---so that's what we did for Stacked Borrows as well.
+After a lot of work, I can now finally present to you our paper [Stacked Borrows: An Aliasing Model for Rust](https://plv.mpi-sws.org/rustbelt/stacked-borrows/) (thanks to open access, the paper is available under the CC-BY 4.0 license).
+
+<!-- MORE -->
+
+If you have already read my blog posts, the paper does not say much you have not seen already.
+However, the paper is the most coherent and most complete introduction to Stacked Borrows so far.
+So if you were always confused by some aspect of Stacked Borrows, or if you have not read the previous blog posts, the paper is a great way to learn about Stacked Borrows.
+
+If there is nothing new, you may wonder what took us so long, then, and why we needed 4 people for that?
+The answer is that my coauthor Hai, with some help from Jeehoon and me, took on the Herculean task of [formalizing Stacked Borrows in Coq](https://gitlab.mpi-sws.org/FP/stacked-borrows) and then formally verifying that some key optimizations are actually sound under that model.
+Those proofs turned out to be more complex to carry out than we expected, but now I am sure that what I claimed about Stacked Borrows in previous posts is actually correct.
+(In fact we were not able to complete the proof in time for the conference submission deadline, but lucky enough the reviewers were convinced enough by the informal description and the evaluation with Miri that they still accepted the paper.)
+
+Let me also use this opportunity to thank Mozilla and the Rust community for their help with this project.
+This work would not have been possible without Mozilla hiring me for two internships, without all the great feedback that you all gave me in response to my blog posts, and without all the support that I got in implementing Stacked Borrows, testing the standard library and a growing number of external crates in Miri, and fixing Stacked Borrows errors as they came up.
+Thank you all so much for working with me on this, this has been (and will hopefully continue to be) great fun. :-)
--- /dev/null
+---
+title: "How to Panic in Rust"
+categories: rust
+forum: https://internals.rust-lang.org/t/how-to-panic-in-rust/11368
+---
+
+What exactly happens when you `panic!()`?
+I recently spent a lot of time looking at the parts of the standard library concerned with this, and it turns out the answer is quite complicated!
+I have not been able to find docs explaining the high-level picture of panicking in Rust, so this feels worth writing down.
+
+<!-- MORE -->
+
+(Shameless plug: the reason I looked at this is that @Aaron1011 implemented unwinding support in Miri.
+I wanted to see that in Miri since forever and never had the time to implement it myself, so it was really great to see someone just submit PRs for that out of the blue.
+After a lot of review rounds, this landed just recently.
+There [are still some rough edges](https://github.com/rust-lang/miri/issues?q=is%3Aissue+is%3Aopen+label%3AA-panics), but the foundations are solid.)
+
+The purpose of this post is to document the high-level structure and the relevant interfaces that come into play on the Rust side of this.
+The actual mechanism of unwinding is a totally different matter (and one that I am not qualified to speak about).
+
+*Note:* This post describes panicking as of [this commit](https://github.com/rust-lang/rust/commit/4007d4ef26eab44bdabc2b7574d032152264d3ad).
+Many of the interfaces described here are unstable internal details of libstd, and subject to change any time.
+
+## High-level structure
+
+When trying to figure out how panicking works by reading the code in libstd, one can easily get lost in the maze.
+There are multiple layers of indirection that are only put together by the linker,
+there is the [`#[panic_handler]` attribute](https://doc.rust-lang.org/nomicon/panic-handler.html) and the ["panic runtime"](https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md) (controlled by the panic *strategy*, which is set via `-C panic`) and ["panic hooks"](https://doc.rust-lang.org/std/panic/fn.set_hook.html),
+and it turns out panicking in `#[no_std]` context takes an entirely different code path... there is just a lot going on.
+To make things worse, [the RFC describing panic hooks](https://github.com/rust-lang/rfcs/blob/master/text/1328-global-panic-handler.md) calls them "panic handler", but that term has since been re-purposed.
+
+I think the best place to start are the interfaces controlling the two indirections:
+
+* The *panic runtime* is used by libstd to control what happens after the panic information has been printed to stderr.
+ It is determined by the panic *strategy*: either we abort (`-C panic=abort`) or we unwind (`-C panic=unwind`).
+ (The panic runtime also provides the implementation for [`catch_unwind`](https://doc.rust-lang.org/std/panic/fn.catch_unwind.html) but we are not concerned with that here.)
+
+* The *panic handler* is used by libcore to implement (a) panics inserted by code generation (such as panics caused by arithmetic overflow or out-of-bounds array/slice indexing) and (b) the `core::panic!` macro (this is the `panic!` macro in libcore itself and in `#[no_std]` context in general).
+
+Both of these interfaces are implemented through `extern` blocks: listd/libcore, respectively, just import some function that they delegate to, and somewhere entirely else in the crate tree, that function gets implemented.
+The import only gets resolved during linking; looking locally at that code, one cannot tell where the actual implementation of the respective interface lives.
+No wonder that I got lost several times along the way.
+
+In the following, both of these interfaces will come up a lot; when you get confused, the first thing to check is if you just mixed up panic *handler* and panic *runtime*.
+(And remember there's also panic *hooks*, we will get to those.)
+That happens to me all the time.
+
+Moreover, `core::panic!` and `std::panic!` are *not* the same; as we will see, they take very different code paths.
+libcore and libstd each implement their own way to cause panics:
+
+* libcore's `core::panic!` does very little, it basically just delegates to the panic *handler* immediately.
+* libstd's `std::panic!` (the "normal" `panic!` macro in Rust) triggers a fully-featured panic machinery that provides a user-controlled [panic *hook*](https://doc.rust-lang.org/std/panic/fn.set_hook.html).
+ The default hook will print the panic message to stderr.
+ After the hook is done, libstd delegates to the panic *runtime*.
+
+ libstd also provides a panic *handler* that calls the same machinery, so `core::panic!` also ends up here.
+
+
+Let us now look at these pieces in a bit more detail.
+
+## Panic Runtime
+
+The [interface to the panic runtime](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L53) (introduced by [this RFC](https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md)) is a function `__rust_start_panic(payload: usize) -> u32` that gets imported by libstd, and that is later resolved by the linker.
+
+The `usize` argument here actually is a `*mut &mut dyn core::panic::BoxMeUp` -- this is where the "payload" of the panic (the information available when it gets caught) gets passed in.
+`BoxMeUp` is an unstable internal implementation detail, but [looking at the trait](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L268-L281) we can see that all it really does is wrap a `dyn Any + Send`, which is the [type of the panic payload](https://doc.rust-lang.org/std/thread/type.Result.html) as returned by `catch_unwind` and `thread::spawn`.
+`BoxMeUp::take_box` returns a `Box<dyn Any + Send>`, but as a raw pointer (because `Box` is not available in the context where this trait is defined); `BoxMeUp::get` just borrows the contents.
+
+The two implementations of this interface Rust ships with are [`libpanic_unwind`](https://github.com/rust-lang/rust/tree/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libpanic_unwind) for `-C panic=unwind` (the default on most platforms) and [`libpanic_abort`](https://github.com/rust-lang/rust/tree/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libpanic_abort) for `-C panic=abort`.
+
+## `std::panic!`
+
+On top of the panic *runtime* interface, libstd implements the default Rust panic machinery in the internal [`std::panicking`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs) module.
+
+#### `rust_panic_with_hook`
+
+The key function that almost everything passes through is [`rust_panic_with_hook`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L439-L441):
+{% highlight rust %}
+fn rust_panic_with_hook(
+ payload: &mut dyn BoxMeUp,
+ message: Option<&fmt::Arguments<'_>>,
+ file_line_col: &(&str, u32, u32),
+) -> !
+{% endhighlight %}
+This function takes a panic source location, an optional unformatted panic message (see the [`fmt::Arguments`](https://doc.rust-lang.org/std/fmt/struct.Arguments.html) docs), and a payload.
+
+Its main job is to call whatever the current panic hook is.
+Panic hooks have a [`PanicInfo`](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html) argument, so we need a panic source location, format information for a panic message, and a payload.
+This matches `rust_panic_with_hook`'s arguments quite nicely!
+`file_line_col` and `message` can be used directly for the first two elements; `payload` gets turned into `&(dyn Any + Send)` through the `BoxMeUp` interface.
+
+Interestingly, the *default* panic hook entirely ignores the `message`; what you actually see printed is [the `payload` downcast to `&str` or `String`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L176-L182) (whatever works).
+Supposedly, the caller should ensure that formatting `message`, if present, gives the same result.
+(And the ones we discuss below do ensure this.)
+
+Finally, `rust_panic_with_hook` dispatches to the current panic *runtime*.
+At this point, only the `payload` is still relevant -- and that is important: `message` (as the `'_` lifetime indicates) may contain short-lived references, but the panic payload will be propagated up the stack and hence must be `'static`.
+The `'static` bound is quite well hidden there, but after a while I realized that [`Any`](https://doc.rust-lang.org/std/any/trait.Any.html) implies `'static` (and remember `dyn BoxMeUp` is just used to obtain a `Box<dyn Any + Send>`).
+
+#### libstd panicking entry points
+
+`rust_panic_with_hook` is a private function to `std::panicking`; the module provides three entry points on top of this central function, and one that circumvents it:
+
+* the [`begin_panic_handler`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L332), the default panic handler implementation that backs (as we will see) panics from `core::panic!` and built-in panics (from arithmetic overflow or out-of-bounds array/slice indexing).
+ This obtains as input a [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html), and it has to turn that into arguments for `rust_panic_with_hook`.
+ Curiously, even though the components of `PanicInfo` and the arguments of `rust_panic_with_hook` match up pretty well and seem like they could just be forwarded, that is *not* what happens.
+ Instead, libstd entirely *ignores* the `payload` component of the `PanicInfo`, and sets up the actual payload (passed to `rust_panic_with_hook`) such that it contains [the formatted `message`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L350).
+
+ In particular, this means that the panic *runtime* is irrelevant for `no_std` applications.
+ It only comes into play when libstd's panic handler implementation is used.
+ (The panic *strategy* selected via `-C panic` still has an effect as it also influences code generation.
+ For example, with `-C panic=abort` code can become simpler as it does not need to support unwinding.)
+
+* [`begin_panic_fmt`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L316), backing the [format string version of `std::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L22-L23) (i.e., this is used when you pass multiple arguments to the macro).
+ This basically just packages the format string arguments into a `PanicInfo` (with a [dummy payload](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L56)) and calls the default panic handler that we just discussed.
+
+* [`begin_panic`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L392), backing the [single-argument version of `std::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L16).
+ Interestingly, this uses a very different code path than the other two entry points!
+ In particular, this is the only entry point that permits passing in an *arbitrary payload*.
+ That payload is just [converted into a `Box<dyn Any + Send>`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L419) so that it can be passed to `rust_panic_with_hook`, and that's it.
+
+ In particular, a panic hook that looks at the `message` field of the `PanicData` it is passed will *not* be able to see the message in a `std::panic!("do panic")`, but it *will* see the message in a `std::panic!("panic with data: {}", data)` as the latter passes through `begin_panic_fmt` instead.
+ That seems quite surprising. (But also note that `PanicData::message()` is not stable yet.)
+
+* [`rust_panic_without_hook`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/panicking.rs#L496) is the odd one out: this entry point backs [`resume_unwind`](https://doc.rust-lang.org/nightly/std/panic/fn.resume_unwind.html), and it actually does *not* call the panic hook.
+ Instead, it dispatches to the panic runtime immediately.
+ Like, `begin_panic`, it lets the caller pick an arbitrary payload.
+ Unlike `begin_panic`, the caller is responsible for boxing and unsizing the payload; `update_count_then_panic` just forwards that pretty much verbatim to the panic runtime.
+
+## Panic Handler
+
+All of the `std::panic!` machinery is really useful, but it relies on heap allocations through `Box` which is not always available.
+To give libcore a way to cause panics, [panic handlers were introduced](https://github.com/rust-lang/rfcs/blob/master/text/2070-panic-implementation.md).
+As we have seen, if libstd is available, it provides an implementation of that interface to wire `core::panic!` into the libstd panic machinery.
+
+The [interface to the panic handler](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L78) is a function `fn panic(info: &core::panic::PanicInfo) -> !` that libcore imports and that is later resolved by the linker.
+The [`PanicInfo` ](https://doc.rust-lang.org/core/panic/struct.PanicInfo.html) type is the same as for panic hooks: it contains a panic source location, a panic message, and a payload (a `dyn Any + Send`).
+The panic message is represented as [`fmt::Arguments`](https://doc.rust-lang.org/std/fmt/struct.Arguments.html), i.e., a format string with its arguments that has not been formatted yet.
+
+## `core::panic!`
+
+On top of the panic handler interface, libcore provides a [minimal panic API](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs).
+The [`core::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L10-L26) macro creates a `fmt::Arguments` which is then [passed to the panic handler](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L82).
+No formatting happens here as that would require heap allocations; this is why `PanicInfo` contains an "uninterpreted" format string with its arguments.
+
+Curiously, the `payload` field of the `PanicInfo` that gets passed to the panic handler is always set to [a dummy value](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panic.rs#L56).
+This explains why the libstd panic handler ignores the payload (and instead constructs a new payload from the `message`), but that makes me wonder why that field is part of the panic handler API in the first place.
+Another consequence of this is that [`core::panic!("message")`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L15) and [`std::panic!("message")`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libstd/macros.rs#L16) (the variants without any formatting) actually result in very different panics: the former gets turned into `fmt::Arguments`, passed through the panic handler interface, and then libstd creates a `String` payload by formatting it.
+The latter, however, directly uses the `&str` as a payload, and the `message` field remains `None` (as already mentioned).
+
+Some elements of the libcore panic API are lang items because the compiler inserts calls to these functions during code generation:
+* The [`panic` lang item](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L39) is called when the compiler needs to raise a panic that does not require any formatting (such as arithmetic overflow); this is the same function that also backs single-argument [`core::panic!`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L15).
+* The [`panic_bounds_check`](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/panicking.rs#L55) lang item is called on a failed array/slice bounds check.
+ It calls into the same method as [`core::panic!` with formatting](https://github.com/rust-lang/rust/blob/4007d4ef26eab44bdabc2b7574d032152264d3ad/src/libcore/macros/mod.rs#L21-L24).
+
+## Conclusion
+
+We have walked through 4 layers of APIs, 2 of which are indirected through imported function calls and resolved by the linker.
+That's quite a journey!
+But we have reached the end now.
+I hope you [didn't panic](https://en.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27s_Guide_to_the_Galaxy#Don't_Panic) yourself along the way. ;)
+
+I mentioned some things as being surprising.
+Turns out they all have to do with the fact that panic hooks and panic handlers share the `PanicInfo` struct in their interface, which contains *both* an optional not-yet-formatted `message` and a type-erased `payload`:
+* The panic *hook* can always find the already formatted message in the `payload`, so the `message` seems pointless for hooks.
+ In fact, `message` can be missing even if `payload` contains a message (e.g., for `std::panic!("message")`).
+* The panic *handler* will never actually receive a useful `payload`, so that field seems pointless for handlers.
+
+Reading the [panic handler RFC](https://github.com/rust-lang/rfcs/blob/master/text/2070-panic-implementation.md), it seems like the plan was for `core::panic!` to also support arbitrary payloads, but so far that has not materialized.
+However, even with that future extension, I think we have the invariant that when `message` is `Some`, then either `payload == &NoPayload` (so the payload is redundant) or `payload` is the formatted message (so the message is redundant).
+I wonder if there is any case where *both* fields will be useful -- and if not, couldn't we encode that by making them two variants of an `enum`?
+There are probably good reasons against that proposal and for the current design; it would be great to get them documented somewhere. :)
+
+There is a lot more to say, but at this point, I invite you to follow the links to the source code that I included above.
+With the high-level structure in mind, you should be able to follow that code.
+If people think this overview would be worth putting somewhere more permanently, I'd be happy to work this blog post into some form of docs -- I am not sure what would be a good place for those, though.
+And if you find any mistakes in what I wrote, please let me know!
--- /dev/null
+---
+title: "Debugging rustc type layouts"
+categories: rust
+forum: https://internals.rust-lang.org/t/psa-debugging-rustc-type-layouts/12082
+---
+
+This post is a "public service announcement" for people working on the guts of rustc.
+I wish I had known about this a year ago, so I hope this post can make this feature more widely known.
+
+<!-- MORE -->
+
+When working with MIR in rustc, one key data structure that comes up a lot is [`Layout`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_target/abi/struct.Layout.html) (formerly `LayoutDetails`), usually paired up with a type in a `TyAndLayout` (formerly `TyLayout`).
+This data structure describes everything that there is to know about how a type "looks like" in memory: size and alignment of the entire type, at which offset we can find which field, how enum variants are represented, which "niche" can be used in this type to optimize enums.
+
+`Layout` is quite versatile and can be hard to interpret, and when debugging Miri I regularly have to know what exactly the `Layout` of a certain type looks like or what exactly some aspect of `Layout` actually *means* in practice.
+While debugging MIR is easy via `rustc --emit mir` or the "MIR" button on the playground, debugging `Layout` was much more tedious.
+But not any more. :)
+
+All you have to do is enter the following code in [the playground](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=7abc008aed2669466d4ebe79ee7767fe):
+
+{% highlight rust %}
+#![feature(rustc_attrs)]
+
+#[rustc_layout(debug)]
+type T = (u8, u16);
+{% endhighlight %}
+
+The (permanently) unstable `rustc_layout` attribute [can now be used](https://github.com/rust-lang/rust/pull/69901) to dump some information about the type it is attached to (also works with `struct`/`enum`/`union` definitions).
+In this case, it prints:
+
+```
+error: layout_of((u8, u16)) = Layout {
+ fields: Arbitrary {
+ offsets: [
+ Size {
+ raw: 0,
+ },
+ Size {
+ raw: 2,
+ },
+ ],
+ memory_index: [
+ 0,
+ 1,
+ ],
+ },
+ variants: Single {
+ index: 0,
+ },
+ abi: ScalarPair(
+ Scalar {
+ value: Int(
+ I8,
+ false,
+ ),
+ valid_range: 0..=255,
+ },
+ Scalar {
+ value: Int(
+ I16,
+ false,
+ ),
+ valid_range: 0..=65535,
+ },
+ ),
+ largest_niche: None,
+ align: AbiAndPrefAlign {
+ abi: Align {
+ pow2: 1,
+ },
+ pref: Align {
+ pow2: 3,
+ },
+ },
+ size: Size {
+ raw: 4,
+ },
+}
+```
+
+That is quite a lot, but it contains all the key information about this type:
+the fields are at offsets 0 and 2, the type has alignment 2 (but preferred alignment 8) and size 4.
+We can also see that it uses the `ScalarPair` abi which is relevant for Miri and when passing data as arguments to another function.
+To learn more about what all this information means, see [the `Layout` type docs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_target/abi/struct.Layout.html).
+
+**Update:** After a suggestions by @jschievink, this can now also be used to print the [underlying type and layout](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=1de2bed0c0d0f9171bfb41969f5028fb) of named opaque types, which is particularly useful [for generators](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=de99ab78a4d77bceee6760021b19de7d). **/Update**
+
+So the next time you work with `Layout` and wonder how exactly the niche gets represented, or whether an `enum` can have `ScalarPair` abi (hint: yes it can), you can easily look at a few examples to see how rustc thinks about this type internally.
+This is basically the type-level equivalent of `--emit mir`.
+I have wanted this since forever, so much that some time ago I wrote an awful hack for this based on rustc debug tracing.
+Only very recently did I learn about the `rustc_layout` attribute and then I had to immediately extend it to support dumping all the information.
+Now `Layout` can be debugged in the browser on the playground, which is so much more convenient. :D
--- /dev/null
+---
+title: "Why even unused data needs to be valid"
+categories: rust
+forum: https://internals.rust-lang.org/t/why-even-unused-data-needs-to-be-valid/12734
+---
+
+The Rust compiler has a few assumptions that it makes about the behavior of all code.
+Violations of those assumptions are referred to as [Undefined Behavior][ub].
+Since Rust is a safe-by-default language, programmers usually do not have to worry about those rules (the compiler and libraries ensure that safe code always satisfies the assumptions),
+but authors of `unsafe` code are themselves responsible for upholding these requirements.
+
+Those assumptions are [listed in the Rust reference](https://doc.rust-lang.org/reference/behavior-considered-undefined.html).
+The one that seems to be most surprising to many people is the clause which says that Rust code may not *produce* "[...] an invalid value, even in private fields and locals".
+The reference goes on to explain that "*producing* a value happens any time a value is assigned to or read from a place, passed to a function/primitive operation or returned from a function/primitive operation".
+In other words, even just *constructing*, for example, an invalid `bool`, is Undefined Behavior---no matter whether that `bool` is ever actually "used" by the program.
+The purpose of this post is to explain why that rule is so strict.
+
+[ub]: https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#undefined-behavior
+
+<!-- MORE -->
+
+First of all, let me clarify what is meant by "used" here, as that term is used to mean very different things.
+The following code "uses" `b`:
+
+{% highlight rust %}
+fn example(b: bool) -> i32 {
+ if b { 42 } else { 23 }
+}
+{% endhighlight %}
+
+I hope it is not very surprising that calling `example` on, e.g., `3` transmuted to `bool` is Undefined Behavior (UB).
+When compiling `if`, the compiler assumes that `0` and `1` are the only possible values; there is no saying what could go wrong when that assumption is violated.
+For example, the compiler might use a [jump table](https://en.wikipedia.org/wiki/Branch_table); an out-of-bounds index in that table could literally execute any code, so there is no way to bound the behavior in that case.
+(This is a compiler-understood *validity invariant* that is fixed in the language specification, which is very different from a user-defined *safety invariant*.
+See [this earlier post]({% post_url 2018-08-22-two-kinds-of-invariants %}) for more details on that distinction.)
+
+What is less obvious is why calling `example` on `3` is UB even when there is no such `if` being executed.
+To understand why that is important, let us consider the following example:
+
+{% highlight rust %}
+fn example(b: bool, num: u32) -> i32 {
+ let mut acc = 0;
+ for _i in 0..num {
+ acc += if b { 42 } else { 23 };
+ }
+ acc
+}
+{% endhighlight %}
+
+Now assume we were working in a slightly different version of Rust, where transmuting `3` to a `bool` is fine as long as you do not "use" the `bool`.
+That would mean that calling `example(transmute(3u8), 0)` is actually allowed, because in that case the loop never gets executed, so we never "use" `b`.
+
+However, this is a problem for a very important transformation called [loop-invariant code motion](https://en.wikipedia.org/wiki/Loop-invariant_code_motion).
+That transformation can be used to turn our `example` function into the following:
+
+{% highlight rust %}
+fn example(b: bool, num: u32) -> i32 {
+ let mut acc = 0;
+ let incr = if b { 42 } else { 23 };
+ for _i in 0..num {
+ acc += incr;
+ }
+ acc
+}
+{% endhighlight %}
+
+The increment `if b { 42 } else { 23 }`, now called `incr`, is "invariant" during the execution of the loop, and thus computing the increment can be moved out.
+Why is this a good transformation?
+Instead of determining the increment each time around the loop, we do that just once, thus saving a lot of conditional jumps that the CPU is unhappy about.
+This also enables further transformations down the road, e.g. the compiler could notice that this is just `num*incr`.
+
+However, in our hypothetical Rust where "unused" values may be invalid, this important optimization is actually incorrect!
+To see why, consider again calling `example(transmute(3u8), 0)`.
+Before the optimization, that call was fine.
+After the optimization, that call is UB because we are doing `if b` where `b` is `3`.
+
+This is because loop-invariant code motion makes dead code live when the loop is not actually executed.
+(We can think of this as a form of "speculative execution", though entirely unrelated to CPU-level speculative execution.)
+To fix the optimization, we could require the compiler to prove that the loop runs at least once (i.e., we could avoid speculative execution), but in general that will be hard (and in `example` it is impossible, since `num` can indeed be `0`).
+Another option is to restructure the code to only compute `incr` if `num > 0`, but again this can be hard to do in general.
+So the alternative that Rust uses is to require that even "unused" data satisfies some basic validity.
+That makes `example(transmute(3u8), 0)` UB in *both* versions of `example`, and as such the optimization is correct for this input (and indeed for all possible inputs).
+
+Now, one might argue that `b` in the example here was not truly "unused", it was just "used in dead code".
+So instead of saying that `b` must be *always* valid, we could somehow try to make the use of `b` in dead code affect whether or not the program is UB.
+However, that is a huge can of worms.
+Right now, we have the fundamental principle that *dead code cannot affect program behavior*.
+This principle is crucial for tools like [Miri](https://github.com/rust-lang/miri/): since Miri is an interpreter, it never even sees dead code.
+I would argue that being able to have tools like Miri is hugely important, and it is worth having a semantics that enables such tools to exist.
+But this means our hands are pretty much tied: since we cannot take into account of `b` is "used in dead code", we simply have to require `b` to always be valid, no matter whether and where it is "used" or not.
+To support inlining and outlining, we also do not want the function boundary to be relevant, which ultimately leads us to the rule that Rust requires today: whenever data of a given type is *produced* anywhere, the data needs to be valid for that type.
+
+I hope this post was helpful in explaining why Undefined Behavior in Rust is defined the way it is.
+As usual, if you have any comments or questions, let me know in the [forums](https://internals.rust-lang.org/t/why-even-unused-data-needs-to-be-valid/12734).
--- /dev/null
+---
+title: "Outlook.com Considered Harmful"
+categories: rant sysadmin
+---
+
+I am administrating the server that hosts this blog myself, and this includes running and maintaining my own mail server.
+Email is a messy and complicated bit of infrastructure, born in a more innocent age where everyone connected to the internet trusted each other.
+Over time, is has grown many layers of band-aids to provide at least some level of security.
+Email is also beautiful, it is the most widely used example of federated infrastructure: in principle, anybody can set up a mail server and communicate with everyone else, no matter which provider they are using.
+This is an amount of organizational resilience and user freedom that most messenger services can only dream of.
+
+Perhaps surprisingly, I have had very little trouble with my own server; most big email providers do a good job blocking spam while permitting small independent mail servers to operate smoothly (this includs even Gmail, to my astonishment).
+There is just one exception: Microsofts Outlook.com (formerly Hotmail.com) and the other services using the same underlying infrastructure (such as live.com).
+
+<!-- MORE -->
+
+Over the years, except for one brief issue with AT&T, only Microsoft ever rejected mails from my server.
+This recently started happening again:
+
+```
+<...>: host eur.olc.protection.outlook.com[104.47.22.161]
+ said: 550 5.7.1 Unfortunately, messages from [109.230.236.95] weren't sent.
+ Please contact your Internet service provider since part of their network
+ is on our block list (S3140). You can also refer your provider to
+ http://mail.live.com/mail/troubleshooting.aspx#errors.
+ [DB8EUR06FT011.eop-eur06.prod.protection.outlook.com] (in reply to MAIL
+ FROM command)
+```
+
+So somehow, this server outright blocks everything coming from my IP, even though I am not listed on the usual blacklists.
+Usually, the process is to find the form that Microsoft expects sysadmins to fill out for such cases (the URL of that form keeps changing and even much of Microsoft's documentation points to broken pages; the current form seems to be [this one](https://support.microsoft.com/en-us/supportrequestform/8ad563e3-288e-2a61-8122-3ba03d6b8d75)).
+I fill out that form, and after a week or so emails work again.
+I had to do this maybe four times over the last eight years (compared to once for AT&T and never for every other provider).
+Last time, their guidelines insisted that I set up [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), which I did even though I am not a fan of that technology since it breaks email forwarding.
+The better alternative is [DKIM](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail), but amusingly, Microsoft [does not implement DKIM](https://sendersupport.olc.protection.outlook.com/pm/policies.aspx) ("Microsoft currently only validates inbound mail via SPF and Sender ID authentication").
+
+However, setting up SPF seemingly was not enough.
+This time, my request for unblocking my IPs was rejected.
+I requested that I be told the reason for this, since as far as I am aware I am following all the usual best practices for mail servers.
+The only response I was able to get is
+
+> As previously stated, your IP's
+> (109.230.236.95,
+> 109.230.224.234)
+> do not qualify for mitigation at this time. I do apologize, but I am unable to provide any details about this situation since we do not have the liberty to discuss the nature of the block.
+
+They did claim that "Hotmail customers have reported email from this IP as unwanted", but were unwilling to provide any details, so this is not an issue that I can diagnose.
+Instead each new response linked to a different [set of guidelines](https://postmaster.live.com/pm/postmaster.aspx) or [whitepaper](https://download.microsoft.com/download/e/3/3/e3397e7c-17a6-497d-9693-78f80be272fb/enhance_deliver.pdf).
+The only explanation I have is that my mailman instance could have caused [backscatter spam](https://en.wikipedia.org/wiki/Backscatter_(email)), which I have since tried to mitigate.
+
+They also referred me to various "programs" I could join to help manage my IPs reputation, but all of these programs require a Microsoft account.
+I have created such accounts in the past several times, most recently to experiment with Azure CI, and after a few months when I try to log in, I always get a message like this:
+
+
+
+(They also reject "+" characters in email addresses, hence the use of an "_" above.
+They say "+" is an invalid character, which of course is just plain wrong.)
+
+I am not sure what "activity" they have detected while I was not using my account for a month (yes I used a strong password); maybe that on its own is already violating their ToS.
+Subsequently, they insist on me giving them my mobile phone number.
+Needless to say, that is an entirely unacceptable request; my phone number is a too sensitive piece of personal information (comparable to my physical mail addresses) for me to share it with Microsoft.
+Better stay away from Azure CI if you care about such matters.
+
+Given the lack of proper security for text messages, this process is also unsuited to establish account security; if that was really their interest, they would insist on setting up two-factor authentication via some standard [OTP protocol](https://en.wikipedia.org/wiki/One-time_password#Standardization).
+But this is not about the security of my account, it is a cheap method to prevent mass account creation and a thinly veiled attempt to collect more personal information.
+Google at least requests the phone number upon account creation (so I know to stay away from the beginning); Microsoft is more sinister in that they let you use their service for some time so you start relying on it before they force you to hand over your phone number or be locked out.
+
+To conclude, if you are using Outlook.com for your email, I am afraid I cannot respond to messages you are sending to me.
+I can only recommend finding a better email provider.
+Microsoft clearly could not care less about a decentralized internet; their policies are actively driving us towards a small oligopoly of big email providers with little choice left for users.
+They are not even themselves implementing a reasonable minimum of email security technology (such as DKIM), but at the same time block servers without any means for sysadmins to learn what they are accused of.
+I am only happy that I hardly notice this because most people already use more reasonable email providers.
--- /dev/null
+---
+title: "Achievement 'PhD' unlocked"
+categories: research rust
+---
+
+It is done!
+My dissertation is finally complete.
+So if you always wanted an in-depth account of my research on Rust (and more),
+[you can go read my dissertation](https://people.mpi-sws.org/~jung/thesis.html).
+With its almost 300 pages, it should keep you busy for a while. ;)
+
+<!-- MORE -->
+
+This also means that after more than six years, my time as a PhD student has come to an end---in fact, I was a student for more than 10 years and that time is also over.
+What a strange feeling.
+
+To make this even stranger, hardly anything will actually change right now.
+Due to the pandemic, my plans of changing cities (continents even) have been postponed.
+The most immediate effect is that I can finally switch from writing and editing to more technical work (in particular, work that does not involve dealing with LaTeX...).
+I have been looking forward to this for quite some time now.
+I'll be looking into some topics that I have not worked on so far, so there will be lots of reading and learning and that is all very exciting.
+But don't worry, I will also still be doing Rusty things.
+
+I am not sure what the next 10 years will bring, but they will have a really hard time to surpass the previous 10, and that is mostly thanks to the wonderful people around me.
+A huge THANK YOU to everyone who supported me on this journey and made it the time of my life. :-)
--- /dev/null
+---
+title: "What (not so) recently happened in Miri"
+categories: rust
+reddit: /rust/comments/j1fxd3/what_not_so_recently_happened_in_miri/
+---
+
+A lot has happened in Miri over the last year and a half, and I figured it would be a good idea to advertise all this progress a bit more widely, so here we go.
+We also recently performed a breaking change that affects some CI configurations, so this post serves as an announcement for you to update your CI configuration if needed.
+
+For the uninitiated, [Miri](https://github.com/rust-lang/miri/) is an interpreter that runs your Rust code and checks if it triggers any [Undefined Behavior](https://doc.rust-lang.org/reference/behavior-considered-undefined.html).
+You can think of it a as very thorough (and very slow) version of valgrind: Miri will detect when your program uses uninitialized memory incorrectly, performs out-of-bounds memory accesses or pointer arithmetic, violates key language invariants, does not ensure proper pointer alignment, or causes incorrect aliasing.
+As such, it is most helpful when writing unsafe code, as it aids in ensuring that you follow all the rules required for unsafe code to be correct and safe.
+Miri also detects memory leaks, i.e., it informs you at the end of program execution if there is any memory that was not deallocated properly.
+
+<!-- MORE -->
+
+However, being an interpreter, Miri is limited in the kinds of code it can execute -- everything that would usually involve interacting with C libraries or the operating system needs to be specifically supported, as C code cannot be interpreted by Miri.
+Miri also lacks support for some Rust features that are hard to interpret, but we are slowly closing these gaps.
+
+## Recent and past progress in Miri
+
+During the last 1.5 years, thanks to a series of excellent contributors, we made a lot of progress towards supporting more and more Rust code to run in Miri.
+I am going to list some highlights below.
+
+If you want to learn how to use Miri yourself, scroll down to the end of this post.
+If you are using Miri already, maybe you are still passing flags like `--exclude-should-panic` or disabling tests that require concurrency; you should be able to update those flags now.
+Also note the breaking change in how `cargo miri` interprets CLI arguments below!
+
+### Randomness and `HashMap`
+
+The Rust `HashMap` picks a new random seed for each execution.
+This seed in obtained from the operating system, an operation which Miri did not support until @Aaron1011 implemented `getrandom` ([#683](https://github.com/rust-lang/miri/pull/683)).
+To ensure the same programs behaves the same way each time it is run by Miri, Miri internally uses a deterministic RNG (seeded with `0`, but that can be changed via `-Zmiri-seed`) to implement getrandom.
+This PR also enabled Miri to be used with projects that use the `rand` crate for randomness.
+
+However, this also means randomness in Miri is actually not random, so *do not use Miri to perform any important cryptographic operations*.
+
+### Unwinding
+
+Miri used to just abort program execution in case of a panic.
+To better match the behavior of real Rust programs, @Aaron1011 implement proper unwinding support in Miri ([#693](https://github.com/rust-lang/miri/pull/693)).
+He even implemented catching panics again, which required aligning quite a few pieces across rustc, the standard library, and Miri itself.
+This means Miri can finally also execute `#[should_panic]` tests.
+Since recently, this is supported even for Windows targets.
+
+### Pointer-integer casts
+
+Thanks to @christianpoveda, Miri now properly supports casting arbitrary pointers to integers and back ([#779](https://github.com/rust-lang/miri/pull/779)).
+
+Recently, I also adjusted the alignment check to fully take this information into account, so that Miri can now run code that performs its own alignment logic ([#1513](https://github.com/rust-lang/miri/pull/1513)).
+Notice however that this can lead to code that just happens to work by pure chance; to properly test such code, the test should be run at least 10 times.
+
+### File system access
+
+@christianpoveda went on to implement file system access (this series of PRs started with [#962](https://github.com/rust-lang/miri/pull/962)).
+Later, @divergentdave improved that support with directory listing and some related operations (starting with [#1152](https://github.com/rust-lang/miri/pull/1152)).
+This means programs running in Miri can now read from and write to files on the host computer.
+This is the first form of communication that we support between the interpreted program and the outside world.
+Communication needs to be explicitly requested via `-Zmiri-disable-isolation`; by default, Miri isolates the program to ensure that each execution is perfectly reproducible.
+
+File system access is only supported on Linux and macOS targets, but due to cross-interpretation this is not a problem even for Windows users -- see the next point.
+
+### Cross-interpretation
+
+Based on earlier work by @Aaron1011 who made Miri use check-only builds both for the standard library and the interpreted crate itself ([#1136](https://github.com/rust-lang/miri/pull/1136)),
+I made Miri support "cross-interpretation" ([#1249](https://github.com/rust-lang/miri/pull/1249)).
+This means even when you are on a Windows host, you can pass `--target x86_64-unknown-linux-gnu` so Miri will interpret the program *as if* it was running on Linux, in particular using all the Linux parts of the standard library for the interaction with the operating system.
+Sine Miri supports the Linux APIs for file system access, it can interpret these programs even when running on a Windows host.
+
+This is particularly useful when testing target features that differ from the host platform: for example, even on a 64bit macOS host, you can run programs for the 32bit Linux target (`--target i686-unknown-linux-gnu`), making sure your logic works for different pointer sizes.
+Miri also supports big-endian targets like `--target mips64-unknown-linux-gnuabi64`, so if your code is endianess-sensitive, you can test if it behaves correctly on big-endian systems.
+And finally cross-interpretation was enormously helpful for developing Miri itself; for example, I relied on this when fixing up our panic and unwinding support for Windows targets.
+
+### Concurrency
+
+Earlier this year, @vakaras surprised me by suddenly showing up with a series of patches that equip Miri with support for concurrency ([#1284](https://github.com/rust-lang/miri/pull/1284)).
+This is work he did during an internship with Amazon, so also thank you to Amazon for sponsoring this work!
+Now Miri programs can spawn threads and interact via locks or atomics.
+There are some caveats though: Miri does not detect data races, so programs with incorrect synchronization can cause Undefined Behavior through data races without Miri noticing.
+Also Miri's scheduler is rather crude, so programs can be stuck in infinite loops under some circumstances.
+
+### Better `cargo` compatibility (breaking change!)
+
+Recently, I mostly re-wrote the main entry point for users to execute programs in Miri, `cargo miri` ([#1540](https://github.com/rust-lang/miri/pull/1540)).
+It is now more compatible with cargo itself: `cargo test` and `cargo miri test` support the exact same flags, and likewise for `cargo run` and `cargo miri run`.
+
+However, this required a breaking change: previously, the way to pass flags to Miri itself and the program when executing the test suite was `cargo miri test -- <miri flags> -- <test suite flags>`.
+Now flags are passed via `cargo miri test -- <test suite flags>` like they are with `cargo test`; if you need to pass flags to Miri, you can set the `MIRIFLAGS` variable which works like `RUSTFLAGS`.
+I also removed support for `cargo miri` without further arguments, which used to be an alias for `cargo miri run`.
+The reason is that (a) `cargo miri test` is actually used much more frequently and (b) disambiguating these options while also supporting arbitrary flags is tricky.
+
+If you have set up your CI to run tests in Miri, please make sure to adjust your configuration to the new format.
+For now, Miri still supports the old style (and emits an appropriate warning), but the intention is to remove that support code eventually.
+If your project is hosted on GitHub and is affected by the change, you should have already received a notification from me, but I might have missed some projects and of course not everything is on GitHub.
+While at it, you can also remove `cargo miri setup` from your CI script; that is no longer needed as thanks to @dtolnay Miri automatically detects when it runs on CI and goes into non-interactive mode.
+
+### ... and more
+
+This list is by far not exhaustive.
+Many small functions, from trigonometry to environment variable access to timekeeping, have been implemented over the last months, ever growing the range of programs that Miri can execute.
+Thank you to @Aaron1011, @christianpoveda, @divergentdave, @JOE1994, and @samrat!
+I hope I did not miss anyone...
+
+## Using Miri
+
+If this post made you curious and you want to give Miri a try, here's how to do that.
+Assuming you have a crate with some unsafe code, and you already have a test suite (you are testing your unsafe code, right?), you can just install Miri (`rustup +nightly component add miri`) and then run `cargo +nightly miri test` to execute all tests in Miri (except for doctests, which are not supported yet).
+Note that this requires the nightly toolchain as Miri is still an experimental tool.
+
+Miri is very slow, so it is likely that some tests will take way too long to be feasible.
+You can adjust iteration counts in Miri without affecting non-Miri testing as follows:
+{% highlight rust %}
+let limit = if cfg!(miri) { 10 } else { 10_000 };
+{% endhighlight %}
+If your test suite needs to access OS facilities such as timers or the file system, set `MIRIFLAGS=-Zmiri-disable-isolation` to enable those.
+(Miri will tell you when that is necessary.)
+If your test suite runs into an unsupported operation, please [report an issue](https://github.com/rust-lang/miri/issues).
+
+If you want to add Miri to your CI to ensure your test suite keeps working in Miri, please consult our [README](https://github.com/rust-lang/miri/#running-miri-on-ci).
+That document is also a great starting point for any other questions you might have.
+
+Miri is also integrated into the [Rust Playground](https://play.rust-lang.org/): you can select Miri in the "Tools" menu to check the code for Undefined Behavior.
+
+If Miri complains about your code and you do not understand why, I am happy to help!
+The best places to ask probably are Zulip (the #general stream seems fine), and the Miri issue tracker.
+Asking publicly is strongly encouraged so other people can help answer the question, and everyone can learn from the responses.
+Questions are much easier to answer if you manage to reproduce the problem in a small self-contained bit of example code (ideally on the playground), but feel free to ask even if you do not know how to reduce the problem.
+
+## Helping Miri
+
+If you want to help improve Miri, that's awesome!
+The [issue tracker](https://github.com/rust-lang/miri/issues) is a good place to start; the list of issues is short enough that you can just browse through it rather quickly to see if anything pikes your interest.
+Another good starting point is to try to implement the missing bit of functionality that keeps your test suite from working.
+If you need any mentoring, just get in touch. :)
+
+That's it for now.
+I am totally blown away by how many people are already using Miri; this endeavor of re-shaping the way we approach correctness of unsafe code has been way more successful than I expected.
+I hope Miri can also help you to ensure correctness of your unsafe code, and I am excited for what the next year of Miri development will bring. :D
--- /dev/null
+---
+title: "Pointers Are Complicated II, or: We need better language specs"
+categories: rust research programming
+forum: https://internals.rust-lang.org/t/pointers-are-complicated-ii-or-we-need-better-language-specs/13562
+license: CC BY-SA 4.0
+license-url: https://creativecommons.org/licenses/by-sa/4.0/
+---
+
+Some time ago, I wrote a blog post about how [there's more to a pointer than meets the eye]({% post_url 2018-07-24-pointers-and-bytes %}).
+One key point I was trying to make is that
+
+> *just because two pointers point to the same address, does not mean they are equal in the sense that they can be used interchangeably.*
+
+This "extra information" that distinguishes different pointers to the same address is typically called [*provenance*](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#pointer-provenance).
+This post is another attempt to convince you that provenance is "real", by telling a cautionary tale of what can go wrong when provenance is not considered sufficiently carefully in an optimizing compiler.
+The post is self-contained; I am not assuming that you have read the first one.
+There is also a larger message here about how we could prevent such issues from coming up in the future by spending more effort on the specification of compiler IRs.
+
+<!-- MORE -->
+
+Below, I will show a series of three compiler transformations that each seem "intuitively justified", but when taken together they lead to a clearly incorrect result.
+I will use LLVM for these examples, but the goal is not to pick on LLVM---other compilers suffer from similar issues.
+The goal is to convince you that to build a correct compiler for languages permitting unsafe pointer manipulation such as C, C++, or Rust,
+we need to take IR semantics (and specifically provenance) more seriously.
+I use LLVM for the examples because it is particularly easy to study with its single, extensively-documented IR that a lot of infrastructure evolved around.
+Let's get started!
+
+## Warm-up: Why IRs need a precise semantics
+
+As a warm-up, I will give a simple example showing that compiler IRs such as LLVM IR need a precise (and precisely documented) semantics.
+If you are already familiar with the idea of treating compiler IRs as proper programming languages in their own right, or if you are just here for the pointers and their provenance, you can skip to the next section.
+
+Consider the following simple (and contrived, for the sake of this example) piece of C code computing `n * (i+j)`:
+{% highlight c %}
+int sum_up(int i, int j, unsigned int n) {
+ int result = 0;
+ while (n > 0) {
+ result += i + j;
+ n -= 1;
+ }
+ return result;
+}
+{% endhighlight %}
+One transformation the compiler might want to do is to move the addition `i+j` out of the loop, to avoid computing the sum each time around the loop (this is called "loop-invariant code motion"[^loop]):
+{% highlight c %}
+int sum_up(int i, int j, unsigned int n) { // optimized version
+ int result = 0;
+ int s = i + j;
+ while (n > 0) {
+ result += s;
+ n -= 1;
+ }
+ return result;
+}
+{% endhighlight %}
+However, that transformation is actually incorrect.
+If we imagine a caller using this function as `sum_up(INT_MAX, 1, 0)`, then this is a perfectly correct way to call `sum_up`: the loop is never entered, so the overflowing addition `INT_MAX+1` is never performed.
+However, after the desired optimization, the program now causes a signed integer overflow, which is UB (Undefined Behavior) and thus May Never Happen![^signed-int-overflow]
+
+[^loop]: If you are a regular reader of my blog, you will recognize this as the same optimization that already played a crucial role in [a previous post of mine]({% post_url 2020-07-15-unused-data %}). Loop-invariant code motion is a great optimization to look at when considering corner cases of IR semantics.
+
+[^signed-int-overflow]: If you are coming here with a Rust mindset, imagine `i + j` was written as `i.unchecked_add(j)`.
+
+One might be tempted to ignore this problem because the UB on integer overflow is a compiler-only concept; every target supported by the compiler will do the obvious thing and just produce an overflowing result.
+However, there might be other compiler passes running after the optimization we are considering.
+One such pass might inline `sum_up`, and another pass might notice the `INT_MAX+1` and replace it by `unreachable` since UB code is "by definition" unreachable, and another pass might then just remove all our code since it is unreachable.
+Each of these passes has a good reason to exist (it can help real code become a lot faster or help prune dead code), but if we combine them all with loop-invariant code motion, the result is a disaster.
+
+One way (and the only systematic way I know) to avoid such problems is to make sure that we can justify the correctness of each optimization *in isolation*.
+Each optimization must be *correct* for any possible program, where *correct* means that the optimized program must only "do things" that the original program could have done as well.
+(This is basically the "as-if" rule in the C standard, and is typically called "refinement" in the academic literature.)
+In particular, no optimization must ever introduce UB into a UB-free program.
+
+It may seem now that under this premise, it is impossible to perform the loop-invariant code motion we are considering.
+But that is not the case!
+So far, what we have seen is that the optimization is not *correct* when being performed on a C program.
+But when LLVM performs these optimizations, it does not consider the program to be written in C---it considers the program to be written in LLVM IR, which has a different semantics than C.
+Specifically, the [LLVM LangRef](https://llvm.org/docs/LangRef.html) says that signed integer overflow in LLVM IR yields a `poison` value.
+It is not UB to produce `poison`, it is just UB to use `poison` in certain ways (the details of this do not matter here).
+In a call to the optimized `sum_up(INT_MAX, 1, 0)`, the `s` variable introduced by loop-invariant code motion is unused, so the fact that its value is `poison` does not matter!
+
+Due to this behavior of signed integer overflow, this case of loop-invariant code motion is *correct* if we consider it as an optimization on programs that are written in LLVM IR.[^cheat]
+The "price" we pay for this is that replacing `INT_MAX+1` by `unreachable` is not *correct* in LLVM IR, since it is not UB.
+
+The great thing about *correct* optimizations is that we can combine any number of them in any order (such as inlining, replacing definite UB by `unreachable`, and removing unreachable code), and we can be sure that the result obtained after all these optimizations is a correct compilation of our original program.
+(In the academic lingo, we would say that "refinement is transitive".)
+
+However, to make the argument that an optimization is *correct*, the exact semantics of LLVM IR (what the behavior of all possible programs is and when they have UB) needs to be documented.
+All involved optimizations need to exactly agree on what is and is not UB, to ensure that whatever code they produce will not be considered UB by a later optimization.
+This is exactly what we also expect from the specification of a programming language such as C, which is why I think we should consider compiler IRs as proper programming languages in their own right, and specify them with the same diligence as we would specify "normal" languages.[^ub-difference]
+Sure, no human is going to write many programs in LLVM IR, so their syntax barely matters, but clang and rustc produce LLVM IR programs all the time, and as we have seen understanding the exact rules governing the behavior of programs is crucial to ensuring that the optimizations LLVM performs do not change program behavior.
+
+[^cheat]: If now you feel like we somehow cheated, since we can always translate the program from C to LLVM IR, optimize there, and translate back, consider this: translating from LLVM IR to C is really hard! In particular, signed integer addition in LLVM IR can *not* be translated into signed integer addition in C, since the former is well-defined with `poison` result in case of overflow, but the latter says overflow is UB. C has strictly more UB than LLVM IR (for integer arithmetic), which makes translation in one direction easy, while the other direction is hard.
+
+[^ub-difference]: In particular, two different variants of the IR with different rules for UB are really *two different programming languages*. A program that is well-defined in one language may have UB in another, so great care needs to be taken when the program is moved from being governed by one set of rules to another.
+
+*Take-away:* If we want to be able to justify the correctness of a compiler in a modular way, considering only one optimization at a time, we need to perform these optimizations in an IR that has a precise specification of all aspects of program behavior, including UB.
+Then we can, for each optimization separately, consider the question: does the optimization ever change program behavior, and does it ever introduce UB into UB-free programs?
+For a *correct* optimization, the answer to these questions is "no".
+
+## How 3 (seemingly) correct optimizations can be incorrect when used together
+
+With the warm-up done, we are now ready to consider some more tricky optimizations, which we will use to explore the question of how precise a language specification needs to be.
+We will look at three different optimizations LLVM can perform, and I will show that they *cannot all be correct* since the first and last program we are considering actually have *different behavior*.
+(More precisely: the last program has a possible behavior that was not possible for the first program.)
+This is only possible if at least one optimization changed program behavior in an incorrect way, but it is actually not entirely clear which optimization is the culprit.
+
+The sequence of examples is taken from [this talk](https://sf.snu.ac.kr/llvmtwin/files/presentation.pdf#page=32) by Chung-Kil Hur; it was discovered while working on a mathematically rigorous specification of LLVM.
+
+Here is the source program:
+{% highlight c %}
+char p[1], q[1] = {0};
+uintptr_t ip = (uintptr_t)(p+1);
+uintptr_t iq = (uintptr_t)q;
+if (iq == ip) {
+ *(char*)iq = 10;
+ print(q[0]);
+}
+{% endhighlight %}
+I am using C syntax here just as a convenient way to write programs in LLVM IR.
+
+This program has two possible behaviors: either `ip` (the address one-past-the-end of `p`) and `iq` (the address of `q`) are different, and nothing is printed.
+Or the two are equal, in which case the program will print "10" (`iq` is the result of casting `q` to an integer, so casting it back will yield the original pointer, or at least a pointer pointing to the same object / location in memory).
+
+The first "optimization" we will perform is to exploit that if we enter the `if` body, we have `iq == ip`, so we can replace all `iq` by `ip`.
+Subsequently the definition of `ip` is inlined:
+{% highlight c %}
+char p[1], q[1] = {0};
+uintptr_t ip = (uintptr_t)(p+1);
+uintptr_t iq = (uintptr_t)q;
+if (iq == ip) {
+ *(char*)(uintptr_t)(p+1) = 10; // <- This line changed
+ print(q[0]);
+}
+{% endhighlight %}
+
+The second optimization notices that we are taking a pointer `p+1`, casting it to an integer, and casting it back, so we can remove the cast roundtrip:
+{% highlight c %}
+char p[1], q[1] = {0};
+uintptr_t ip = (uintptr_t)(p+1);
+uintptr_t iq = (uintptr_t)q;
+if (iq == ip) {
+ *(p+1) = 10; // <- This line changed
+ print(q[0]);
+}
+{% endhighlight %}
+
+The final optimization notices that `q` is never written to, so we can replace `q[0]` by its initial value `0`:
+{% highlight c %}
+char p[1], q[1] = {0};
+uintptr_t ip = (uintptr_t)(p+1);
+uintptr_t iq = (uintptr_t)q;
+if (iq == ip) {
+ *(p+1) = 10;
+ print(0); // <- This line changed
+}
+{% endhighlight %}
+
+However, this final program is different from the first one!
+Specifically, the final program will either print nothing or print "0", while the original program *could never print "0"*.
+This shows that the sequence of three optimizations we performed, as a whole, is *not correct*.
+
+#### What went wrong?
+
+Clearly, one of the three optimizations is incorrect in the sense that it introduced a change in program behavior.
+But which one is it?
+
+In an ideal world, we would have a sufficiently precise semantics for LLVM IR that we would just have to read the docs (or, even better, run some Miri-like tool) to figure out the answer.
+However, describing language semantics at this level of precision is *hard*, and full of trade-offs.
+The LLVM LangRef will not give us a clear answer here, and indeed obtaining a clear answer requires some decisions that have not been explicitly made yet.
+
+To proceed, we will use the three optimizations that we considered above as cues: assuming that the optimization is correct for LLVM IR, what does that tell us about the semantics?
+
+Let us start with the last optimization, where the `print` argument is changed from `q[0]` to `0`.
+This optimization is based on alias analysis:
+`q[0]` gets initialized to `0` at the beginning of the program, and the only write between that initialization and the `print` is to the pointer `p+1`.
+Since `q` and `p` point to different local variables, a pointer derived from `p` cannot alias `q[0]`, and hence we know that this write cannot affect the value stored at `q[0]`.
+
+Looking more closely, however, reveals that things are not quite so simple!
+`p+1` is a one-past-the-end pointer, so it actually *can* have the same address as `q[0]`
+(and, in fact, inside the conditional we know this to be the case).
+However, LLVM IR (just like C) does not permit memory accesses through one-past-the-end pointers.
+It makes a difference whether we use `p+1` or `q` inside the `if`, even though we know (in that branch) that both pointers point to the same memory location.
+This demonstrates that in LLVM IR, there is more to a pointer than just the address it points to---it also matters how this address was computed.
+This extra information is typically called *provenance*.
+It is impossible to argue for the *correct*ness of the third optimization without acknowledging that provenance is a real part of the semantics of an LLVM IR program.
+In a flat memory model where pointers are just integers (such as most assembly languages), this optimization is simply wrong.
+
+Now that we know that provenance exists in pointers, we have to also consider what happens to provenance when a pointer gets cast to an integer and back.
+The second optimization gives us a clue into this aspect of LLVM IR semantics: casting a pointer to an integer and back is optimized away, which means that *integers have provenance*.
+To see why, consider the two expressions `(char*)(uintptr_t)(p+1)` and `(char*)(uintptr_t)q`:
+if the optimization of removing pointer-integer-pointer roundtrips is correct, the first operation will output `p+1` and the second will output `q`, which we just established are two different pointers (they differ in their provenance).
+The only way to explain this is to say that the input to the `(char*)` cast is different, since the program state is otherwise identical in both cases.
+But we know that the integer values computed by `(uintptr_t)(p+1)` and `(uintptr_t)q` (i.e., the bit pattern as stored in some CPU register) are the same, and hence a difference can only arise if these integers consist of more than just this bit pattern---just like pointers, integers have provenance.
+
+Finally, let us consider the first optimization.
+Here, a successful equality test `iq == ip` prompts the optimizer to replace one value by the other.
+This optimization demonstrates that *integers do not have provenance*:
+the optimization is only correct if a successful run-time equality test implies that the two values are equivalent in the "Abstract Machine" that is used to describe the language semantics.
+But this means that the Abstract Machine version of this value cannot have any "funny" extra parts that are not represented at run-time.
+Of course, provenance is exactly such a "funny" extra part.
+A different way to phrase the same argument is to say that this optimization is correct only if `iq == ip` evaluating to `true` implies that both values have the same "Abstract Machine" representation, so if that representation involves provenance, both values must have the same provenance.
+This would be a possible definition of `==` in LLVM IR, but only in principle---in practice this means the LLVM backends have to compile `==` in a way that pointer provenance is taken into account, which of course is impossible.
+
+*Take-away:*
+By considering each of these three optimizations in terms of what they tell us about the semantics of LLVM IR, we learned that pointers have provenance, that integers remember the provenance of the pointer they come from in case of a pointer-to-integer cast, and that integers do not have provenance.
+This is a contradiction, and this contradiction explains why we saw incorrect compilation results when applying all three optimizations to the same program.
+
+#### How can we fix this?
+
+To fix the problem, we will have to declare one of the three optimizations incorrect and stop performing it.
+Speaking in terms of the LLVM IR semantics, this corresponds to deciding whether pointers and/or integers have provenance:
+* We could say both pointers and integers have provenance, which invalidates the first optimization.
+* We could say pointers have provenance but integers do not, which invalidates the second optimization.
+* We could say nothing has provenance, which invalidates the third optimization.
+
+In my opinion, the first and last options are not tenable.
+Removing provenance altogether kills all but the most simple alias analyses.[^alias]
+On the other hand, declaring that integers have provenance does not just disable the first optimization in the chain shown above, it also disables common arithmetic optimizations such as `x - x` being equivalent to `0`.
+Even achieving commutativity and associativity of `+` becomes non-trivial once integers have provenance.
+
+[^alias]: Sadly, I do not know of a systematic study of the performance impact of going with the third option: which part of the alias analysis is still correct, and how much slower is a compiler that only performs such a restricted form of alias analysis? It is my understanding that many compiler developers "obviously" consider this way too costly of a fix to the problem, but it would still be great to underpin this with some proper data. In any case, the third option would entail a complete re-design of LLVM's `noalias`, which would be bad news both for Rust (which uses `noalias` to encode alias information embedded in Rust's reference types) and clang (which uses `noalias` to encode C's `restrict`).
+
+So, I think that the issue should be resolved by saying that pointers have provenance but integers do not, which means that it is the second optimization that is wrong.
+This also corresponds to [what has been recently proposed to the C standard committee](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2577.pdf).
+That's why [LLVM bug #34548](https://bugs.llvm.org/show_bug.cgi?id=34548) says that optimizing away pointer-integer-pointer roundtrips is incorrect, and LLVM should stop doing this in the general case.
+There might still be special cases where this can be done, but figuring out the limits of this requires a more precise description of LLVM IR semantics such as what we proposed [in this paper](https://people.mpi-sws.org/~jung/twinsem/twinsem.pdf).
+
+But ultimately, it will be up to the LLVM community to make this decision.
+All I can say for sure is that they have to make a choice, because the status quo of doing all three of these optimizations leads to incorrect compilation results.
+
+## Conclusion
+
+What did we learn?
+First of all, pointers are complicated.
+Precisely describing their semantics in a way that is consistent with common alias analyses requires adding a notion of "provenance".
+In a language such as Java or ML where pointers are opaque types whose representation cannot be observed, this is actually fairly easy to do.
+But in a language such as Rust, C, or C++ that supports pointer-integer casts, the introduction of provenance poses some really tricky questions, and at least one of the commonly performed optimizations in this space has to give.
+
+We also learned that LLVM has a bug, but that was *not* the point of this blog post.
+The GCC developers [made exactly the same mistake](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82282), and I got word that MSVC and ICC have the same issue (though I do not know how to verify this).
+And I cannot blame them; the way compiler development typically works, I think bugs like this are inevitable: when exactly UB arises in an IR is often only loosely specified, in many cases "by omission" (where cases not covered in the spec are implicitly UB), so evaluating whether some optimization is *correct* in the sense defined above can be very tricky or even impossible.
+Pointer provenance is just a particularly good (and subtle) example.
+For another example, see [§2.3 of this paper](https://plv.mpi-sws.org/validc/paper.pdf) (Figure 3 contains the code) which shows how a sequence of two optimizations can lead to a miscompilation, where the first optimization is *correct* under the LLVM concurrency model, and the second optimization is *correct* under the C++11 concurrency model---but there is no concurrency model under which *both* optimizations are correct, so each compiler (or rather, each compiler IR) needs to pick one or the other.
+Finally, this [paper on `undef` and `poison`](https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf) gives examples for optimizations that are broken by the presence of `undef` in LLVM, and describes some of the trade-offs that arise when defining the semantics of `poison`.
+Again miscompilations arise because the consequence of a statement in one place of the specification (`undef` picks a fresh value at each use) are not considered elsewhere (testing an integer for equality with zero dos not imply it is zero; it could also be `undef`).
+
+Which brings me to my main conclusion for this post: to avoid the problem of incompatible optimizations, I think we need to take compiler IRs more serious as programming languages in their own right, and give them a precise specification---including all the UB.
+Now, you may object by saying that LLVM has an [extensive LangRef](https://llvm.org/docs/LangRef.html), and still, by reading the LLVM specification one could convince oneself that each of the three optimizations above is correct, which as we have seen is contradictory.
+What is missing? Do we need a formal mathematical definition to avoid such ambiguities?
+I do not think so; I think there is something simpler that will already help a lot.
+The source of the contradiction is that some parts of the specification *implicitly* assume that pointers have provenance, which is easy to forget when considering other operations.
+That's why I think it is very important to make such assumptions more explicit: the specification should *explicitly* describe the information that "makes up" a value, which will include things like provenance and [whether the value is (wholly or partially) initialized]({% post_url 2018-07-24-pointers-and-bytes %}).[^C]
+This information needs to be extensive enough such that a hypothetical interpreter can use it to detect all the UB.
+Doing so makes it obvious that a pointer has provenance, since otherwise it is impossible to correctly check for out-of-bounds pointer arithmetic while also permitting one-past-the-end pointers.
+
+[^C]: The astute reader will notice that this information is also absent in the C and C++ specifications. That is indeed one of my main criticisms of these specifications. It leads to many open questions not only when discussing provenance (including `restrict`, which I did not even go into here) but also when discussing indeterminate and unspecified values or the rules around effective types.
+
+This is my bar for what I consider a sufficiently precise language specification: it needs to contain all the information needed such that writing a UB-checking interpreter is just a matter of "putting in the work", but does not require introducing anything new that is not described in the specification.
+
+The good news is that the process of developing these more precise specifications is already underway!
+
+On the Rust side, this is mainly apparent in the [Miri interpreter](https://github.com/rust-lang/miri/), which is a concrete realization of the "hypothetical" interpreter that I mentioned above.
+In fact, this is why I originally started working on Miri.
+Nowadays, its main purpose is to help unsafe code authors avoid UB, but for me personally, I find it equally important that it helps us think about the semantics of Rust and MIR in a different way.
+It also feeds back into the design of the UB rules by discovering patterns that people want or need to use but that are not currently accepted by Miri.
+
+On the LLVM side, the main development in this area is [Alive](https://blog.regehr.org/archives/1722), a tool that can automatically validate[^validate] optimizations performed by LLVM.
+Alive has found [many bugs in LLVM optimizations](https://github.com/AliveToolkit/alive2/blob/master/BugList.md), and indeed much of the recent dialog with the LLVM community aimed at a more precise IR semantics is pushed by the people building Alive, led by Nuno P. Lopes and John Regehr.
+
+[^validate]: A note on terminology: "validating" an optimization means that given the program text before and after the optimization, the tool will try to prove that this particular transformation is correct. This is in contrast to "verifying" an optimization where a once-and-forall proof is carried out showing that the optimization will always perform a correct transformation. Verification gives a much stronger result, but is also extremely difficult to carry out, so validation is a great middle-ground that is still able to find plenty of bugs.
+
+Progress on these specification efforts is slow, though, in particular when it turns out that [LLVM IR semantics should be changed](https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf).
+I hope this post can raise awareness for the subtle problems optimizing compilers are facing, and convince some people that figuring out the specification of compiler IRs is an important and interesting problem to work on. :)
+
+That's all I have for today, thanks for sticking with me!
+As usual, this post can be [discussed in the Rust forums](https://internals.rust-lang.org/t/pointers-are-complicated-ii-or-we-need-better-language-specs/13562) and [on Reddit](https://www.reddit.com/r/rust/comments/kd157i/pointers_are_complicated_ii_or_we_need_better/).
+I am curious what your thoughts are on how we can build compilers that do not suffer from the issues I have discussed here.
+
+#### Footnotes
--- /dev/null
+---
+title: "Safe Systems Programming in Rust"
+categories: research rust
+reddit: /rust/comments/mb08i2/safe_systems_programming_in_rust/
+---
+
+It has been a long time coming; now our Communications of the ACM article [Safe Systems Programming in Rust](https://cacm.acm.org/magazines/2021/4/251364-safe-systems-programming-in-rust/fulltext) has finally been published.
+A [pdf version](https://cacm.acm.org/magazines/2021/4/251364-safe-systems-programming-in-rust/pdf) is also available.
+We explain at a high level what makes Rust so innovative and interesting, and how we are studying Rust formally in the [RustBelt project](https://plv.mpi-sws.org/rustbelt/).
+The ACM even produced a [short video](https://vimeo.com/514402648) which includes Derek and me explaining the main points of the article.
+Have fun. :)
--- /dev/null
+---
+title: "A podcast about GhostCell"
+categories: rust research
+reddit: /rust/comments/nw634j/podcast_build_with_rust_ralf_jung_on_ghostcell
+---
+
+I recently got asked to appear as a guest on the podcast [Building with Rust](https://anchor.fm/building-with-rust) to talk about our recent work on [GhostCell](http://plv.mpi-sws.org/rustbelt/ghostcell/).
+I never was a guest on a podcast before, so this was very exciting and of course I said yes. :)
+That episode has been released now, so you can listen to an hour of me talking about GhostCell and about PL research more generally:
+
+> [Ralf Jung on GhostCell and Working as a PL Researcher](https://anchor.fm/building-with-rust/episodes/Building-with-Rust-Ralf-Jung-on-GhostCell-and-Working-as-a-PL-Researcher-e12auje)
+
+Have fun, and I am sorry for talking so fast. ;)
--- /dev/null
+---
+title: "Undefined Behavior deserves a better reputation"
+categories: rust research
+reddit: /rust/comments/qx168t/undefined_behavior_deserves_a_better_reputation/
+---
+
+*This is a cross-post of an [article that I wrote for the SIGPLAN blog](https://blog.sigplan.org/2021/11/18/undefined-behavior-deserves-a-better-reputation/).*
+
+"Undefined Behavior" often has a bad reputation. People see it as an excuse compiler writers use to break code, or an excuse by a lazy language designer to not complete the specification and properly define all this behavior.
+But what, really, is Undefined Behavior, and is it as bad as its reputation?
+In this blog post, I will look at this topic from a PL perspective, and argue that Undefined Behavior (or UB for short) is a valuable tool in a language designer's toolbox, and that it can be used responsibly to convey more of the programmer's insight about their code to the compiler with the goal of enabling more optimizations.
+I will also explain why I spent a significant amount of time adding *more* UB to Rust.
+
+<!-- MORE -->
+
+## A simple example
+
+In the best PL tradition, let us consider an artificial example to demonstrate the benefit of UB.
+Imagine we want to implement a function that returns the element in the middle of an array.
+If we are using Rust, we would probably write something like this:
+```rust
+fn mid(data: &[i32]) -> Option<i32> {
+ if data.is_empty() { return None; }
+ return Some(data[data.len()/2]);
+}
+```
+The argument is of type `&[i32]`, which is called a "slice" and consists of a pointer to some array and information about how long the array is.
+`mid` itself returns an integer wrapped in `Option` (corresponding to `Maybe` in Haskell) to properly signal the case where the array is empty.
+In the non-empty case, it computes the index in the middle of `data`, and returns that element.
+
+Now imagine this function is called in a tight loop in the benchmark for our next paper, so performance *really* matters.
+Is there any performance improvement we can hope to achieve in this function?
+It might seem like `mid` already does the absolute minimum amount of work required for the task, but there is some hidden cost in the array access `data[_]`:
+the compiler has to insert a bounds-check here to ensure that we do not access data beyond the size of the array that `data` points to.
+But as the programmer we know that bounds-check to be entirely unnecessary, since `data.len()/2` will always be smaller than `data.len()`!
+Wouldn't it be great if there was a way to tell the compiler about this, such that we can be sure no bounds check happens?
+
+Here is one way to accomplish that in Rust:
+```rust
+fn mid(data: &[i32]) -> Option<i32> {
+ if data.is_empty() { return None; }
+ match data.get(data.len()/2) {
+ Some(&x) => return Some(x),
+ None => unsafe { unreachable_unchecked() }
+ }
+}
+```
+We are now using the `get` operation to access the array, which returns an `Option` that is `None` for out-of-bounds accesses.
+And in case we get `None`, we call a special function `unreachable_unchecked` which makes a *binding promise to the compiler* that this piece of code is unreachable.
+The keyword `unsafe` here indicates that what we are doing is not covered by the type safety guarantees of the language: the compiler will not actually check that the promise we made holds true, it will just trust us on that.
+(The phrase "unchecked" is a Rust idiom; this is the "unchecked" version of `unreachable`, which inserts a run-time check that safely aborts the program should this code ever be reached -- or, to be more precise, it triggers a Rust panic.)
+
+After some inlining, the relevant part of this code looks as follows:
+```rust
+ let idx = data.len()/2;
+ if idx < data.len() { // Automatically inserted bounds-check.
+ ... // Access the array at `idx`.
+ } else {
+ unreachable_unchecked()
+ }
+```
+Since we told the compiler that the `else` branch is unreachable, it is easy to optimize away the conditional, so we end up with just a direct access to element `idx` in the array.
+Problem solved!
+(In fact, Rust provides `get_unchecked` as an alternative to `get` where the caller has to promise that the index is in-bounds, so a Rust programmer would just write `data.get_unchecked(data.len()/2)` to implement `mid` efficiently.)
+
+I expect some readers will not be happy with the way I achieved the desired optimization in the initial example, and argue that the compiler should be smart enough to do this automatically.
+I will get back to this point later; for now, just note that the latest stable version of Rust at the time of writing does [not perform this optimization](https://rust.godbolt.org/z/G34Ezzb9c) (as indicated by the call to `panic_bounds_check`).
+
+## Where is the Undefined Behavior?
+
+Hang on, you might say at this point, wasn't this blog post supposed to be about Undefined Behavior?
+That term did not even appear in the discussion of the example!
+Indeed, I was a bit sneaky and used different terminology that I think better captures a constructive way to think about Undefined Behavior.
+In the typical terminology, I would have said that calling the special function `unreachable_unchecked` causes immediate Undefined Behavior.
+Following the definition in the latest C standard (also shared by C++), the standard "imposes no requirements" on programs that exhibit Undefined Behavior.
+The compiler can hence basically replace the `else` branch by whatever code it wants, famously including "to make [demons fly out of your nose](http://www.catb.org/jargon/html/N/nasal-demons.html)", but also including just executing the `then` branch instead.
+
+This line of reasoning leads to the same result, but it paints an unnecessarily antagonistic picture of compiler writers. It makes it sound like compilers use complicated analyses to detect Undefined Behavior. Once they find UB, they have an excuse to emit broken code and hide behind the standard should anyone complain.
+This is not what actually happens.
+As we have seen in our example, the compiler really has no idea if this code has Undefined Behavior or not -- all it does is perform optimizations that are correct *under the extra assumption* that there is no Undefined Behavior.
+
+## UB is a double-edged sword
+
+Another reaction you might have is that `unreachable_unchecked` is not a "typical" example of UB.
+Most people probably associate that term with C or C++, which do not even have `unreachable_unchecked` (though many compilers provide an intrinsic with the same effect, e.g., `__builtin_unreachable` in GCC).
+So it may seem like I picked a strange example.
+Shouldn't I be talking about how, say, signed integer overflow is UB?
+
+This is the right time to admit that I am *not* going to defend all UB in C/C++.
+I think UB as a concept is a great idea, and `unreachable_unchecked` is the "most pure" form of UB that shows how it can be used by the programmer to convey extra information to the compiler -- but I also think that C and C++ are massively overusing UB.
+Of course, it is easy to say this with the benefit of hindsight; the first C compilers were extremely simple and today's use of UB for optimizations only emerged over time.
+It took a while for the implications of the modern interpretation of UB in the standard to become clear -- and C and C++, being very successful languages, have massive existing codebases which makes it super hard to revise any prior decision.
+This post is about defending and promoting UB as a concept, not UB in C/C++.
+
+Speaking of signed integer overflow, I think this is actually a good example for how to *not* use UB.
+An innocent-looking `+` turns into a promise of the programmer that this addition will never overflow, but the programmer probably will not carefully do a mental no-overflow proof for every single addition in their program.
+Instead, `+` could perform overflow checks or well-defined wrap-around, and the language could provide an `unchecked_add` function where overflows are UB.
+This lets the programmer opt in to providing extra no-overflow promises, to be used in situations where it is really beneficial for performance that the compiler can make this assumption (such as [this example](https://youtu.be/yG1OZ69H_-o?t=2357)).
+Basically, I am considering this a language (and library) design problem: UB is a sharp knife; when used well it gets the job done better, but it can also hurt a lot when used without enough care.
+
+Language and library design is not everything that can be used to tame UB, however.
+Good tooling can also make a big difference: if programmers can easily run their programs in "UB-checking mode", they can write tests to at least ensure the absence of UB for certain inputs.
+(Shameless plug: I am working on [Miri](https://github.com/rust-lang/miri/), a tool that provides exactly this for Rust.)
+Library authors can run their test suites with such a tool, and the tool can also be used exploratively to learn about what exactly is and is not UB in the first place.
+I think this is absolutely crucial, and language designers should design UB in a way that makes UB-checking tools more feasible.
+For the examples of UB we have seen so far (`unreachable_unchecked`, `get_unchecked`, and `unchecked_add`), this is obviously trivial.
+
+## How far can we push UB?
+
+That said, not all UB is that simple to teach and test.
+Even Rust, with its benefit of learning from several decades of experience with UB in C and C++, has UB that is a lot more subtle than this. The most glaring example of this is probably UB related to incorrect aliasing of mutable references.
+(Other, less extreme examples would be UB due to using uninitialized memory, or UB due to data races.)
+
+The Rust type system ensures that mutable references never alias any other reference that is currently being used in the program, i.e., they never point to the same memory as any other reference.
+This is a juicy guarantee for compiler writers, because while reordering memory accesses is often beneficial, it can be very hard to figure out if the transformation is even allowed -- if two accesses alias, then their original order must be preserved.
+
+However, `unsafe` code in Rust could easily create aliasing mutable references.
+So what can we do?
+We make the programmer promise that they do not do this!
+This is a lot like saying "the programmer promises that `unreachable_unchecked` is never called", so we can put on our UB lens and say that it is Undefined Behavior to have aliasing mutable references.
+
+The devil is of course in the details of defining what exactly this means.
+[Stacked Borrows](https://plv.mpi-sws.org/rustbelt/stacked-borrows/) (part of [my PhD thesis](https://www.ralfj.de/research/thesis.html) and also described in a series of blog posts: [v1.0](https://www.ralfj.de/blog/2018/11/16/stacked-borrows-implementation.html), [v2.0](https://www.ralfj.de/blog/2019/04/30/stacked-borrows-2.html), [v2.1](https://www.ralfj.de/blog/2019/05/21/stacked-borrows-2.1.html)) goes into all that detail by giving an operational semantics that exactly defines the promises programmers have to make.
+And that semantics is non-trivial!
+According to Stacked Borrows, the following code has UB:
+```rust
+let x = &mut 42; // Safely create a reference.
+let xptr = x as *mut i32; // Turn that reference into a raw (unchecked) pointer.
+let x1 = unsafe { &mut *xptr }; // Turn the pointer back into a reference...
+let x2 = unsafe { &mut *xptr }; // ...twice, so uniqueness is violated.
+*x1 = 0; // Undefined Behavior!
+```
+The reason this code has UB is that creating `x2` makes a promise that this is the unique reference created from `xptr`, so the previously created `x1` is invalidated when `x2` gets created.
+This means future uses of `x1` are Undefined Behavior.
+
+So this raises the question: can we really expect every author of `unsafe` Rust code to internalize Stacked Borrows to the extent that they can faithfully promise to the Rust compiler that their code will comply by this bespoke set of rules?
+Is it a good idea to interpret `&mut expr` as a promise that all aliasing was carefully checked and this reference is definitely unique?
+As with other UB, we can help programmers by providing tools; Miri contains an implementation of Stacked Borrows which both helps us to evaluate whether actual Rust code is compatible (or can reasonably be made compatible) with Stacked Borrows, and it helps Rust programmers by giving them a way to at least test for aliasing violations, and to interactively play with the semantics to gain a better understanding.
+I think that puts us in a pretty good spot overall, but some people still argue that Stacked Borrows goes too far and Rust will end up in a situation similar to the one C and C++ find themselves in -- where too few programmers actually know how to write UB-free code, and a significant amount of the code people rely on exhibits UB.
+
+Stacked Borrows is not part of the Rust spec, and is not the final word for aliasing-related UB in Rust.
+So there is still the chance that future revisions of this model can be made to better align with programmer intuition.
+The above code might get accepted because `x2` is not actually being used to access memory.
+Or maybe `&mut expr` should only make such promises when used outside an `unsafe` block -- but then, should adding `unsafe` really change the semantics of the program?
+As usual, language design is a game of trade-offs.
+
+## Conclusion
+
+I have presented Undefined Behavior as a tool that enables the programmer to write code that the compiler cannot check for correctness, and argued that -- used responsibly -- it is a useful component in a language designer's toolbox.
+
+As I alluded to earlier, the "obvious" alternative would be to make the compiler smarter.
+However, real programs are typically a lot more complicated than my simple example (which already outsmarts Rust's LLVM backend), and the reasoning required to justify an optimization can become arbitrarily complicated.
+Language designers should acknowledge that optimizers have their limitations and give programmers the tools they need to help the optimizer.
+Indeed, I think the fact that Rust combines a clever type checker with the idea of using `unsafe` code for the cases where the type checker is not clever enough is crucial for its success: `unsafe` is not a bug; it is a feature without which Rust would not be able to make systems programming safer in practice.
+It is also worth mentioning that many languages that we all know and love provide comparable "trusted" operations or annotations, e.g., `Obj.magic` in OCaml or the rewrite rules in GHC.
+Rust only differs in how prevalent unsafe code is in the ecosystem (and in emphasizing the importance of [encapsulating such code within safe APIs](https://blog.sigplan.org/2019/10/17/what-type-soundness-theorem-do-you-really-want-to-prove/)).
+
+In closing, I would like to propose that "Undefined Behavior" might need a rebranding.
+The term focuses on the negative case, when really all we ever care about as programmers or compiler authors is that programs do *not* have Undefined Behavior.
+Can we get rid of this double negation?
+Maybe we should talk about "ensuring Well-Defined Behavior" instead of "avoiding Undefined Behavior".
+
+To sum up: most of the time, ensuring Well-Defined Behavior is the responsibility of the type system, but as language designers we should not rule out the idea of sharing that responsibility with the programmer.
+
+*Thanks to Anish Athalye and Adrian Sampson for feedback on earlier drafts of this post.*
--- /dev/null
+---
+title: "Do we really need Undefined Behavior?"
+categories: rust research
+---
+
+I recently published a [blog post on why Undefined Behavior is actually not a bad idea]({% post_url 2021-11-18-ub-good-idea %}).
+Coincidentally, this is just a few weeks after the publication of [this paper by Victor Yodaiken](https://dl.acm.org/doi/pdf/10.1145/3477113.3487274) which basically argues that Undefined Behavior (UB for short) made C unusable for one of its core audiences, OS developers.
+Here I refer to the typical modern interpretation of UB: assumptions the compiler may trust, without bounds on what happens if they are violated.
+The paper makes many good points, but I think the author is throwing out the baby with the bathwater by concluding that we should entirely get rid of this kind of Undefined Behavior.
+The point of this blog post is to argue that we do need UB by showing that even some of the most basic optimizations that all compilers perform require this far-reaching notion of Undefined Behavior.
+
+<!-- MORE -->
+
+To avoid ambiguity, I will refer to the above notion of UB as "unrestricted UB".
+The alternative interpretation of UB promoted by Yodaiken is what one might call "platform-specific UB".
+This requires that even programs with Undefined Behavior should behave in a consistent way: for example, the result of an out-of-bounds write may be 'unpredictable', it may either not actually happen or mutate some data somewhere.
+However, if a write occurs, the program must still behave in a way that is consistent with performing a write to the given address in the target platform.
+(At least, that is my understanding. I hope I am not misrepresenting their position here. The paper does not go into a lot of detail on how the situation could be improved, but it mentions proposals "where compilers map source operations to well-defined instruction sequences, in either a virtual or real machine, from which compiler optimisations may not observably stray".)[^N2769]
+
+[^N2769]: The paper also cites [C committee proposal N2769](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2769.pdf). However, N2769 explicitly says that `a + 1 < a` can still be optimized to `false`, while Yodaiken mentions this as an undesirable optimization. In fact, N2769 says it is okay and of "great value" to "assume the absence of UB". I admit I do not understand the distinction N2769 makes between "assuming the absence of UB" and "making assumptions about the result of UB", but it seems clear that Yodaiken goes even further than N2769 in restricting UB-based optimizations.
+
+## Examples of unrestricted UB
+
+So what is the problem with platform-specific UB?
+First of all, it does not reflect what the major compilers actually do in practice.
+I have seen claims in the past that GCC and LLVM are the only compilers making use of unrestricted UB; this is simply not true.
+Here is an [example of ICC performing such an optimization](https://godbolt.org/z/j18oW6YaE) (based on example code by Yodaiken):
+
+{% highlight c %}
+#include <stdlib.h>
+#include <stdio.h>
+
+int main () {
+ int *i = malloc(sizeof(int));
+ *i = 1;
+ int *j = malloc(sizeof(int));
+ *j = 1;
+ int *k = malloc(sizeof(int));
+ *k = 1;
+
+ int *x = j+(32/4);
+ *x = 40;
+ printf("*i=%d (%p) *j=%d (%p) *k=%d (%p) *x=%d (%p)", *i, i, *j, j, *k, k, *x, x);
+}
+{% endhighlight %}
+
+This program prints the values and addresses of a few pointers.
+The concrete addresses are different on each execution, but the pattern is always the same:
+```
+*i=1 (0x1aef2a0) *j=1 (0x1aef2c0) *k=1 (0x1aef2e0) *x=40 (0x1aef2e0)
+```
+Notice how `k` and `x` point to the same address (`0x1aef2e0` in this particular execution), but seem to contain different values.
+This is impossible under "platform-specific UB": no sequence of target platform operations can lead to a situation where the same address contains two different values.[^N2769-2]
+This example demonstrates that even ICC with `-O1` already requires unrestricted UB.
+(For completeness' sake, [here is a similar example for GCC](https://godbolt.org/z/c8qaWhnEG); at the time of writing, `i` and `x` have the same address but different values.
+And [here is an example for clang/LLVM](https://godbolt.org/z/r8TM7Ga8q), this time it's again `k` and `x` that behave inconsistently.
+godbolt supports MSVC but does not seem to be willing to execute the generated programs, but I have no doubt that similar examples can be found for this compiler.)
+
+[^N2769-2]: I assume N2769 would also not be happy with this outcome of our example program.
+
+What about niche compilers specifically built for reliable software?
+In their paper, Yodaiken claims that the verified C compiler CompCert "does not do any undefined behavior based optimization"
+(with a footnote saying "Except for assuming objects do not overlap in memory"; I am not quite sure what exactly is meant by this).
+This is incorrect.
+First of all, since CompCert has a proof of correctness, we can have a look at its specification to see what exactly it promises to its users---and that specification quite clearly follows the "unrestricted UB" approach, allowing the compiled program to produce arbitrary results if the source program has Undefined Behavior.
+Secondly, while CompCert's optimizer is very limited, it is still powerful enough that we can actually demonstrate inconsistent behavior for UB programs in practice:
+
+{% highlight c %}
+#include <stdio.h>
+
+int y, x;
+
+int f(void)
+{
+ y = 0;
+ *(&x + 1) = 1;
+ return y;
+}
+
+int main()
+{
+ int eq = (&x+1 == &y);
+ if (eq) {
+ printf("%d ", f());
+ printf("%d\n", y);
+ }
+ return 0;
+}
+{% endhighlight %}
+
+(Putting the result of the comparison into a local variable `eq` prevents CompCert from optimizing away the entire conditional.)
+This program, after being compiled with CompCert, prints "0 1".
+Again, this is printing "the same thing" twice, in this case the value stored at `y`, and produces two different results.
+CompCert exploited UB in a way that leads to a situation which should be "impossible" on the underlying machine.
+
+## Platform-specific UB is not an option
+
+Both of these examples highlight a fundamental problem with "platform-specific UB": *any* out-of-bounds write could potentially modify any other variable (at least any variable that has an address in memory).
+This can make even the most basic parts of high-quality code generation, such as register allocation, tricky or impossible: a variable that has its address taken has to be re-loaded from that same address any time an out-of-bounds write might have happened, since that write might just have hit the right address to change this variable's value.
+This applies even if the address has not yet been leaked to the outside world, as the first example shows.
+This is probably why there is hardly any compiler that follows the platform-specific interpretation of UB.
+(I say "hardly any" without knowing a counterexample, but I would not be surprised if some compilers for high-assurance embedded code are so simple that platform-specific UB is sufficient for them. But that is hardly representative for how C is used---and as we have seen with CompCert, even some high-assurance compilers do rely on unrestricted UB.)
+
+I honestly think *trying* to write a highly optimizing compiler based on a different interpretation of UB would be a worthwhile experiment.
+We sorely lack data on how big the performance gain of exploiting UB actually is.
+However, I strongly doubt that the result would even come close to the most widely used compilers today---and programmers that can accept such a big performance hit would probably not use C to begin with.
+Certainly, any proposal for *requiring* compilers to curtail their exploitation of UB must come with evidence that this would even be possible while keeping C a viable language for performance-sensitive code.
+
+To conclude, I fully agree with Yodaiken that C has a problem, and that reliably writing C has become incredibly hard since undefined behavior is so difficult to avoid.
+It is certainly worth reducing the amount of things that can cause UB in C, and developing practical tools to detect more advanced kinds of UB such as strict aliasing violations.
+I also wonder whether strict aliasing can be made more compatible with low-level programming patterns---or whether C should provide alternative means of alias control to programmers, such as `restrict` (not that its specification doesn't have its own set of problems, but an opt-in mechanism like `restrict` seems fundamentally more suited when the goal is to ensure compatibility with existing code).
+
+However, I do not think this problem can be solved with a platform-specific interpretation of UB.
+That would declare all but the most basic C compilers as non-compliant.
+We need to find some middle ground that actually permits compilers to meaningfully optimize the code, while also enabling programmers to actually write standards-compliant programs.
+I am not involved in the work that happens here on the C side, but for Rust, I think we can achieve this through a combination of being diligent about how much UB we really need, using language and API design to make it easier for the programmer to be aware of UB requirements imposed by the code they write, and providing [tools](https://github.com/rust-lang/miri/) that help programmers determine if their code exhibits UB or not.
--- /dev/null
+---
+title: "Pointers Are Complicated III, or: Pointer-integer casts exposed"
+categories: rust research programming
+license: CC BY-SA 4.0
+license-url: https://creativecommons.org/licenses/by-sa/4.0/
+reddit: /rust/comments/u1bbqn/pointers_are_complicated_iii_or_pointerinteger/
+---
+
+In my [previous blog post on pointer provenance]({% post_url 2020-12-14-provenance %}), I have shown that not thinking carefully about pointers can lead to a compiler that is internally inconsistent:
+programs that are intended to be well-behaved get miscompiled by a sequence of optimizations, each of which seems intuitively correct in isolation.
+We thus have to remove or at least restrict at least one of these optimizations.
+In this post I will continue that trend with another example, and then I will lay down my general thoughts on how this relates to the recent [Strict Provenance](https://github.com/rust-lang/rust/issues/95228) proposal, what it could mean for Rust more generally, and compare with C's PNVI-ae-udi.
+We will end on a very hopeful note about what this could all mean for Rust's memory model.
+There's a lot of information packed into this post, so better find a comfortable reading position. :)
+
+<!-- MORE -->
+
+In case you don't know what I mean by "pointer provenance", you can either read that previous blog post or the [Strict Provenance documentation](https://doc.rust-lang.org/nightly/core/ptr/index.html#provenance).
+The gist of it is that a pointer consists not only of the address that it points to in memory, but also of its *provenance*: an extra piece of "shadow state" that is carried along with each pointer and that tracks which memory the pointer has permission to access and when.
+This is required to make sense of restrictions like "use-after-free is Undefined Behavior, even if you checked that there is a new allocation at the same address as the old one".
+Architectures like CHERI make this "shadow state" explicit (pointers are bigger than usual so that they can explicitly track which part of memory they are allowed to access),
+but even when compiling for AMD64 CPUs, compilers act "as if" pointers had such extra state -- it is part of the specification, part of the Abstract Machine, even if it is not part of the target CPU.
+
+## Dead cast elimination considered harmful
+
+The key ingredient that will help us understand the nuances of provenance is `restrict`, a C keyword to promise that a given pointer `x` does not alias any other pointer not derived from `x`.[^restrict]
+This is comparable to the promise that a `&mut T` in Rust is unique.
+However, just like last time, we want to consider the limits that `restrict` combined with integer-pointer casts put on an optimizing compiler -- so the actual programming language that we have to be concerned with is the IR of that compiler.
+Nevertheless I will use the more familiar C syntax to write down this example; you should think of this just being notation for the "obvious" equivalent function in LLVM IR, where `restrict` is expressed via `noalias`.
+Of course, if we learn that the IR has to put some limitations on what code may do, this also applies to the surface language -- so we will be talking about all three (Rust, C, LLVM) quite a bit.
+
+[^restrict]: The exact semantics of `restrict` are subtle and I am not aware of a formal definition. (Sadly, the one in the C standard does not really work, as you can see when you try to apply it to my example.) My understanding is as follows: `restrict` promises that this pointer, and all pointers derived from it, will not be used to perform memory accesses that *conflict* with any access done by pointers outside of that set. A "conflict" arises when two memory accesses overlap and at least one of them is a write. This promise is scoped to the duration of the function call when `restrict` appears in an argument type; I have no good idea for what the scope of the promise is in other situations.
+
+With all that out of the way, consider the following program:
+{% highlight c %}
+#include <stdio.h>
+#include <stdint.h>
+
+static int uwu(int *restrict x, int *restrict y) {
+ *x = 0;
+
+ uintptr_t xaddr = (uintptr_t)x;
+ int *y2 = y-1;
+ uintptr_t y2addr = (uintptr_t)y2;
+ if (xaddr == y2addr) {
+ int *ptr = (int*)xaddr;
+ *ptr = 1;
+ }
+
+ return *x;
+}
+
+int main() {
+ int i[2] = {0, 0};
+ int res = uwu(&i[0], &i[1]);
+ // Always prints 1.
+ printf("%d\n", res);
+}
+{% endhighlight %}
+This function takes as argument two `restrict` pointers `x` and `y`. We first write `0` into `*x`.
+Then we compute `y2` as pointing to the `int` right before `*y`, and cast that and `x` to integers.
+If the addresses we get are the same, we cast `xaddr` back to a pointer and write `1` to it.
+Finally, we return the value stored in `*x`.
+
+The `main` function simply calls `uwu` with two pointers pointing to the first two elements of an array.
+Note, in particular, that this *will* make `xaddr` and `y2addr` always equal!
+`&i[1] - 1` denotes the same address as `&i[0]`.
+
+Now, let us imagine we run a few seemingly obvious optimizations on `uwu`:
+- Inside the `if`, we can replace `xaddr` by `y2addr` since they are both equal integers.
+- Since this is a `static` function and the only caller makes `y2addr` always equal to `xaddr`, we know that the conditional in the `if` will always evaluate to `true`. We thus remove the test. (Alternatively, the same transformation can happen by inlining `uwu` into `main` while preserving the alias information, which [LLVM explicitly aims for](https://lists.llvm.org/pipermail/llvm-dev/2019-March/131127.html).)
+- Finally, we observe that `xaddr` is unused, so we can remove it entirely.
+
+`uwu` now looks as follows:
+{% highlight c %}
+static int uwu(int *restrict x, int *restrict y) {
+ *x = 0;
+
+ int *y2 = y-1;
+ uintptr_t y2addr = (uintptr_t)y2;
+ int *ptr = (int*)y2addr; // <-- using y2addr
+ *ptr = 1;
+
+ return *x;
+}
+{% endhighlight %}
+
+This might still look harmless.
+However, we can do even more!
+Notice how this function now consists of a store of `0` to `*x`, then a bunch of code *that does not involve `x` at all*, and then a load from `*x`.
+Since `x` is a `restrict` pointer, this "code that does not involve `x`" cannot possibly mutate `*x`, as that would be a violation of the `restrict`/`noalias` guarantee.
+Hence we can optimize the `return *x` to `return 0`.
+This kind of optimization is the primary reason to have `restrict` annotations in the first place, so this should be uncontroversial.
+Formally speaking: only pointers "derived from" `x` may access `*x`, and while the details of defining "derived from" are nasty, it should be clear that doing a bunch of operations that literally don't involve `x` at all cannot by any stretch of the imagination produce a result that is "derived from" `x`.
+(If they could, `restrict` would be basically worthless.)
+
+Now, the whole program looks like this:
+{% highlight c %}
+static int uwu(int *restrict x, int *restrict y) {
+ *x = 0;
+
+ int *y2 = y-1;
+ uintptr_t y2addr = (uintptr_t)y2;
+ int *ptr = (int*)y2addr;
+ *ptr = 1;
+
+ return 0; // <-- hard-coded return value
+}
+
+int main() {
+ int i[2] = {0, 0};
+ int res = uwu(&i[0], &i[1]);
+ // Now this prints 0!
+ printf("%d\n", res);
+}
+{% endhighlight %}
+We started out with a program that always prints `1`, and ended up with a program that always prints `0`.
+This is bad news. Our optimizations changed program behavior. That must not happen! What went wrong?
+
+Fundamentally, this is the same situation as in the previous blog post: this example demonstrates that either the original program already had Undefined Behavior, or (at least) one of the optimizations is wrong.
+However, the only possibly suspicious part of the original program is a pointer-integer-pointer round-trip -- and if casting integers to pointers is allowed, *surely* that must work.
+I will, for the rest of this post, assume that replacing `x` by `(int*)(uintptr_t)x` is always allowed.
+So, which of the optimizations is the wrong one?
+
+## The blame game
+
+Remember what I said earlier about `restrict` and how it matters which pointer `ptr` is "derived from"?
+If we follow this lead, it may seem like the bogus optimization is the one that replaced `xaddr` by `y2addr`.
+After this transformation, `ptr` is obviously "derived from" `y2` (and thus transitively from `y`) and not `x`, and so obviously `uwu` (as called from `main`) is wrong since we are doing two memory accesses (at least one of which is a write) to the same location, using two pointers that are "derived from" different `restrict` pointers!
+
+However, that optimization doesn't even have anything to do with pointers.
+It just replaces one equal integer by another!
+How can that possibly be incorrect?
+
+What this example shows is that the notion of one value being "derived from" another is not very meaningful when considering an optimizing compiler.[^consume]
+It *is* possible to "fix" this problem and have a notion of "derived from" that works correctly even with pointer-integer round-trips.
+However, this requires saying that not only pointers but also *integers carry provenance*, such that casting a pointer to an integer can preserve the provenance.
+We solved one problem and created many new ones.
+For once, we have to stop doing optimizations that replace one `==`-equal integer by another, unless we know they carry no provenance.
+(Alternatively we could say `==`-comparing such integers is Undefined Behavior. But clearly we want to allow people to `==`-compare integers they obtained from pointer-integer casts, so this is not an option.)
+That seems like a bad deal, since the code that benefits from such optimizations doesn't even do anything shady -- it is the pointer-manipulating code that is causing trouble.
+The list doesn't end here though, and because of that, this option was discarded by the C standardization process during its provenance work, and they ended up picking a "PNVI" model -- provenance *not* via integers.
+I think Rust should follow suit.
+
+[^consume]: This is, in fact, a common problem -- it is what makes the `consume` memory order for atomic accesses basically impossible to specify in a programming language! While instruction sets often have very explicit rules about which instructions are assumed to "depend" on which previous instructions, that notion is hard to rationalize in a language where the compiler can replace `a + (b-a)` by `b` -- and thus *remove* dependencies from the program.
+
+But, if it's not the replacement of `xaddr` by `y2addr` that is wrong, then which optimization *is* the wrong one?
+I will argue that the incorrect optimization is the one that removed `xaddr`.
+More specifically, the bad step was removing the cast `(uintptr_t)x`, irrespective of whether the result of that cast is used or not.
+Had this cast been preserved, it would have been a marker for the compiler to know that "the `restrict` guarantee of `x` ends here", and it would not have done the final optimization of making `uwu` always return `0`.
+
+## Casts have a side-effect
+
+How can it *not* be correct to remove an operation if its result is unused?
+If we take a step back, then in general, the answer is simple -- if calling `foo()` has some side-effect on the global state, like changing the value of a global variable, then of course we have to keep the call to `foo` around even if we ignore its return value.
+But in this case, the operation in question is `(uintptr_t)x`, which has no side-effect -- right?
+
+Wrong.
+This is exactly the key lesson that this example teaches us: casting a pointer to an integer *has a side-effect*, and that side-effect has to be preserved even if we don't care about the result of the cast (in this case, the reason we don't care is that we *already know* that `x` and `y2` will cast to the same `uintptr_t`).
+
+To explain what that side-effect is, we have to get deep into the pointer provenance mindset.
+`x` and `y` are both pointers, so they carry provenance that tracks which memory they have permission to access.
+Specifically, `x` has permission to access `i[0]` (declared in `main`), and `y` has permission to access `i[1]`.[^dyn]
+`y2` just inherits the permission from `y`.
+
+[^dyn]: As mentioned in a previous footnote, this is not actually how `restrict` works. The exact set of locations these pointers can access is determined *dynamically*, and the only constraint is that they cannot be used to access *the same location* (except if both are just doing a load). However, I carefully picked this example so that these subtleties should not change anything.
+
+But which permission does `ptr` get?
+Since integers do not carry provenance, the details of this permission information are lost during a pointer-integer cast, and have to somehow be 'restored' at the integer-pointer cast.
+And that is exactly the point where our problems begin.
+In the original program, we argued that doing a pointer-integer-pointer round-trip is allowed (as is the intention of the C standard).
+It follows that `ptr` must pick up the permission from `x` (or else the write to `*ptr` would be Undefined Behavior: `x` is `restrict`, nothing else can access that memory).
+However, in the final program, `x` plays literally no role in computing `ptr`!
+It would be a disaster to say that `ptr` could pick up the permission of `x` -- just imagine all that `y`-manipulating code is moved into a different function.
+Do we have to assume that any function we call can just do a cast to "steal" `x`'s permission?
+That would entirely defeat the point of `restrict` and make `noalias` optimizations basically impossible.
+
+But how can it be okay for `ptr` to pick up `x`'s permission in the original program, and *not* okay for it to pick up the same permission in the final program?
+The key difference is that in the original program, `x` *has been cast to an integer*.
+When you cast a pointer to an integer, you are basically declaring that its permission is "up for grabs", and any future integer-pointer cast may end up endowing the resulting pointer with this permission.
+We say that the permission has been "exposed".
+And *that* is the side-effect that `(uintptr_t)x` has!
+
+Yes, this way of resolving the conflict *does* mean we will lose some optimizations.
+We *have to* lose some optimization, as the example shows.
+However, the crucial difference to the previous section is that *only code which casts pointers to integers is affected*.
+This means we can keep the performance cost localized to code that does 'tricky things' around pointers -- that code needs the compiler to be a bit conservative, but all the other code can be optimized without regard for the subtleties of pointer-integer-pointer round-trips.
+(Specifically, *both* pointer-integer and integer-pointer casts have to be treated as impure operations, but for different reasons.
+Pointer-integer casts have a side-effect as we have seen.
+Integer-pointer casts are *non-deterministic* -- they can produce different results even for identical inputs.
+I moved the discussion of this point into the appendix below.)
+
+## Strict provenance: pointer-integer casts *without* side-effects
+
+This may sound like bad news for low-level coding tricks like pointer tagging (storing a flag in the lowest bit of a pointer).
+Do we have to optimize this code less just because of corner cases like the above?
+As it turns out, no we don't -- there are some situations where it is perfectly fine to do a pointer-integer cast *without* having the "exposure" side-effect.
+Specifically, this is the case if we never intend to cast the integer back to a pointer!
+That might seem like a niche case, but it turns out that most of the time, we can avoid 'bare' integer-pointer casts, and instead use an operation like [`with_addr`](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.with_addr) that explicitly specifies which provenance to use for the newly created pointer.[^with_addr]
+This is more than enough for low-level pointer shenanigans like pointer tagging, as [Gankra demonstrated](https://gankra.github.io/blah/tower-of-weakenings/#strict-provenance-no-more-getting-lucky).
+Rust's [Strict Provenance experiment](https://doc.rust-lang.org/nightly/std/ptr/index.html#strict-provenance) aims to determine whether we can use operations like `with_addr` to replace basically all integer-pointer casts.
+
+[^with_addr]: `with_addr` has been unstably added to the Rust standard library very recently. Such an operation has been floating around in various discussions in the Rust community for quite a while, and it has even made it into [an academic paper](https://iris-project.org/pdfs/2022-popl-vip.pdf) under the name of `copy_alloc_id`. Who knows, maybe one day it will find its way into the C standard as well. :)
+
+As part of Strict Provenance, Rust now has a second way of casting pointers to integers, `ptr.addr()`, which does *not* "expose" the permission of the underlying pointer, and hence can be treated like a pure operation![^experiment]
+We can do shenanigans on the integer representation of a pointer *and* have all these juicy optimizations, as long as we don't expect bare integer-pointer casts to work.
+As a bonus, this also makes Rust work nicely on CHERI *without* a 128bit wide `usize`, and it helps Miri, too.
+
+[^experiment]: My lawyers advised me to say that all of this is provisional and the specification for `addr` and all other Strict Provenance operations might change until their eventual stabilization.
+
+But that is not the focus of this blog post, Gankra has [already written most of what there is to say here](https://gankra.github.io/blah/tower-of-weakenings/).
+For this blog post, we are happy with what we learned about casts between pointers and integers.
+We have found a way to resolve the conflict uncovered by the example, while keeping performance cost (due to lost optimizations) confined to just the code that is truly ambiguous, and even found alternative APIs that can be used to replace most (all?) uses of ambiguous integer-pointer casts.
+All is well that ends well?
+Unfortunately, no -- we are not quite done yet with pointer provenance nightmares.
+
+## Let's do some transmutation magic
+
+Languages like C or Rust typically allow programmers to re-interpret the underlying representation of a value at a different type.
+In Rust, this is often called "transmutation"; in C, a common term for this is "type punning".
+The easiest way to do this in Rust is via the [`mem::transmute`](https://doc.rust-lang.org/std/mem/fn.transmute.html) function, but alternatively transmutation is possible via `union`s or by casting a `*mut T` raw pointer to `*mut U`.
+In C, the easiest way is to use a `memcpy` between variables of different types, but `union`-based type punning is also sometimes allowed, as is loading data of arbitrary type using a character-typed pointer.
+(Other kinds of pointer-based type punning are forbidden by C's strict aliasing rules, but Rust has no such restriction.)
+The next question we are going to treat in this blog post is: what happens when we transmute a pointer to an integer?
+
+Basically, imagine the original example after we replace the two casts (computing `xaddr` and `y2addr`) with a call to a function like
+{% highlight c %}
+static uintptr_t transmute_memcpy(int *ptr) {
+ uintptr_t res;
+ memcpy(&res, &ptr, sizeof(uintptr_t));
+ return res;
+}
+{% endhighlight %}
+or
+{% highlight c %}
+static uintptr_t transmute_union(int *ptr) {
+ typedef union { uintptr_t res; int *ptr; } Transmute;
+ Transmute t;
+ t.ptr = ptr;
+ return t.res;
+}
+{% endhighlight %}
+All the same optimizations still apply -- right?
+This requires a compiler that can "see through" `memcpy` or union field accesses, but that does not seem too much to ask.
+But now we have the same contradiction as before!
+Either the original program already has Undefined Behavior, or one of the optimizations is incorrect.
+
+Previously, we resolved this conundrum by saying that removing the "dead cast" `(uintptr_t)x` whose result is unused was incorrect, because that cast had the side-effect of "exposing" the permission of `x` to be picked up by future integer-pointer casts.
+We could apply the same solution again, but this time, we would have to say that a `union` access (at integer type) or a `memcpy` (to an integer) can have an "expose" side-effect and hence cannot be entirely removed even if its result is unused.
+And that sounds quite bad!
+`(uintptr_t)x` only happens in code that does tricky things with pointers, so urging the compiler to be careful and optimize a bit less seems like a good idea (and at least in Rust, `x.addr()` even provides a way to opt-out of this side-effect).
+However, `union` and `memcpy` are all over the place.
+Do we now have to treat *all* of them as having side-effects?
+In Rust, due to the lack of a strict aliasing restriction (or in C with `-fno-strict-aliasing`), things get even worse, since literally *any* load of an integer from a raw pointer might be doing a pointer-integer transmutation and thus have the "expose" side-effect!
+
+To me, and speaking from a Rust perspective, that sounds like bad idea.
+Sure, we want to make it as easy as possible to write low-level code in Rust, and that code sometimes has to do unspeakable things with pointers.
+But we *don't* like the *entire ecosystem* to carry the cost of that decision by making it harder to remove every raw pointer load everywhere!
+So what are the alternatives?
+
+Well, I would argue that the alternative is to treat the original program (after translation to Rust) as having Undefined Behavior.
+There are, to my knowledge, generally two reasons why people might want to transmute a pointer to an integer:
+- Chaining many `as` casts is annoying, so calling `mem::transmute` might be shorter.
+- The code doesn't actually care about the *integer* per se, it just needs *some way* to hold arbitrary data in a container of a given type.
+
+The first kind of code should just use `as` casts, and we should do what we can (via lints, for example) to identify such code and get it to use casts instead.[^compat]
+Maybe we can adjust the cast rules to remove the need for chaining, or add some [helper methods](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.expose_addr) that can be used instead.
+
+[^compat]: We could even, if we are really desperate, decide to special-case `mem::transmute::<*const T, usize>` (and likewise for `*mut T`) and declare that it *does* have the "expose" side-effect if the current crate is using some old edition. Sometimes, you have to do ugly things to move forwards. This would not apply to `union`- or raw-pointer-based transmutation.
+
+The second kind of code should not use integers!
+Putting arbitrary data into an integer type is already somewhat suspicious due to the trouble around padding (if we want to make use of those shiny new `noundef` annotations that LLVM offers, we have to disallow transmuting data with padding to integer types).
+The right type to use for holding arbitrary data is `MaybeUninit`, so e.g. `[MaybeUninit<u8>; 1024]` for up to 1KiB of arbitrary data.
+`MaybeUninit` can also hold pointers with their provenance without any trouble.
+
+Because of that, I think we should move towards discouraging, deprecating, or even entirely disallowing pointer-integer transmutation in Rust.
+That means a cast is the only legal way to turn a pointer into an integer, and after the discussion above we got our casts covered.
+A [first careful step](https://github.com/rust-lang/rust/pull/95547) has recently been taken on this journey; the `mem::transmute` documentation now cautions against using this function to turn pointers into integers.
+
+**Update (2022-09-14):** After a lot more discussion, the current model pursued by the Unsafe Code Guidelines WG is to say that pointer-to-integer transmutation is permitted, but just strips provenance without exposing it.
+That means the program with the casts replaced by transmutation is UB, because the `ptr` it ends up dereferencing has invalid provenance.
+However, the transmutation itself is not UB.
+Basically, pointer-to-integer transmutation is equivalent to [the `addr` method](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.addr), with all its caveats -- in particular, transmuting a pointer to an integer and back is like calling `addr` and then calling [`ptr::invalid`](https://doc.rust-lang.org/nightly/std/ptr/fn.invalid.html).
+That is a *lossy* round-trip: it loses provenance information, making the resulting pointer invalid to dereference.
+It is lossy even if we use a regular integer-to-pointer cast (or `from_exposed_addr`) for the conversion back to a pointer, since the original provenance might never have been exposed.
+Compared to declaring the transmutation itself UB, this model has some nice properties that help compiler optimizations (such as removing unnecessary store-load round-trips). **/Update**
+
+## A new hope for Rust
+
+All in all, while the situation may be very complicated, I am actually more hopeful than ever that we can have both -- a precise memory model for Rust *and* all the optimizations we can hope for!
+The three core pillars of this approach are:
+- making pointer-integer casts "expose" the pointer's provenance,
+- offering `ptr.addr()` to learn a pointer's address *without* exposing its provenance,
+- and making pointer-integer transmutation round-trips lossy (such that the resulting pointer cannot be dereferenced).
+
+Together, they imply that we can optimize "nice" code (that follows Strict Provenance, and does not "expose" or use integer-pointer casts) perfectly, without any risk of breaking code that does use pointer-integer round-trips.
+In the easiest possible approach, the compiler can simply treat pointer-integer and integer-pointer casts as calls to some opaque external function.
+Even if the rest of the compiler literally entirely ignores the existence of pointer-integer round-trips, it will still support such code correctly!
+
+However, it's not just compilers and optimizers that benefit from this approach.
+One of my biggest quests is giving a [precise model](https://plv.mpi-sws.org/rustbelt/stacked-borrows/) of the Rust aliasing rules, and that task has just gotten infinitely easier.
+I used to worry *a lot* about pointer-integer round-trips while developing Stacked Borrows.
+This is the entire reason why all of this "untagged pointer" mess exists.
+
+Under this brave new world, I can entirely ignore pointer-integer round-trips when designing memory models for Rust.
+Once that design is done, support for pointer-integer round-trips can be added as follows:
+- When a pointer is cast to an integer, its provenance (whatever information it is that the model attaches to pointers -- in Stacked Borrows, this is called the pointer's *tag*) is marked as "exposed".
+- When an integer is cast to a pointer, we *guess* the provenance that the new pointer should have from among all the provenances that have been previously marked as "exposed".
+ (And I mean *all* of them, not just the ones that have been exposed "at the same address" or anything like that. People will inevitably do imperfect round-trips where the integer is being offset before being cast back to a pointer, and we should support that. As far as I know, this doesn't really cost us anything in terms of optimizations.)
+
+This "guess" does not need to be described by an algorithm.
+Through the magic that is formally known as [angelic non-determinism](https://en.wikipedia.org/wiki/Angelic_non-determinism), we can just wave our hands and say "the guess will be maximally in the programmer's favor": if *any* possible choice of (previously exposed) provenance makes the program work, then that is the provenance the new pointer will get.
+Only if *all* choices lead to Undefined Behavior, do we consider the program to be ill-defined.
+This may sound like cheating, but it is actually a legit technique in formal specifications.
+
+Also note how it's really *just* the integer-pointer casts that are making things so complicated here.
+If it weren't for them, we would not even need all that "exposure" machinery.
+Pointer-integer casts on their own are perfectly fine!
+That's why [`addr`](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.addr)+[`with_addr`](https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.with_addr) is such a nice API from a memory model perspective.[^fake_alloc]
+
+[^fake_alloc]: Even more specifically, it's the integer-pointer cast as part of a pointer-integer round-trip that are a problem. If you are just casting an integer constant to a pointer because on your platform that's where some fixed memory region lies, and if that memory is entirely outside of the global, stack, and heap allocations that the Rust language itself is aware of, we can still be friends.
+
+This approach *does* have the disadvantage that it becomes near impossible to write a tool like Miri that precisely matches the specification, since Miri cannot possibly implement this "guessing" accurately.
+However, Miri can still properly check code that uses Strict Provenance operations, so hopefully this is just yet another incentive (besides the more precise specification and better optimization potential) for programmers to move their code away from integer-pointer casts and towards Strict Provenance.
+And who knows, maybe there *is* a clever way that Miri can actually get reasonably close to checking this model?
+It doesn't have to be perfect to be useful.
+
+What I particularly like about this approach is that it makes pointer-integer round-trips a purely local concern.
+With an approach like Stacked Borrows "untagged pointers", *every* memory operation has to define how it handles such pointers.
+Complexity increases globally, and even when reasoning about Strict Provenance code we have to keep in mind that some pointers in other parts of the program might be "untagged".
+In contrast, this "guessing maximally in your favor"-based approach is entirely local; code that does not syntactically contain exposing pointer-integer or integer-pointer casts can literally forget that such casts exist at all.
+This is true both for programmers thinking about their `unsafe` code, and for compiler authors thinking about optimizations.
+Compositionality at its finest!
+
+## But what about C?
+
+I have talked a lot about my vision for "solving" pointer provenance in Rust.
+What about other languages?
+As you might have heard, C is moving towards making [PNVI-ae-udi](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2676.pdf) an official recommendation for how to interpret the C memory model.
+With C having so much more legacy code to care about and many more stakeholders than Rust does, this is an impressive achievement!
+How does it compare to all I said above?
+
+First of all, the "ae" part of the name refers to "address-exposed" -- that's exactly the same mechanism as what I described above!
+In fact, I have taken the liberty to use their terminology.
+So, on this front, I see Rust and C as moving into the same direction, which is great.
+(Now we just need to get LLVM to also move in that direction.)
+I should mention that PNVI-ae-udi does *not* account for the `restrict` modifier of C, so in a sense it is solving an easier problem than the Rust memory model which has no choice but to contend with interesting questions around aliasing restrictions.
+However, if/when a more precise model of C with `restrict` emerges, I don't think they will be moving away from the "address-exposed" model -- to the contrary, as I just argued this model means we can specify `restrict` without giving a thought to pointer-integer round-trips.
+
+The "udi" part of the name means "user disambiguation", and is basically the mechanism by which an integer-pointer cast in C "guesses" the provenance it has to pick up.
+The details of this are complicated, but the end-to-end effect is basically exactly the same as in the "best possible guess" model I have described above!
+Here, too, my vision for Rust aligns very well with the direction C is taking.
+(The set of valid guesses in C is just a lot more restricted since they do not have `wrapping_offset`, and the model does not cover `restrict`.
+That means they can actually feasibly give an algorithm for how to do the guessing.
+They don't have to invoke scary terms like "angelic non-determinism", but the end result is the same -- and to me, the fact that it is equivalent to angelic non-determinism is what justifies this as a reasonable semantics.
+Presenting this as a concrete algorithm to pick a suitable provenance is then just a stylistic choice.)
+Kudos go to Michael Sammler for opening my eyes to this interpretation of "user disambiguation", and arguing that angelic non-determinism might not be such a crazy idea after all.
+
+What is left is the question of how to handle pointer-integer transmutation, and this is where the roads are forking.
+PNVI-ae-udi explicitly says loading from a union field at integer type exposes the provenance of the pointer being loaded, if any.
+So, the example with `transmute_union` would be allowed, meaning the optimization of removing the "dead" load from the `union` would *not* (in general) be allowed.
+Same for `transmute_memcpy`, where the proposal says that when we access the contents of `ret` at type `uintptr_t`, that will again implicitly expose the provenance of the pointer.
+
+I think there are several reasons why this choice makes sense for C, that do not apply to Rust:
+- There is a *lot* of legacy code. A *LOT*.
+- There is no alternative like `MaybeUninit` that could be used to hold data without losing provenance.
+- Strict aliasing means that not *all* loads at integer type have to worry about provenance; only loads at character type are affected.
+
+On the other hand, I am afraid that this choice might come with a significant cost in terms of lost optimizations.
+As the example above shows, the compiler has to be very careful when removing any operation that can expose a provenance, since there might be integer-pointer casts later that rely on this.
+(Of course, until this is actually implemented in GCC or LLVM, it will be hard to know the actual cost.)
+Because of all that, I think it is reasonable for Rust to make a different choice here.
+
+## Conclusion
+
+This was a long post, but I hope you found it worth reading. :)
+To summarize, my concrete calls for action in Rust are:
+- Code that uses pointer-integer transmutation round-trips should migrate to regular casts or `MaybeUninit` transmutation ASAP.
+ I think we should declare pointer-integer transmutation as "losing" provenance, so code that assumes a lossless transmutation round-trip has Undefined Behavior.
+- Code that uses pointer-integer or integer-pointer *casts* might consider migrating to the Strict Provenance APIs.
+ You can do this even on stable with [this polyfill crate](https://crates.io/crates/sptr).
+ However, such code *is and remains* well-defined. It just might not be optimized as well as one could hope, it might not compile on CHERI, and Miri will probably miss some bugs.
+ If there are important use-cases not covered by Strict Provenance, we'd like to hear about them!
+
+This is a large undertaking and will require a lot of work!
+However, at the end of this road is a language with a coherent, well-defined memory model *and* support for doing unspeakable things to pointers *without* incurring a (reasoning or optimization) cost on code that is perfectly nice to its pointers.
+Let us work towards this future together. :)
+
+## Appendix
+
+#### Integer-pointer casts are not pure, either
+
+I promised an example of how integer-pointer casts are "impure", in the sense that two casts with the same input integer can produce different pointers:
+
+{% highlight c %}
+static int uwu(int *restrict x, int *restrict y) {
+ *x = 0;
+ *y = 0;
+
+ uintptr_t xaddr = (uintptr_t)x;
+ int *y2 = y-1;
+ uintptr_t y2addr = (uintptr_t)y2;
+ assert(xaddr == y2addr);
+
+ int *xcopy = (int*)xaddr;
+ int *y2copy = (int*)y2addr;
+ int *ycopy = y2copy+1;
+
+ return *xcopy + *ycopy;
+}
+
+int main() {
+ int i[2] = {0, 0};
+ uwu(&i[0], &i[1]);
+}
+{% endhighlight %}
+
+If we ignore the pointer-integer round-trips, this uses `x` and `xcopy` to access `i[0]`, while using `y` and `ycopy` to access `i[1]`, so this should be uncontroversial.
+`ycopy` is computed via `(y-1)+1`, but hopefully nobody disagrees with that.
+Then we just add some pointer-integer round-trips.
+
+But now, consider that `(int*)xaddr` and `(int*)y2addr` take the same integer as input!
+If the compiler were to treat integer-pointer casts as a pure, deterministic operation, it could replace `(int*)y2addr` by `xcopy`.
+However, that would mean `xcopy` and `ycopy` have the same provenance!
+And there exists no provenance in this program that has access to both `i[0]` and `i[1]`.
+So, either the cast has to synthesize a new provenance that has never been seen before, or doing common subexpression elimination on integer-pointer casts is wrong.
+
+My personal stance is that we should not let the cast synthesize a new provenance.
+This would entirely lose the benefit I discussed above of making pointer-integer round-trips a *local* concern -- if these round-trips produce new, never-before-seen kinds of provenance, then the entire rest of the memory model has to define how it deals with those provenances.
+We already have no choice but treat pointer-integer casts as an operation with side-effects; let's just do the same with integer-pointer casts and remain sure that no matter what the aliasing rules are, they will work fine even in the presence of pointer-integer round-trips.
+
+That said, under this model integer-pointer casts still have no side-effect, in the sense that just removing them (if their result is unused) is fine.
+Hence, it *could* make sense to implicitly perform integer-pointer casts in some situations, like when an integer value (without provenance) is used in a pointer operation (due to an integer-to-pointer transmutation).
+This breaks some optimizations like load fusion (turning two loads into one assumes the same provenance was picked both times), but most optimizations (in particular dead code elimination) are unaffected.
+
+#### What about LLVM?
+
+I discussed above how my vision for Rust relates to the direction C is moving towards.
+What does that mean for the design space of LLVM?
+Which changes would have to be made to fix (potential) miscompilations in LLVM and to make it compatible with these ideas for C and/or Rust?
+Here's the list of open problems I am aware of:
+- LLVM would have to to stop [removing `inttoptr(ptrtoint(_))`](https://github.com/llvm/llvm-project/issues/33896) and stop doing [replacement of `==`-equal pointers](https://github.com/llvm/llvm-project/issues/34577).
+- As the first example shows, LLVM also needs to treat `ptrtoint` as a side-effecting operation that has to be kept around even when its result is unused. (Of course, as with everything I say here, there can be special cases where the old optimizations are still correct, but they need extra justification.)
+- I think LLVM should also treat `inttoptr` as a side-effecting (and, in particular, non-deterministic) operation, as per the last example. However, this could possibly be avoided with a `noalias` model that specifically accounts for new kinds of provenance being synthesized by casts. (I am being vague here since I don't know what that provenance needs to look like.)
+
+So far, this all applies to LLVM as a Rust and C backend equally, so I don't think there are any good alternatives.
+On the plus side, adapting this strategy for `inttoptr` and `ptrtoint` means that the recent LLVM ["Full Restrict Support"](https://lists.llvm.org/pipermail/llvm-dev/2019-March/131127.html) can also handle pointer-integer round-trips "for free"!
+
+Adding `with_addr`/`copy_alloc_id` to LLVM is not strictly necessary, since it can be implemented with `getelementptr` (without `inbounds`).
+However, optimizations don't seem to always deal well with that pattern, so it might still be a good idea to add this as a primitive operation to LLVM.
+
+Where things become more subtle is around pointer-integer transmutation.
+If LLVM wants to keep doing replacement of `==`-equal integers (which I strongly assume to be the case), *something* needs to give: my first example, with casts replaced by transmutation, shows a miscompilation.
+If we focus on doing an `i64` load of a pointer value (e.g. as in the LLVM IR produced by `transmute_union`, or pointer-based transmutation in Rust), what are the options?
+Here are the ones I have seen so far (but there might be more, of course):
+1. The load could be said to behave like `ptrtoint`. This means it strips provenance and as a side-effect, it also exposes the pointer.
+2. The load could be said to just strip provenance *without* exposing the pointer.
+3. The load could be simply UB or return `poison`.
+4. The load could produce an integer with provenance, *and moreover* any computation on such an integer (including `icmp`) is UB (or returns `poison`).
+ This has some subtle consequences, but they might be mostly harmless. For example, `x` can no longer be replaced by `x+0`.
+ We cannot assume that it is safe to compare arbitrary `i64` and branch on the result, even if they are `noundef`. Or maybe `noundef` also excludes provenance?
+ This is certainly the least obvious alternative.
+
+Except for the first option, these all say that my example with transmutation instead of the pointer-integer casts is UB, which avoids the optimization problems that arise from accepting that example.
+That is fine for my vision for Rust, but a problem for C with PNVI-ae-udi.
+Only the first option is compatible with that, but that option also means entirely removing a load is non-trivial even if its result is unused!
+I hope we can avoid that cost for Rust.
+
+Another interesting difference between these options is whether the resulting semantics are "monotone" with respect to provenance: is "increasing" the provenance of a value (i.e., letting it access more memory) a legal program transformation?
+With the last two options, it is not, since adding provenance to a value that did not have it can introduce Undefined Behavior.
+The first two options are "monotone" in this sense, which seems like a nice property.
+(This is comparable to how the semantics are "monotone" with respect to `undef` and `poison`: replacing either of them by a fixed value is a legal program transformation. For `undef`/`poison` this is crucially important, for provenance it seems more like a sanity check of the semantics.)
+
+In all of these cases except the last one, LLVM would probably need something like a [byte type](https://gist.github.com/georgemitenkov/3def898b8845c2cc161bd216cbbdb81f) so that a load of arbitrary data (including a pointer with provenance) can be done without losing the provenance attached to the data.
+
+A similar question arises for doing a pointer-typed load of a bare integer (integer-pointer transmutation):
+1. The load could have the effects of a `inttoptr`. This is less clearly bad than a `ptrtoint`, but is still tricky since (at least without extra work) `inttoptr` is non-deterministic and depends on the global set of exposed provenances (so, it cannot be easily reordered up across potentially exposing operations).
+ I also have [another example](https://github.com/rust-lang/unsafe-code-guidelines/issues/286#issuecomment-860189806) showing that if *both* pointer-integer transmutation and integer-pointer transmutation work like the corresponding casts (i.e., if the first of my options is picked for both loads of pointers at integer type, and integers at pointer type), then more optimizations fail:
+ removing a store that follows a load and just writes back the same value that has just been loaded is no longer correct.
+ Yet, I think this is what PNVI-ae-udi mandates. Again I hope Rust can opt-out of this.
+2. The load could create a pointer with "invalid" provenance.
+ That means transmutation of a pointer to an integer and back produces a pointer that cannot be used to access memory, but avoids all the analysis difficulties that come with an `inttoptr`.
+ This is what I think would be best for Rust.
+3. The load could produce `poison`, but I see no good reason for doing that.
+
+Since LLVM generally errs on the side of delaying UB as long as possible if that is not in conflict with optimizations, the second option for both questions feels most "on-brand" to me personally -- but in the end, these are some hard choices that the LLVM community will have to make.
+I can help evaluate these trade-offs by giving structure to the design space and pointing out the inevitable consequences of certain decisions, but I can only lend a hand here -- while I think and care a lot about LLVM semantics, I haven't done any direct work on LLVM myself.
+I am also not enough of an expert for which optimizations are important and the performance impact of the various options here, so I hope we can get people with that kind of background involved in the discussion as well.
+For the sake of the entire ecosystem I mostly hope that LLVM will make *some* choice so that we can, eventually, leave this limbo state we are currently in.
+
+#### Footnotes
--- /dev/null
+---
+title: "The last two years in Miri"
+categories: rust
+reddit: /rust/comments/vq3mmu/the_last_two_years_in_miri/
+---
+
+It has been [almost two years]({% post_url 2020-09-28-miri %}) since my last Miri status update.
+A lot has happened in the mean time that I would like to tell you all about!
+If you are using Miri, you might also be seeing new errors in code that previously worked fine; read on for more details on that.
+
+For the uninitiated, [Miri](https://github.com/rust-lang/miri/) is an interpreter that runs your Rust code and checks if it triggers any [Undefined Behavior](https://doc.rust-lang.org/reference/behavior-considered-undefined.html) (UB for short).
+You can think of it a as very thorough (and very slow) version of valgrind/ASan/TSan/UBSan:
+Miri will detect when your program uses uninitialized memory incorrectly, performs out-of-bounds memory accesses or pointer arithmetic, causes a data race, violates key language invariants, does not ensure proper pointer alignment, or causes incorrect aliasing.
+As such, it is most helpful when writing unsafe code, as it aids in ensuring that you follow all the rules required for unsafe code to be correct and safe.
+Miri also detects memory leaks, i.e., it informs you at the end of program execution if there is any memory that was not deallocated properly.
+
+<!-- MORE -->
+
+Moreover, Miri is able to run code for other targets: for example, you might be developing code on x86_64, a 64-bit little-endian architecture.
+When you do low-level bit manipulation, it is easy to introduce bugs that only show up on 32-bit systems or big-endian architectures.
+You can run Miri with `--target i686-unknown-linux-gnu` and `--target mips64-unknown-linux-gnuabi64` to test your code in those situations -- and this will work even if your host OS is macOS or Windows!
+
+That said, it's not all roses and rainbows.
+Since Miri just knows how to interpret Rust code, it will get stuck when you call into C code.
+Miri knows how to execute a certain small set of well-known C functions (e.g. to access environment variables or open files), but it is still easy to run into an "unsupported operation" error due to missing C library implementations.
+In many cases you should be able to still write tests that cover the remaining code that does not need to, for example, directly access the network;
+but I also hope that Miri will keep growing its support for key platform APIs.
+
+## Miri progress
+
+So, what progress has Miri made in the last two years?
+
+### Concurrency
+
+The story of concurrency in Miri continues to surprise me: I had not even planned for Miri to support concurrency, but people just keep showing up and implement one part of it after the other, so now we have pretty good support for finding concurrency bugs!
+
+In that spirit, @JCTyblaidd implemented a data race detector.
+So if your code does not use appropriate atomic operations to make sure all accesses are suitably synchronized, Miri will now detect that problem and report Undefined Behavior.
+[Here's a demo](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=2dc29d339658dd2e1e74b84fcffc3926).
+(Click that link and then select "Tools - Miri" to see this in action.)
+Our data race error reports could be improved a lot (in particular they only show one of the two conflicting accesses involved in a data race), but they are still useful and have already found several data races in the wild.
+
+@thomcc changed our `compare_exchange_weak` implementation so that it randomly just fails with 80% probability.
+(The exact rate is adjustable via `-Zmiri-compared-exchange-weak-failure-rate=<x>`.)
+[Here's a demo](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0af8443b760985ce01640135ffb83749).
+This is super useful to find issues where code uses `compare_exchange_weak` but cannot handle spurious failures, since those are very unlikely to occur in the wild.
+
+@henryboisdequin added support for the atomic `fetch_min` and `fetch_max` operations, completing our support of the `Atomic*` types.
+
+And finally, @cbeuw showed up and added "weak memory emulation".
+This means that when you do an atomic load, you might not observe the latest value written to that location; instead, a previous value can be returned.
+[Here's a demo](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=7809a750bda54f0cc458b81823c79db7).
+This happens on real hardware, so having this supported in Miri helps to find more potential bugs.
+The caveat is that Miri still cannot produce *all* the behaviors that the actual program might exhibit.
+Also, the C++20 revision of the C++ memory model disallowed some possible behaviors that were previously allowed, but Miri might produce those behaviors -- there is currently no known algorithm that would prevent that.
+This should be very rare though.
+
+I then just put the icing on the cake by fixing some long-standing issues in our scheduler, so that it no longer gets stuck in spin loops.
+Miri now has a chance to preempt the running thread at the end of each basic block; the preemption probability is 1% but you can adjust it (using `-Zmiri-preemption-rate=<x>`).
+
+All of this made our concurrency support sufficiently solid that it no longer shows any warning about being "experimental".
+For example, it has already found a [data race in the standard library](https://github.com/rust-lang/rust/issues/98498).
+I can barely express how happy and proud I am that I had to do basically none of this work. :)
+
+One warning though: several of the improvements mentioned above rely on doing random choices.
+So, it is now more likely than before that Miri will work fine one day, and then show an error after some seemingly inconsequential change to the program the next day.
+I will get back to these problems later.
+
+### Pointer provenance and Stacked Borrows
+
+One of the most subtle aspects of Miri is [Stacked Borrows](https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md).
+The aliasing model is already quite complicated, and actually debugging what happens when Miri finds an aliasing violation in your code can be pretty tricky.
+However, @saethlin made this a lot easier!
+The error messages now show a lot more detail and point to several relevant locations in the code: not only where the bad access happened, but also where the pointer tag used for that access was created, and where that tag was invalidated.
+I am very impressed by how good some of these errors are, just [check this out](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e831ed0f262e039dda4d24c159a9f5b0).
+
+Another big thing that happened recently is the entire ["Strict Provenance" story](https://doc.rust-lang.org/nightly/std/ptr/index.html#strict-provenance).
+I am super excited by these developments, because they offer the chance to fix some long-standing open problems in Miri:
+the issues with "untagged" raw pointers in Stacked Borrows, and Miri not properly supporting integer-to-pointer casts.
+
+After a lot of work by @carbotaniuman and myself, the situation now is as follows:
+- Miri always properly tags raw pointers.
+ So there are no longer any counter-intuitive behaviors caused by Miri "mixing up" two raw pointers that point to the same address, but were computed in a different way.
+ (We had a `-Zmiri-tag-raw-pointers` flag for a while that also achieves this; that flag is now on-by-default.)
+- If you do not use any integer-to-pointer casts, then you can stop reading here!
+ You can pass `-Zmiri-strict-provenance` to Miri to ensure that this is indeed the case.
+- If you *are* using integer-to-pointer casts, then Miri will warn about that. You now have two options.
+ - The ideal solution is to avoid using integer-to-pointer casts, and to follow Strict Provenance instead.
+ The [pointer library docs](https://doc.rust-lang.org/nightly/std/ptr/index.html#strict-provenance) explain in more detail what exactly that means.
+ Note that the APIs described there are still unstable, but a [polyfill](https://crates.io/crates/sptr) is available for stable Rust.
+ Also see [Gankra's blog post](https://gankra.github.io/blah/tower-of-weakenings/) and [my own blog post]({% post_url 2022-04-11-provenance-exposed %}) for some more background on this subject.
+ - If the casts are in code you do not control, or if you cannot currently avoid integer-to-pointer casts, you can pass `-Zmiri-permissive-provenance` to Miri to silence the warning.
+ Know that this means that Miri might miss some bugs in your code:
+ integer-to-pointer casts make it impossible to precisely track which pointer came from where, so Miri will conservatively accept some code that actually should be rejected.
+
+This is overall much better than previously -- there is nothing funky going on with raw pointers any more, and we should never incorrectly report UB any more even when integer-to-pointer casts are used.
+:-)
+
+### Other areas
+
+Concurrency and pointer aliasing are the two big changes, but there is also a long tail of smaller changes that together make Miri a hack of a lot more useful than it used to be:
+
+- @teryror made Miri support doctests, so now `cargo miri test` will also check your doctests for UB!
+- @Smittyvb fixed our fast-math intrinsics to properly report UB when they are used on non-finite values.
+- @hyd-dev added "symbol resolution" support to Miri, so if one part of your Rust code defines a function with a given `link_name`, and another piece of Rust code imports that function via an `extern` block, Miri now knows how to find the right function implementation.
+- @atsmtat added a `-Zmiri-isolation-error=<action>` flag so when a function call is rejected due to isolation, evaluation can continue by reporting an error code to the interpreted program.
+- @landaire added `-Zmiri-panic-on-unsupported`, which makes Miri raise a panic rather than stopping evaluation when an unsupported system function is encountered.
+This can be useful to keep going with the next test in a test suite.
+However, it also raises panics where usually that would be impossible, which can lead to surprising behavior.
+- @DrMeepster added support for running programs that use the `#[start]` attribute, and @oli-obk made that work even for targets without `libstd`.
+(You need to set `MIRI_NO_STD=1` to make the latter work.)
+- @DrMeepster also implemented support for the `#[global_allocator]` attribute.
+- @camelid made Miri optionally detect UB due to uninitialized integers, which has since become the default.
+- @saethlin made our errors more readable by pruning irrelevant details from the backtraces.
+- I have implemented support for calling methods on types like `Pin<Box<dyn Trait>>`.
+- @oli-obk fixed our handling of types like `MaybeUninit<u64>`, where previously we did not properly support only *some* of the bytes being initialized.
+
+We also improved out platform API and intrinsic support:
+
+- Thanks to @m-ou-se Miri now supports the Linux futex APIs used by the Rust standard library.
+ This was crucial for std's `park()` and `unpark()`, but meanwhile is also used for many other synchronization primitives.
+- On the file system side, @Aaron1011 implemented `readlink`, which makes `std::fs::read_link` work on Linux and macOS.
+- @asquared31415 made the three-argument form of `open` work.
+- @tavianator implemented `readdir64` so we can still list directories on Linux (the Rust standard library was changed to use that function rather than `readdir64_r`).
+- @Aaron1011 has also improved the rendering of panic backtraces inside the interpreter.
+- @frewsxcv implemented the missing bits to make the aarch64-apple-darwin target work in Miri.
+- I implemented the intrinsics required by `std::simd`, so portable-simd code should work with Miri.
+ It will not be very fast, though...
+- @V0ldek made our Windows `GetSystemInfo` shim work in more situations.
+- @saethlin added support for `*_COARSE` clocks on Linux.
+- @InfRandomness has started on getting Miri to work on FreeBSD targets (but this support is still incomplete).
+
+### Bugfixes and cleanup
+
+And of course there were tons of bugfixes.
+I want to particularly call out @hyd-dev who fixed a *lot* of issues in our `cargo miri` frontend.
+@dtolnay did a lot of code cleanup, making Miri pass by clippy's critical eyes and ensuring all our tests are properly formatted.
+And last not least, @oli-obk completely re-wrote our test suite so that we can finally actually test the full output of Miri.
+
+I have probably forgotten to mention something interesting as well.
+[See here](https://github.com/rust-lang/miri/graphs/contributors?from=2020-09-29&to=2022-07-02&type=c) for the full list of amazing people who contributed to Miri since my last update.
+I cannot thank all of you enough! <3
+
+## Help, Miri suddenly says my code is broken
+
+Several of the changes mentioned above, in particular with regards to concurrency and Stacked Borrows, mean that Miri is now able to detect more problems than before.
+On the one hand, that's of course great, but on the other hand, it can mean that when you re-test Miri on some code that seemed fine, it might suddenly complain!
+And because of all the non-determinism, it might also be the case that Miri *sometimes* complains, and sometimes doesn't (or that it works fine locally but complains on CI).
+What can you do when that happens?
+
+If Miri shows a new Stacked Borrows error, then that is probably caused by raw pointers now being properly tagged.
+The new Stacked Borrows messages should make it easier than before to diagnose these problems, but in the end this still remains a case-by-case issue.
+For example, [this program](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=14ddec169895d111578ec96757df95d1) will print:
+```
+error: Undefined Behavior: attempting a read access using <3255> at alloc1770[0x4], but that tag does not exist in the borrow stack for this location
+ --> src/main.rs:4:25
+ |
+4 | let _val = unsafe { *ptr.add(1) }; // ...and use it to access the *second* element.
+ | ^^^^^^^^^^^
+ | |
+ | attempting a read access using <3255> at alloc1770[0x4], but that tag does not exist in the borrow stack for this location
+ | this error occurs as part of an access at alloc1770[0x4..0x8]
+ |
+ = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
+ = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
+help: <3255> was created by a retag at offsets [0x0..0x4]
+ --> src/main.rs:3:15
+ |
+3 | let ptr = &x[0] as *const i32; // We create a pointer to the *first* element...
+ | ^^^^^
+ = note: backtrace:
+ = note: inside `main` at src/main.rs:4:25
+```
+In this case, the clue is in the offsets: note that the tag was created for offsets `[0x0..0x4]` (as usual in Rust, this *excludes* `0x4`), and the access was at `alloc1770[0x4]`.
+The pointer was thus used outside the offset range for which its tag (`<3255>`) is valid.
+The fix is to use `x.as_ptr()` rather than `&x[0] as *const i32` to get a pointer that is valid for the entire array.
+
+If the error only shows up sometimes, then it probably has something to do with concurrency.
+Miri is not *truly* random, but uses a pseudo-random number generator to make all concurrency-related choices (such as when to schedule another thread).
+This means you can explore various different possible choices by passing different *seeds* for Miri to use for its pseudo-random number generator.
+The following little shell snippet will run Miri with many different seeds, which is great to be able to locally reproduce a failure that you saw on CI, but that you are having trouble reproducing:
+```
+for SEED in $({ echo obase=16; seq 0 255; } | bc); do
+ echo "Trying seed: $SEED"
+ MIRIFLAGS=-Zmiri-seed=$SEED cargo miri test || { echo "Failing seed: $SEED"; break; };
+done
+```
+It is important that you use exactly the same `MIRIFLAGS` as CI to ensure the failure can even happen!
+It is also a good idea to use a filter with `cargo miri test FILTER` to ensure only the test you care about is being run.
+
+Once you confirmed that this is indeed a non-deterministic test failure, you can narrow it down further by reducing Miri's non-determinism:
+- You can pass `-Zmiri-preemption-rate=0` to make the scheduler non-preemptive (only schedule to other threads when a thread explicitly yields).
+ This *can* lead to infinite loops if there are spin-loops that do not yield, but if it makes the problem go away, then the problem needs some very particular scheduling decisions to surface, which might help you track down its source.
+- You can also pass `-Zmiri-disable-weak-memory-emulation` which has the effect of making atomic loads always return the latest value stored in that location.
+ If that makes the problem go away, then the issue is likely caused by insufficient synchronization somewhere. It might be a missing fence, or a `Relaxed` access that should be `Release`/`Acquire`.
+- Finally, `-Zmiri-compare-exchange-weak-failure-rate=0` makes `compared_exchange_weak` behave exactly like `compare_exchange`.
+ If that makes the problem go away, then some code using `compared_exchange_weak` is not properly handling spurious failures.
+
+Passing all of these flags will make Miri's concurrency entirely deterministic.
+That can be useful to avoid non-deterministic test failures, but note that this will also mask many real-world bugs.
+Those test failures are often real, even if they can be hard to track down!
+
+If you are still having trouble, feel free to come visit us in our [Zulip stream](https://rust-lang.zulipchat.com/#narrow/stream/269128-miri), which is the official communication channel for Miri.
+
+By the way, if you are still disabling some tests on Miri because Miri used to not support panics/concurrency, it's time to give those tests another try. :)
+So this is a good opportunity to go over your `cfg(miri)` and similar attributes and re-evaluate if they are still needed.
+
+## Using Miri
+
+If this post made you curious and you want to give Miri a try, here's how to do that.
+Assuming you have a crate with some unsafe code, and you already have a test suite (you are testing your unsafe code, right?), you can just install Miri (`rustup +nightly component add miri`) and then run `cargo +nightly miri test` to execute all tests in Miri.
+Note that this requires the nightly toolchain as Miri is still an experimental tool.
+
+Miri is very slow, so it is likely that some tests will take way too long to be feasible.
+You can adjust iteration counts in Miri without affecting non-Miri testing as follows:
+{% highlight rust %}
+let limit = if cfg!(miri) { 10 } else { 10_000 };
+{% endhighlight %}
+If your test suite needs to access OS facilities such as timers or the file system, set `MIRIFLAGS=-Zmiri-disable-isolation` to enable those.
+(Miri will tell you when that is necessary.)
+If your test suite runs into an unsupported operation, please [report an issue](https://github.com/rust-lang/miri/issues).
+However, note that we can only really support sufficiently "generic" operations -- like accessing file systems and network sockets.
+To implement things like `Py_IsInitialized` would mean putting a Python interpreter into Miri; that is not going to happen. ;)
+
+If you want to add Miri to your CI to ensure your test suite keeps working in Miri, please consult our [README](https://github.com/rust-lang/miri/#running-miri-on-ci).
+That document is also a great starting point for any other questions you might have.
+
+Miri is also integrated into the [Rust Playground](https://play.rust-lang.org/): you can select Miri in the "Tools" menu to check the code for Undefined Behavior.
+
+If Miri complains about your code and you do not understand why, we are happy to help!
+The best place to ask for support is our [Zulip stream](https://rust-lang.zulipchat.com/#narrow/stream/269128-miri).
+Questions are much easier to answer if you manage to reproduce the problem in a small self-contained bit of example code (ideally on the playground), but feel free to ask even if you do not know how to reduce the problem.
+
+## Helping Miri
+
+If you want to help improve Miri, that's awesome!
+The [issue tracker](https://github.com/rust-lang/miri/issues) is a good place to start; the list of issues is short enough that you can just browse through it rather quickly to see if anything pikes your interest.
+The ones that are particularly suited for getting started are marked with a green label, but notice that even "E-easy" issues can require some amount of Rust experience -- Miri is not a good codebase for your first steps in Rust.
+Another good starting point is to try to implement the missing bit of functionality that keeps your test suite from working.
+If you need any mentoring, just [get in touch](https://rust-lang.zulipchat.com/#narrow/stream/269128-miri). :)
+
+That's it for now.
+I am totally blown away by how many people (and companies!) are already using and even contributing to Miri.
+This endeavor of re-shaping the way we approach correctness of unsafe code has been way more successful than my wildest dreams.
+I hope Miri can also help you to ensure correctness of your unsafe code, and I am excited for what the next year(s) of Miri development will bring. :D
--- /dev/null
+---
+title: "Announcing: MiniRust"
+categories: rust research
+reddit: /rust/comments/wiwjch/announcing_minirust/
+---
+
+I have been thinking about the semantics of Rust -- as in, the intended behavior of Rust programs when executed, in particular those containing unsafe code -- a lot.
+Probably too much.
+But all of these thoughts are just in my head, which is not very useful when someone else wants to try and figure out how some tricky bit of unsafe Rust code behaves.
+As part of the [Unsafe Code Guidelines](https://github.com/rust-lang/unsafe-code-guidelines/) project, we often get questions asking whether a *concrete* piece of code is fine or whether it has Undefined Behavior.
+But clearly, that doesn't scale: there are just too many questions to be asked, and figuring out the semantics by interacting with an oracle with many-day latency is rather frustrating.
+We have [Miri](https://github.com/rust-lang/miri/), which is a much quicker oracle, but it's also not always right and even then, it can just answer questions of the form "is this particular program fine"; users have to do all the work of figuring out the model that *generates* those answers themselves.
+
+<!-- MORE -->
+
+So I have promised for a long time to find some more holistic way to write down my thoughts on unsafe Rust semantics.
+I thought I could do it in 2021, but I, uh, "slightly" missed that deadline... but better late than never!
+At long last, I can finally present to you: [**MiniRust**](https://github.com/RalfJung/minirust).[^name]
+
+[^name]: I am beginning to wonder if this name was a bad choice. Naming is not my strong suit. Maybe "CoreRust" would have been better? Alas...
+
+The purpose of MiniRust is to describe the semantics of an interesting fragment of Rust in a way that is both precise and understandable to as many people as possible.
+These goals are somewhat at odds with each other -- the most precise definitions, e.g. carried out in the Coq Proof Assistant, tend to not be very accessible.
+English language, on the other hand, is not very precise.
+So my compromise solution is to write down the semantics in a format that is hopefully known to everyone who could be interested: in Rust code.
+Specifically, MiniRust is specified by a *reference interpreter* that describes the step-by-step process of executing a MiniRust program, *including* checking at each step whether the program has Undefined Behavior.
+
+"Hold on", I hear a [Cool Bear](https://fasterthanli.me/articles/) say, "you are defining Rust in Rust code? Isn't that cyclic?"[^bear]
+Well, yes and no. It's not *really* Rust code.
+It's what I call "pseudo Rust", uses only a tiny fragment of the language (in particular, no `unsafe`), and then extends the language with some conveniences to make things less verbose.
+The idea is that anyone who knows Rust should immediately be able to understand what this code means, but also hopefully eventually if this idea pans out we can have tooling to translate pseudo Rust into "real" languages -- in particular, real Rust and Coq.
+Translating it to real Rust means we can actually execute the reference interpreter and test it, and translating it to Coq means we can start proving theorems about it.
+But I am getting waaaay ahead of myself, these are rather long-term plans.
+
+[^bear]: Thanks to fasterthanlime for facilitating the bear's appearance on this blog.
+
+**Update (2023-02-13):** "Pseudo Rust" has now been renamed to "specr lang", the language of the work-in-progress "specr" tool that can translate specr lang into Rust code to make specifications executable. **/Update**
+
+So, if you want to look into my brain to see how I see Rust programs, then please go check out [MiniRust](https://github.com/RalfJung/minirust).
+The README explains the scope and goals, the general structure, and the details of ~~pseudo Rust~~ specr lang, as well as a comparison with some related efforts.
+
+In particular I find that the concept of "places" and "values", which can be rather mysterious, becomes a lot clearer when spelled out like that, but that might just be me.
+I hasten to add that this is *very early work-in-progress*, and it is *my own personal experiment*, not necessarily reflecting the views of anyone else.
+It is also *far from feature-complete*, in fact it has just barely enough to be interesting.
+There are lots of small things missing (like integers that aren't exactly 2 bytes in size, or tuples that don't have exactly 2 elements), but the biggest omission by far is the total lack of an aliasing model.
+And unsized types. And concurrency. And probably other things.
+
+On the other hand, there are many things that it *can* explain in full precision:
+- validity invariants, and how they arise from the mapping between a high-level concept of "values" and a low-level concept of "sequences of bytes"
+- the basic idea of provenance tracking the "allocation" a pointer points to, and how that interacts with pointer arithmetic (including `offset` and `wrapping_offset`)
+- how pointer provenance behaves when doing transmutation between pointers and integers
+- what happens when *casting* between pointers and integers
+- padding (that's why tuples can have 2 elements, so there can be padding between them)
+
+If you are not used to reading interpreter source code, then I guess this can be rather jarring, and there is certainly a *lot* of work that could and should be done to make this more accessible.
+(Like, examples. I hear people like examples.)
+But just being able to talk about these questions with precision *at all* has already lead to some interesting discussions in the UCG WG, some of which made me change my mind -- thanks in particular to @digama0, @JakobDegen, and @alercah for engaging deeply with my ideas.
+So for now it is serving its purpose, and maybe some of you can find it useful, too.
+Hopefully we can even use this as a starting place for seriously tackling the issue of an *official* specification of Rust.
+More on that soon. :)
--- /dev/null
+---
+title: "A New Beginning"
+categories: research rust
+---
+
+I have some very exciting news to share: starting November 1st, I will work at ETH Zürich as an assistant professor!
+Becoming a professor in the first place is a dream come true, and becoming a professor at a place like ETH Zürich is not something I even dared to dream of.
+I still cannot quite believe that this is actually happening (I will be *professor*?!??), but [the news is out](https://twitter.com/CSatETH/status/1548944615285350400) so I guess this is real. :D
+
+<!-- MORE -->
+
+I feel excited and terrified in about equal parts.
+Excited by all the new possibilities, by the prospect of working with students and inspiring the next generation of researchers;
+terrified by all the responsibility and the prospect of having to stand in a classroom and give a lecture in only a few months' time.
+But somehow everyone else seems confident that I can do this, so I guess I'll just play along and hope that I will not prove them wrong...
+
+I am also humbled and eternally thankful for being given this opportunity.
+Being able to work in an environment like ETH is a privilege beyond imagination, and I don't know how I got so lucky.
+I probably used up all my Karma points for the rest of my life, and will do my best to honor this privilege.
+I feel hugely indebted to everyone I worked with, first and foremost of course my PhD advisor [Derek Dreyer](https://people.mpi-sws.org/~dreyer/).
+But I would also like to specifically call out the Rust community, because I don't think this would have happened without Rust -- thanks to *everyone* who contributed to this language that I am essentially building my career on[^rust], and thanks in particular to everyone who indulged in my ideas for how Rust should approach unsafe code and helped me shape that corner of the language.
+
+[^rust]: Before anyone gets worried, I also have some [ideas](https://iris-project.org/) I want to pursue that are unrelated to Rust. But Rust is currently by far the biggest inspiration for new research problems for me, and without Rust I don't think my research would be anywhere near as applied and impactful as it is today, which I am sure played a key role in the decision of ETH to hire me.
+
+So what's next?
+I will soon finish my post-doc at MIT and move back to Europe, and then move to Zürich in October.
+And then I will have to figure out how this being-a-professor thing works. ;)
+My first main priority is building a research group: the "Programming Language Foundations Lab"[^lab].
+So if you are interested in doing a PhD or post-doc working on, well, programming language foundations, and in particular formal foundations for Rust, or if you are an ETH student interested in a Master Thesis in that area -- please [reach out](https://research.ralfj.de/contact.html)!
+I am still figuring out how to do things like hiring people and finding suitable projects, but there is no shortage of open problems that need solving and theorems that need proving. :)
+
+[^lab]: Yes, I have a lab coat. I don't usually wear it though... and if you want to see me wear it, that will cost you some beer.
--- /dev/null
+---
+title: "cargo careful: run your Rust code with extra careful debug checking"
+categories: rust
+reddit: /rust/comments/xogayv/cargo_careful_run_your_rust_code_with_extra/
+---
+
+Did you know that the standard library is full of useful checks that users never get to see?
+There are plenty of debug assertions in the standard library that will do things like check that `char::from_u32_unchecked` is called on a valid `char`, that `CStr::from_bytes_with_nul_unchecked` does not have internal nul bytes, or that pointer functions such as `copy` or `copy_nonoverlapping` are called on suitably aligned non-null (and non-overlapping) pointers.
+However, the regular standard library that is distributed by rustup is compiled without debug assertions, so there is no easy way for users to benefit from all this extra checking.
+
+<!-- MORE -->
+
+[`cargo careful`](https://github.com/RalfJung/cargo-careful) is here to close this gap:
+when invoked the first time, it builds a standard library with debug assertions from source, and then runs your program or test suite with that standard library.
+Installing `cargo careful` is as easy as `cargo install cargo-careful`, and then you can do `cargo +nightly careful run`/`cargo +nightly careful test` to execute your binary crates and test suites with an extra amount of debug checking.
+
+This will naturally be slower than a regular debug or release build, but it is *much* faster than executing your program in [Miri](https://github.com/rust-lang/miri) and still helps find some Undefined Behavior.
+Unlike Miri, it is fully FFI-compatible (though the code behind the FFI barrier is completely unchecked).
+Of course Miri is much more thorough and `cargo careful` will miss many problems (for instance, it cannot detect out-of-bounds pointer arithmetic -- but it *does* perform bounds checking on `get_unchecked` slice accesses).
+
+Note that for now, some of these checks (in particular for raw pointer methods) cause an abrupt abort of the program via SIGILL without a nice error message or backtrace.
+There are probably ways to improve this in the future.
+Meanwhile, if you have some `unsafe` code that for one reason or another you cannot test with Miri, give [`cargo careful`] a try and let me know how it is doing. :)
+
+[`cargo careful`]: https://github.com/RalfJung/cargo-careful
+
+*By the way, I am soon [starting as a professor at ETH Zürich]({% post_url 2022-08-16-eth %}), so if you are interested in working with me on programming language theory as a master student, PhD student, or post-doc, then please [reach out](https://research.ralfj.de/contact.html)!*
--- /dev/null
+---
+title: "From Stacks to Trees: A new aliasing model for Rust"
+categories: rust research
+reddit: /rust/comments/13y8a9b/from_stacks_to_trees_a_new_aliasing_model_for_rust/
+---
+
+Since last fall, [Neven](https://perso.crans.org/vanille/) has been doing an internship to develop a new aliasing model for Rust: Tree Borrows.
+Hang on a second, I hear you say -- doesn't Rust already have an aliasing model?
+Isn't there this "Stacked Borrows" that Ralf keeps talking about?
+Indeed there is, but Stacked Borrows is just one proposal for a possible aliasing model -- and it [has its problems](https://github.com/rust-lang/unsafe-code-guidelines/issues?q=is%3Aopen+is%3Aissue+label%3AA-stacked-borrows).
+The purpose of Tree Borrows is to take the lessons learned from Stacked Borrows to build a new model with fewer issues, and to take some different design decisions such that we get an idea of some of the trade-offs and fine-tuning we might do with these models before deciding on the official model for Rust.
+
+Neven has written a detailed introduction to Tree Borrows [on his blog](https://perso.crans.org/vanille/treebor/), which you should go read first.
+He presented this talk at a recent RFMIG meeting, so you can also [watch his talk here](https://www.youtube.com/watch?v=zQ76zLXesxA).
+In this post, I will focus on the differences to Stacked Borrows.
+I assume you already know Stacked Borrows and want to understand what changes with Tree Borrows and why.
+
+<!-- MORE -->
+
+As a short-hand, I will sometimes write SB for Stacked Borrows and TB for Tree Borrows.
+
+## Two-phase borrows
+
+The main novelty in Tree Borrows is that it comes with proper support for two-phase borrows.
+Two-phase borrows are a mechanism introduced with NLL which allows code like the following to be accepted:
+
+```rust
+fn two_phase(mut x: Vec<usize>) {
+ x.push(x.len());
+}
+```
+
+The reason this code is tricky is that it desugars to something like this:
+
+```rust
+fn two_phase(mut x: Vec<usize>) {
+ let arg0 = &mut x;
+ let arg1 = Vec::len(&x);
+ Vec::push(arg0, arg1);
+}
+```
+
+This code clearly violates the regular borrow checking rules since `x` is mutably borrowed to `arg0` when we call `x.len()`!
+And yet, the compiler will accept this code.
+The way this works is that the `&mut x` stored in `arg0` is split into two phases:
+in the *reservation* phase, `x` can still be read via other references.
+Only when we actually need to write to `arg0` (or call a function that might write to it) will the reference be "activated", and it is from that point onwards (until the end of the lifetime of the borrow) that no access via other references is allowed.
+For more details, see [the RFC](https://github.com/rust-lang/rfcs/blob/master/text/2025-nested-method-calls.md) and [the rustc-dev-guide chapter on two-phase borrows](https://rustc-dev-guide.rust-lang.org/borrow_check/two_phase_borrows.html).
+The only point relevant for this blog post is that when borrowing happens implicitly for a method call (such as `x.push(...)`), Rust will treat this as a two-phase borrow.
+When you write `&mut` in your code, it is treated as a regular mutable reference without a "reservation" phase.
+
+For the aliasing model, two-phase borrows are a big problem: by the time `x.len()` gets executed, `arg0` already exists, and as a mutable reference it really isn't supposed to allow reads through other pointers.
+Therefore Stacked Borrows just [gives up](https://github.com/rust-lang/unsafe-code-guidelines/issues/85) here and basically treats two-phase borrows like raw pointers.
+That is of course unsatisfying, so for Tree Borrows we are adding proper support for two-phase borrows.
+What's more, we are treating *all* mutable references as two-phase borrows: this is more permissive than what the borrow checker accepts, but lets us treat mutable references entirely uniformly.
+(This is a point we might want to tweak, but as we will see soon this decision actually has some major unexpected benefits.)
+
+This is why we need a tree in the first place: `arg0` and the reference passed to `Vec::len` are both children of `x`.
+A stack is no longer sufficient to represent the parent-child relationships here.
+Once the use of a tree is established, modeling of two-phase borrows is fairly intuitive: they start out in a `Reserved` state which tolerates reads from other, unrelated pointers.
+Only when the reference (or one of its children) is written to for the first time, its state transitions to `Active` and now reads from other, unrelated pointers are not accepted any more.
+(See Neven's post for more details. In particular note that there is one unpleasant surprise lurking here: if there are `UnsafeCell` involved, then a reserved mutable reference actually has to tolerate *mutation* via unrelated pointers!
+In other words, the aliasing rules of `&mut T` are now affected by the presence of `UnsafeCell`. I don't think people realized this when two-phase borrows were introduced, but it also seems hard to avoid so even with hindsight, it is not clear what the alternative would have been.)
+
+## Delayed uniqueness of mutable references
+
+One of the most common source of Stacked Borrows issues is its [very eager enforcement of uniqueness of mutable references](https://github.com/rust-lang/unsafe-code-guidelines/issues/133).
+For example, the following code is illegal under Stacked Borrows:
+
+```rust
+let mut a = [0, 1];
+let from = a.as_ptr();
+let to = a.as_mut_ptr().add(1); // `from` gets invalidated here
+std::ptr::copy_nonoverlapping(from, to, 1);
+```
+
+The reason it is illegal is that `as_mut_ptr` takes `&mut self`, which asserts unique access to the entire array, therefore invalidating the previously created `from` pointer.
+In Tree Borrows, however, that `&mut self` is a two-phase borrow! `as_mut_ptr` does not actually perform any writes, so the reference remains reserved and never gets activated.
+That means the `from` pointer remains valid and the entire program is well-defined.
+The call to `as_mut_ptr` is treated like a read of `*self`, but `from` (and the shared reference it is derived from) are perfectly fine with reads via unrelated pointers.
+
+It happens to be the case that swapping the `from` and `to` lines actually makes this code work in Stacked Borrows.
+However, this is not for a good reason: this is a consequence of the rather not-stack-like rule in SB which says that on a read, we merely *disable all `Unique`* above the tag used for the access, but we keep raw pointers derived from those `Unique` pointers enabled.
+Basically, raw pointers can live longer than the mutable references they are derived from, which is highly non-intuitive and potentially problematic for program analyses.
+With TB, the swapped program is still fine, but for a different reason:
+when `to` gets created first, it remains a reserved two-phase borrow.
+This means that creating a shared reference and deriving `from` from it (which acts like a read on `self`) is fine; reserved two-phase borrows tolerate reads via unrelated pointers.
+Only when `to` is written to does it (or rather the `&mut self` it was created from) become an active mutable reference that requires uniqueness, but that is after `as_ptr` returns so there is no conflicting `&self` reference.
+
+It turns out that consistently using two-phase borrows lets us entirely eliminate this hacky SB rule and also fix one of the most common sources of UB under SB.
+I didn't expect this at all, so this is a happy little accident. :)
+
+However, note that the following program is fine under SB but invalid under TB:
+
+```rust
+let mut a = [0, 1];
+let to = a.as_mut_ptr().add(1);
+to.write(0);
+let from = a.as_ptr();
+std::ptr::copy_nonoverlapping(from, to, 1);
+```
+
+Here, the write to `to` activates the two-phase borrow, so uniqueness is enforced.
+That means the `&self` created for `as_ptr` (which is considered reading all of `self`) is incompatible with `to`, and so `to` is invalidated (well, it is made read-only) when `from` gets created.
+So far, we do not have evidence that this pattern is common in the wild.
+The way to avoid issues like the code above is to *set up all your raw pointers before you start doing anything*.
+Under TB, calling reference-receiving methods like `as_ptr` and `as_mut_ptr` and using the raw pointers they return on disjoint locations is fine even if these references overlap, but you must call all those methods before the first write to a raw pointer.
+Once the first write happens, creating more references can cause aliasing violations.
+
+## No strict confinement of the accessible memory range
+
+The other major source of trouble with Stacked Borrows is [restricting raw pointers to the type and mutability they are initially created with](https://github.com/rust-lang/unsafe-code-guidelines/issues/134).
+Under SB, when a reference is cast to `*mut T`, the resulting raw pointer is confined to access only the memory covered by `T`.
+This regularly trips people up when they take a raw pointer to one element of an array (or one field of a struct) and then use pointer arithmetic to access neighboring elements.
+Moreover, when a reference is cast to `*const T`, it is actually read-only, even if the reference was mutable!
+Many people expect `*const` vs `*mut` not to matter for aliasing, so this is a regular source of confusion.
+
+Under TB, we resolve this by no longer doing any retagging for reference-to-raw-pointer casts.
+A raw pointer simply uses the same tag as the parent reference it is derived from, thereby inheriting its mutability and the range of addresses it can access.
+Moreover, references are not strictly confined to the memory range described by their type:
+when an `&mut T` (or `&T`) gets created from a parent pointer, we initially record the new reference to be allowed to access the memory range describe by `T` (and we consider this a read access for that memory range).
+However, we also perform *lazy initialization*: when a memory location outside this initial range is accessed, we check if the parent pointer would have had access to that location, and if so then we also give the child the same access.
+This is repeated recursively until we find a parent that has sufficient access, or we reach the root of the tree.
+
+This means TB is compatible with [`container_of`-style pointer arithmetic](https://github.com/rust-lang/unsafe-code-guidelines/issues/243) and [`extern` types](https://github.com/rust-lang/unsafe-code-guidelines/issues/276), overcoming two more SB limitations.
+
+This also means that the following code becomes legal under TB:
+
+```rust
+let mut x = 0;
+let ptr = std::ptr::addr_of_mut!(x);
+x = 1;
+ptr.read();
+```
+
+Under SB, `ptr` and direct access to the local `x` used two different tags, so writing to the local invalidated all pointers to it.
+Under TB, this is no longer the case; a raw pointer directly created to the local is allowed to alias arbitrarily with direct accesses to the local.
+
+Arguably the TB behavior is more intuitive, but it means we can no longer use writes to local variables as a signal that all possible aliases have been invalidated.
+However, note that TB only allows this if there is an `addr_of_mut` (or `addr_of`) immediately in the body of a function!
+If a reference `&mut x` is created, and then some other function derives a raw pointer from that, those raw pointers *do* get invalidated on the next write to `x`.
+So to me this is a perfect compromise: code that uses raw pointers has a lower risk of UB, but code that does not use raw pointers (which is easy to see syntactically) can be optimized as much as with SB.
+
+Note that this entire approach in TB relies on TB *not* needing the stack-violating hack mentioned in the previous section.
+If raw pointers in SB just inherited their parent tag, then they would get invalidated together with the unique pointer they are derived from, disallowing all the code that this hack was specifically added to support.
+This means that backporting these improvements to SB is unlikely to be possible.
+
+## `UnsafeCell`
+
+The handling of `UnsafeCell` also changed quite a bit with TB.
+First of all, another [major issue](https://github.com/rust-lang/unsafe-code-guidelines/issues/303) with SB was fixed: turning an `&i32` into an `&Cell<i32>` *and then never writing to it* is finally allowed.
+This falls out of how TB handles the aliasing allowed with `UnsafeCell`: they are treated like casts to raw pointers, so reborrowing an `&Cell<i32>` just inherits the tag (and therefore the permissions) of the parent pointer.
+
+More controversially, TB also changes how precisely things become read-only when an `&T` involves `UnsafeCell` somewhere inside `T`.
+In particular, for `&(i32, Cell<i32>)`, TB allows mutating *both* fields, including the first field which is a regular `i32`, since it just treats the entire reference as "this allows aliasing".[^1]
+In contrast, SB actually figured out that the first 4 bytes are read-only and only the last 4 bytes allow mutation via aliased pointers.
+
+[^1]: This does not mean that we bless such mutation! It just means that the compiler cannot use immutability of the first field for its optimizations. Basically, immutability of that field becomes a [safety invariant instead of a validity invariant]({% post_url 2018-08-22-two-kinds-of-invariants %}): when you call foreign code, you can still rely on it not mutating that field, but within the privacy of your own code you are allowed to mutate it. See [my response here](https://www.reddit.com/r/rust/comments/13y8a9b/comment/jmlvgun/) for some more background.
+
+The reason for this design decision is that the general philosophy with TB was to err on the side of allowing more code, having less UB (which is the opposite direction than what I used with SB).
+This is a deliberate choice to uncover as much of the design space as we can with these two models.
+Of course we wanted to make sure that TB still allows all the desired optimizations, and still has enough UB to justify the LLVM IR that rustc generates -- those were our "lower bounds" for the minimum amount of UB we need.
+And it turns out that under these constraints, we can support `UnsafeCell` with a fairly simple approach: for the aliasing rules of `&T`, there are only 2 cases.
+Either there is no `UnsafeCell` anywhere, then this reference is read-only, or else the reference allows aliasing.
+As someone who thinks a lot about proving theorems about the full Rust semantics including its aliasing model, this approach seemed pleasingly simple. :)
+
+I expected this decision to be somewhat controversial, but the amount of pushback we received has still been surprising.
+The good news is that this is far from set in stone: we can [change TB to treat `UnsafeCell` more like SB did](https://github.com/rust-lang/unsafe-code-guidelines/issues/403).
+Unlike the previously described differences, this one is entirely independent of our other design choices.
+While I prefer the TB approach, the way things currently stand, I do expect that we will end up with SB-like `UnsafeCell` treatment eventually.
+
+## What about optimizations?
+
+I have written a lot about how TB differs from SB in terms of which coding patterns are UB.
+But what about the other side of the coin, the optimizations?
+Clearly, since SB has more UB, we have to expect TB to allow fewer optimizations.
+And indeed there is a major class of optimizations that TB loses: speculative writes, i.e. inserting writes in code paths that would not previously have written to this location.
+This is a powerful optimization and I was quite happy that SB could pull it off, but it also comes at a major cost: mutable references have to be "immediately unique".
+Given how common of a problem "overeager uniqueness" is, my current inclination is that we most likely would rather make all that code legal than allow speculative writes.
+We still have extremely powerful optimization principles around reads, and when the code *does* perform a write that gives rise to even more optimizations, so my feeling is that insisting on speculative writes is just pushing things too far.
+
+On another front, TB actually allows a set of crucial optimizations that SB ruled out by accident: reordering of reads!
+The issue with SB is that if we start with "read mutable reference, then read shared reference", and then reorder to "read shared reference, then read mutable reference", then in the new program, reading the shared reference might invalidate the mutable reference -- so the reordering might have introduced UB!
+This optimization is possible without having any special aliasing model, so SB not allowing it is a rather embarrassing problem.
+If it weren't for the stack-violating hack that already came up several times above, I think there would be a fairly easy way of fixing this problem in SB, but alas, that hack is load-bearing and too much existing code is UB if we remove it.
+Meanwhile, TB does not need any such hack, so we can do the Right Thing (TM): when doing a read, unrelated mutable references are not entirely disabled, they are just made read-only.
+This means that "read shared reference, then read mutable reference" is equivalent to "read mutable reference, then read shared reference" and the optimization is saved.
+(A consequence of this is that retags can also be reordered with each other, since they also act as reads. Hence the order in which you set up various pointers cannot matter, until you do the first write access with one of them.)
+
+## Future possibility: `Unique`
+
+Tree Borrows paves the way for an extension that we have not yet implemented, but that I am quite excited to explore: giving meaning to `Unique`.
+`Unique` is a private type in the Rust standard library that was originally meant to express `noalias` requirements.
+However, it was never actually wired up to emit that attribute on the LLVM level.
+`Unique` is mainly used in two places in the standard library: `Box` and `Vec`.
+SB (and TB) treat `Box` special (matching rustc itself), but not `Unique`, so `Vec` does not come with any aliasing requirements.
+And indeed the SB approach totally does not work for `Vec`, since we don't actually know how much memory to make unique here.
+However, with TB we have lazy initialization, so we don't need to commit to a memory range upfront -- we can make it unique "when accessed".
+This means we can explore giving meaning to the `Unique` in `Vec`.
+
+Now, this might not actually work.
+People actually do blatantly-aliasing things with `Vec`, e.g. to implement arenas.
+On the other hand, `Vec`'s uniqueness would only come in when it is moved or passed *by value*, and only for the memory ranges that are actually being accessed.
+So it is quite possible that this is compatible with arenas.
+I think the best way to find out is to implement `Unique` semantics behind a flag and experiment.
+If that works out, we might even be able to remove all special handling of `Box` and rely on the fact that `Box` is defined as a newtype over `Unique`.
+This would slightly reduce the optimization potential (`Box<T>` is known to point to a memory range at least the size of `T`, whereas `Unique` has no such information), but making `Box` less magic is a long-standing quest so this might be an acceptable trade-off.
+
+I should note that there are many people who think neither `Box` nor `Vec` should have any aliasing requirements. I think it's worth first exploring whether we can have aliasing requirements which are sufficiently light-weight that they are compatible with common coding patterns, but even if we end up saying `Box` and `Vec` behave like raw pointers, it can still be useful to have `Unique` in our toolbox and expose it for unsafe code authors to eke out the last bits of performance.
+
+## Conclusion
+
+These are the major differences between Stacked Borrows and Tree Borrows.
+As you can see, almost all of them are cases where TB allows more code than SB, and indeed TB fixes what I consider to be SB's two biggest problems: overeager uniqueness for mutable references, and confining references and raw pointers to the size of the type they are created with.
+These are great news for unsafe code authors!
+
+What TB *doesn't* change is the presence of "protectors" to enforce that certain references remain valid for the duration of an entire function call (whether they are used again or not); protectors are absolutely required to justify the LLVM `noalias` annotations we would like to emit and they also do enable some stronger optimizations than what would otherwise be possible.
+I do expect protectors to be the main remaining source of unexpected UB from Tree Borrows, and I don't think there is a lot of wiggle-room that we have here, so this might just be a case where we have to tell programmers to adjust their code, and invest in documentation material to make this subtle issue as widely known as possible.
+
+Neven has implemented Tree Borrows in Miri, so you can play around with it and check your own code by setting `MIRIFLAGS=-Zmiri-tree-borrows`.
+If you run into any surprises or concerns, please let us know!
+The [t-opsem Zulip](https://rust-lang.zulipchat.com/#narrow/stream/136281-t-opsem) and the [UCG issue tracker](https://github.com/rust-lang/unsafe-code-guidelines/) are good places for such questions.
+
+That's all I got, thanks for reading -- and a shout out to Neven for doing all the actual work here (and for giving feedback on this blog post), supervising this project has been a lot of fun!
+Remember to read [his write up](https://perso.crans.org/vanille/treebor/) and [watch his talk](https://www.youtube.com/watch?v=zQ76zLXesxA).
--- /dev/null
+---
+title: "Talk about Undefined Behavior, unsafe Rust, and Miri"
+categories: rust
+---
+
+I recently gave a talk at a local Rust meetup in Zürich about Undefined Behavior, unsafe Rust, and Miri.
+The recording is available [here](https://www.youtube.com/watch?v=svR0p6fSUYY).
+It targets an audience that is familiar with Rust but not with the nasty details of unsafe code, so I hope many of you will enjoy it!
+Have fun. :)
--- /dev/null
+---
+title: "Google Open Source Peer Bonus"
+categories: rust
+---
+
+We are all used to spam emails, supposedly from Google, that say "You won" and I just need to send all my data to somewhere to receive my lottery payout.
+When I recently received an email about Google's "Open Source Peer Bonus" program, I almost discarded it as yet another version of that kind of spam.
+But it turns out sometimes these emails are real!
+Meanwhile the [official announcement](https://opensource.googleblog.com/2023/12/google-open-source-peer-bonus-program-announces-second-group-of-2023-winners.html) has been released which lists me as a recipient of this bonus as a thank you for my work on Rust.
+So this one time, it wasn't spam after all!
+
+<!-- MORE -->
+
+Thanks a lot to Google for this program at the $250 reward; it is great to see open source work honored this way.
+I have donated the amount in full to [noyb](https://noyb.eu/en), who I'm sure will be using it [for good](https://noyb.eu/en/noyb-win-first-major-fine-eu-1-million-using-google-analytics).
+
+**Update (2024-01-07):**
+In fact, this is already my second Google Open Source Peer Bonus.
+The first was in the [first half of 2023](https://opensource.googleblog.com/2023/05/google-open-source-peer-bonus-program-announces-first-group-of-winners-2023.html).
+Due to issues with the payment process, it took a while for that bonus to be transferred, but I can confirm that it has now arrived in my bank account.
+I will have to find a suitable non-for-profit to donate this to... or it might be noyb again, we will see.
+**/Update**
--- /dev/null
+---
+title: "Sandboxing All The Things with Flatpak and BubbleBox"
+categories: sysadmin
+---
+
+A few years ago, I have [blogged]({% post_url 2019-03-09-firejail %}) about my approach to sandboxing less-trusted applications that I have to or want to run on my main machine.
+The approach has changed since then, so it is time for an update.
+
+<!-- MORE -->
+
+Over time I grew increasingly frustrated with Firejail: configurations would frequently break on updates,
+and debugging Firejail profiles is extremely hard. When considering all the included files, we are talking
+about many hundred lines of configuration with a subtle interplay of allowlists and blocklists.
+Even when I knew which folder I wanted to give access to, it was often non-trivial to ensure that
+this access would actually be possible.
+
+Now I am instead using a combination of two different approaches: Flatpak and BubbleBox.
+
+## Flatpak
+
+The easiest sandbox to maintain is the sandbox maintained by someone else.
+So when a Flatpak exists for software I want to or have to use, such as Signal or Zoom, that is generally my preferred approach.
+
+Unfortunately, Flatpaks can come with extremely liberal default profiles that make the sandbox mostly pointless.
+The following global overrides help ensure that this does not happen:
+```
+[Context]
+sockets=!gpg-agent;!pcsc;!ssh-auth;!system-bus;!session-bus
+filesystems=~/.XCompose:ro;xdg-config/fontconfig:ro;!~/.gnupg;!~/.ssh;!xdg-documents;!home;!host
+
+[Session Bus Policy]
+org.freedesktop.Flatpak=none
+org.freedesktop.secrets=none
+```
+
+I also use [Flatseal], an amazing application that helps to check which permissions applications get, and change them if necessary.
+
+[Flatseal]: https://flathub.org/apps/com.github.tchx84.Flatseal
+
+## BubbleBox
+
+However, not all software exists as Flatpak.
+Also, sometimes I want software to run basically on my host system (i.e., to use the regular `/usr`), just without access to literally *everything* in my home directory.
+Examples of this are Factorio and VSCodium.
+The latter doesn't work in Flatpak as I want to use it with LaTeX, and realistically this means it needs to run the LaTeX installed via `apt`.
+The official recommendation is to effectively disable the Flatpak sandbox, but that entirely defeats the point, so I went looking for alternatives.
+
+[bubblewrap] provides a very convenient solution: it can start an application in its own private filesystem namespace with full control over which part of the host file system is accessible from inside the sandbox.
+I wrote a small wrapper around bubblewrap to make this configuration a bit more convenient to write and manage;
+this project is called [BubbleBox].
+This week-end I finally got around to adding support for [xdg-dbus-proxy] so that sandboxed applications can now access particular D-Bus functions without having access to the entire bus (which is in general not safe to expose to a sandboxed application).
+That means it's finally time to blog about this project, so here we go -- if you are interested, check out [BubbleBox];
+the project page explains how you can use it to set up your own sandboxing.[^1]
+
+[^1]: One day I should probably rewrite this in Rust... maybe this will be my test project for when [cargo-script](https://rust-lang.github.io/rfcs/3424-cargo-script.html) becomes available.
+
+I should also note that this is not the only bubblewrap-based sandboxing solution.
+[bubblejail] is fairly similar but provides a configuration GUI and a good set of default provides;
+it was a very useful resource when figuring out the right bubblewrap flags to make complex GUI applications work properly.
+(Incidentally, "bubblejail" is also how I called my own script originally, but then I realized that the name is already taken.)
+Joachim Breitner also recently [blogged](https://www.joachim-breitner.de/blog/812-Convenient_sandboxed_development_environment) about his own bubblewrap-based sandboxing script.
+sloonz has a similar [script](https://gist.github.com/sloonz/4b7f5f575a96b6fe338534dbc2480a5d) as well, with a nice yaml-based configuration format and [great explanations](https://sloonz.github.io/posts/sandboxing-1/) for what all the flags exactly do.
+Had their script existed when I started what eventually became BubbleBox, I would have used it as a starting point.
+But it was also fun to figure out my own solution.
+
+Using bubblewrap and xdg-dbus-proxy for this was an absolute joy.
+Both of these components came out of the Flatpak project, but the authors realized that they could be independently useful,
+so in best Unix tradition they turned them into tools that provide all the required mechanism without hard-coding any sort of policy.
+Despite doing highly non-trivial tasks, they are both pretty easy to use and compose and very well-documented.
+Thanks a lot to everyone involved!
+
+[bubblewrap]: https://github.com/containers/bubblewrap
+[BubbleBox]: {{ site.baseurl }}/projects/bubblebox
+[xdg-dbus-proxy]: https://github.com/flatpak/xdg-dbus-proxy
+[bubblejail]: https://github.com/igo95862/bubblejail
.comment {
margin-top: 1.5em;
}
+
+table td, table th {
+ border: 1px solid $light-text-color;
+ padding: 6px 13px;
+ font-weight: normal;
+}
margin-top: 0.8em;
}
}
+
+/* Images (~350px wide) floating to the right but only if there is enough space */
+.float-right-350 {
+ float: right;
+ margin-left: 0.8em;
+ margin-bottom: 0.3em;
+}
+@media screen and (max-width:600px) {
+ .float-right-350 {
+ max-width: 300px;
+ float: none;
+ margin: auto;
+ }
+}
---
layout: feed
-slug: Subscribe
+slug: RSS Feed
---
<title>{{ site.blog.title | xml_escape }}</title>
<div>
<h2>Welcome</h2>
<p>
-Welcome at ralfj.de, my server for experiments and digital independence.
-On this machine I maintain several services which are part of modern life, and which I would rather not entrust to data krakens in some cloud (what if it's raining?):
+Welcome at ralfj.de, my server for experiments and digital self-sufficiency.
+On this machine I maintain several services which are part of modern life, and which I would rather not entrust to data krakens and the cloud (what if it's raining?):
</p>
<ul>
<li><a href="mailto:post-AT-ralfj-DOT-de">E-Mail</a>,</li>
- <li>Jabber
: My Jabber-ID is <a href="xmpp:ralf-AT-jabber-DOT-ralfj-DOT-de">ralf-AT-jabber-DOT-ralfj-DOT-de</a> (also see <a href="https://en.wikipedia.org/wiki/Extensible_Messaging_and_Presence_Protocol" target="_blank">the Wikipedia article about Jabber</a>),</li>
+ <li>Jabber (also see <a href="https://en.wikipedia.org/wiki/Extensible_Messaging_and_Presence_Protocol" target="_blank">the Wikipedia article about Jabber</a>),</li>
<li><a href="https://git.ralfj.de/">Git repositories</a>,</li>
<li><a href="https://lists.ralfj.de/">Mailing lists</a>,</li>
<li>a DNS server (incl. DynDNS and DNSSEC),</li>
<li>and more.</li>
</ul>
<p>
-All software running on my server is of course <a href="https://en.wikipedia.org/wiki/Free_Software" target="_blank">free</a>.
-This is the only way for me to expect that it is not actually acting against my interests.
-Besides, all communication (if at all possible) is encrypted. This self-defense is seemingly necessary today to achieve freedom and basic rights (like privacy of telecommunications) in the digital world.<br>
-Speaking of which, my GPG key has the ID 0x1B24F3FF. You can find it <a href="0x1B24F3FF.asc">here</a> or on any <a href="https://sks-keyservers.net/" target="_blank">keyserver of the SKS network</a>.
+All software running on my server is of course <a href="https://en.wikipedia.org/wiki/Free_Software" target="_blank">free</a> (as in freedom, not just free beer).
+That way, I can expect it to not act against my interests.
+Besides, all communication (if at all possible) is encrypted. This self-defense is seemingly necessary today to achieve basic civil rights (like privacy of telecommunications) in the digital world.<br>
+Speaking of which, my GPG key has the ID 0x0CB18E521B24F3FF. You can find it <a href="0x0CB18E521B24F3FF.asc">here</a> or on <a href="https://keys.openpgp.org/vks/v1/by-fingerprint/BD3F5AF6D95D77C48A85AF680CB18E521B24F3FF" target="_blank">keys.openpgp.org</a>.
</p>
<p>
You may wonder why I spend so much time on this. Well, I already mentioned independence of the usual internet giants.
<div lang="de">
<h2>Willkommen</h2>
<p>
-Willkommen auf ralfj.de, meinem Server für Experimente und digitale Unabhängigkeit.
+Willkommen auf ralfj.de, meinem Server für Experimente und digitale Eigenversorgung.
Auf dieser Maschine laufen diverse Dienste, die zum modernen Alltag gehören, und die ich ungern Datenkraken in irgendwelchen Wolken anvertrauen möchte (was, wenn es regnet?):
</p>
<ul>
<li><a href="mailto:post-ÄT-ralfj-PUNKT-de">E-Mail</a>,</li>
- <li>Jabber
: Meine Jabber-ID lautet <a href="xmpp:ralf-ÄT-jabber-PUNKT-ralfj-PUNKT-de">ralf-ÄT-jabber-PUNKT-ralfj-PUNKT-de</a> (siehe auch <a href="https://de.wikipedia.org/wiki/Extensible_Messaging_and_Presence_Protocol" target="_blank">den Wikipedia-Artikel zu Jabber</a>),</li>
+ <li>Jabber (siehe auch <a href="https://de.wikipedia.org/wiki/Extensible_Messaging_and_Presence_Protocol" target="_blank">den Wikipedia-Artikel zu Jabber</a>),</li>
<li><a href="https://git.ralfj.de/">Git-Repositories</a>,</li>
<li><a href="https://lists.ralfj.de/">Mailing-Listen</a>,</li>
<li>ein DNS-Server (inkl. DynDNS und DNSSEC),</li>
<p>
Alle Software, die auf meinem Server läuft, ist selbstverständlich <a href="https://de.wikipedia.org/wiki/Freie_Software" target="_blank">frei</a>.
Nur so kann ich davon ausgehen, dass sie nicht gegen mich bzw. meine Interessen handelt.
-Zudem läuft alle Kommunikation (soweit irgend möglich) verschlüsselt ab. Diese Art von Selbstverteidigung ist heutzutage offenbar nötig, um verbriefte Rechte (wie das Postgeheimnis) und unsere Freiheit auch im Internet durchzusetzen.<br>
-Mein GPG-Schlüssel hat übrigens die ID 0x1B24F3FF. Du kannst ihn <a href="0x1B24F3FF.asc">hier</a> oder von einem beliebigen <a href="https://sks-keyservers.net/" target="_blank">Schlüsselserver des SKS-Netzes</a> herunterladen.
+Zudem läuft alle Kommunikation (soweit irgend möglich) verschlüsselt ab. Diese Art von Selbstverteidigung ist heutzutage offenbar nötig, um verbriefte Rechte (wie das Postgeheimnis) auch im Internet durchzusetzen.<br>
+Mein GPG-Schlüssel hat übrigens die ID 0x0CB18E521B24F3FF. Du kannst ihn <a href="0x0CB18E521B24F3FF.asc">hier</a> oder von <a href="https://keys.openpgp.org/vks/v1/by-fingerprint/BD3F5AF6D95D77C48A85AF680CB18E521B24F3FF" target="_blank">keys.openpgp.org</a> herunterladen.
</p>
<p>
Warum ich mir diese Mühe mache? Nun, die Unabhängigkeit von den üblichen Internetgiganten habe ich ja schon erwähnt.
The tools were usually not written with general re-usability in mind.
But maybe you are having just the same problem as I did, in which case I hope they can be helpful.
-* [LiLaSS](lilass/): A simple xrandr-based application to configure laptop screens on Linux. If you are using a
+* [BubbleBox](bubblebox): A simple script to sandbox Linux applications.
+* [dyn-nsupdate](dyn-nsupdate): A tool to dynamically and securely update DNS zones via CGI.
+ This provides self-hosted DynDNS services.
+* [git-mirror](git-mirror): This can keep multiple git repositories of the same project in sync automatically.
+* [LiLaSS](lilass): A simple xrandr-based application to configure laptop screens on Linux. If you are using a
Laptop, frequently work both with and without an external screen, and you are not happy with
the configuration options your desktop provides, this may be for you.
-* [dyn-nsupdate](dyn-nsupdate/): A tool to dynamically and securely update DNS zones via CGI.
- This provides self-hosted DynDNS services.
-* [zonemaker](zonemaker/): A small script to generate DNS zone files from Python.
-* [schsh](schsh/): A collection of scripts and configuration files which can be used to grant
+* [Rust-101](rust-101): A small tutorial for the [Rust language](https://www.rust-lang.org).
+* [schsh](schsh): A collection of scripts and configuration files which can be used to grant
someone secure (SSH-based) access to a machine, without giving them a shell or read access
to the entire file system.
-* [Rust-101](rust-101/): A small tutorial for the [Rust language](https://www.rust-lang.org).
-* [git-mirror](git-mirror/): This can keep multiple git repositories of the same project in sync automatically.
+* [zonemaker](zonemaker): A small script to generate DNS zone files from Python.
For some more of my projects, check out the [public git repositories](https://www.ralfj.de/git/)
hosted on my server and my [GitHub profile](https://github.com/RalfJung/).
+++ /dev/null
----
-title: "Exponential blowup when using typeclasses with nested indices"
-categories: research coq
----
-
-When formalizing a proof in an interactive theorem prover like [Coq](https://coq.inria.fr/), one reoccuring issue is handling algebraic hierarchies.
-Such hierarchies are everywhere: Some operations are associative, while others also commute; some types have an equivalence relation, while others also have a (pre-)order or maybe even a well-ordering; and so on.
-So the question arises: What is the best way to actually encode these hierarchies in Coq?
-Coq offers two mechanism that are suited to solve this task: typeclasses and canonical structures.
-Both can be instrumented in different ways to obtain a (more or less) convenient-to-use algebraic hierarchy.
-A common approach using typeclasses is the ["unbundled" approach by Bas Spitters and Eelis van der Weegen](https://arxiv.org/abs/1102.1323).
-However, as [has been observed before](https://pastel.archives-ouvertes.fr/pastel-00649586), and as learned the hard way in the Coq formalization of the [original Iris paper](https://iris-project.org/pdfs/2015-popl-iris1-final.pdf), this approach quickly leads to terms that seem to be exponential in size.
-
# Site settings
-baseurl: "/~jung"
-url: "https://www.mpi-sws.org"
+baseurl: ""
+url: "https://research.ralfj.de"
timezone: "Europe/Berlin"
defaults:
exclude: ['*.sh']
# Custom settings
-mpi_sws_imprint: True
+mpi_sws_imprint: False
-../ralf/_includes/
\ No newline at end of file
+../personal/_includes
\ No newline at end of file
-../ralf/_layouts/
\ No newline at end of file
+../personal/_layouts
\ No newline at end of file
-../ralf/_plugins/
\ No newline at end of file
+../personal/_plugins
\ No newline at end of file
-../ralf/_sass/
\ No newline at end of file
+../personal/_sass
\ No newline at end of file
sort: 2
---
-<h3>E-Mail Address</h3>
+<p><b>Email:</b> ralf DOT jung AT inf DOT ethz DOT ch</p>
-<p>jung AT mpi-sws DOT org</p>
+<p><b>Phone:</b> +41 44 632 5598</p>
-<h3>Phone Number</h3>
-
-<p>+49 (681) 9303 8717</p>
-
-<h3>Work Address</h3>
-
-<p>
-Room 438<br/>
-MPI-SWS (Campus E1.5)<br/>
-66123 Saarbrücken<br/>
-Germany
+<p><b>Office / Mail:</b><br/>
+<a href='https://ethz.ch/staffnet/de/service/raeume-gebaeude/orientierung/gebaeude.html?args0=CNB' target='_blank'>CNB</a> H 109<br/>
+Universitätstrasse 6<br/>
+8092 Zürich<br/>
+Switzerland
</p>
title: Ralf Jung
---
-<div style="float:right; margin-left:0.8em; margin-bottom: 0.3em"><img src="me.jpg"></div>
+<div class="float-right-350"><img style="max-width: 350px; width: 100%;" src="me.jpg" srcset="me.jpg, me-2x.jpg 2x"></div>
-<p>I am a PhD student at <a href="https://www.mpi-sws.org/" target="_blank">Max Planck Institute for Software Systems (MPI-SWS)</a> and <a href="https://www.uni-saarland.de/" target="_blank">Saarland University</a> under the supervision of <a href="https://www.mpi-sws.org/~dreyer/" target="_blank">Derek Dreyer</a>, head of the <a href="https://plv.mpi-sws.org/" target="_blank">Foundations of Programming group</a>.</p>
+<p>I am assistant professor at <a href='https://ethz.ch' target="_blank">ETH Zürich</a>, leading the <a href="https://plf.inf.ethz.ch/" target="_blank">Programming Language Foundations Lab</a>.
+We are part of the <a href='https://pls.inf.ethz.ch/' target="_blank">Institute for Programming Languages and Systems</a>.</br>
+Previously, I completed my <a href="thesis.html">PhD</a> at <a href="https://www.mpi-sws.org/">MPI-SWS</a> and <a href="https://www.uni-saarland.de/" target="_blank">Saarland University</a> in Saarbrücken, Germany; my advisor was <a href="https://people.mpi-sws.org/~dreyer/">Derek Dreyer</a>.
+I also did a post-doc in the <a href="https://pdos.csail.mit.edu/">PDOS group</a> at <a href="https://www.csail.mit.edu/" target="_blank">MIT CSAIL</a>.
+</p>
-<p>Previously, I did my Bachelor's thesis in computer science at the <a href="http://www.cdl.uni-saarland.de/index.php">Compiler Design chair</a> of the university (also see below).</p>
+<!-- <p><i>I am offering up to two fully funded PhD positions in my newly founded research group at <a href="https://ethz.ch/en/the-eth-zurich/working-teaching-and-research.html">ETH Zürich</a>, with flexible start date.
+I am looking for strong students that want to do research at the foundations of programming language theory, in program verification and separation logic, with a focus on Rust and Iris.
+Knowledge of Coq is greatly appreciated. Interested candidates can <a href="contact.html">contact me directly</a>.
+Please explain why you are interested in a PhD in this field and what your prior experience is. Also include a CV and possible contacts for recommendation letters.
+Note that doing a PhD at ETH Zürich generally requires a Master's degree, but there is a <a href="https://inf.ethz.ch/doctorate/direct-doctorate-computer-science.html">direct doctorate program</a> that you can enter with a Bachelor's degree (application deadline December 15th).</i></p> -->
-<p>I am currently working on giving a formal model to <a href="https://www.rust-lang.org/">Rust's</a> type system.
-This work is part of the <a href="https://plv.mpi-sws.org/rustbelt/">RustBelt</a> project. <br/>
-The Rust work builds on my previous work on a logic to support modular reasoning about higher-order concurrent imperative programs. The focus there was on providing simple building blocks that are powerful enough to recover more sophisticated reasoning techniques that were often axiomatized in previous logics. <br/>
-For some more information, check out my <a href="https://www.ralfj.de/blog/categories/research.html">research blog</a> and my <a href="research-statement.pdf">research statement</a>.</p>
+<p>My two main lines of work are about <a href="https://www.rust-lang.org/">Rust</a> and <a href="https://iris-project.org/">Iris</a>.<br>
+On the Rust side, me and my group are working (also in collaboration with the Rust language team) towards a solid formal foundation for the language, including in particular the unsafe parts.
+As part of this we are developing <a href="https://github.com/rust-lang/miri/">Miri</a>, a practical tool for detecting Undefined Behavior bugs in unsafe Rust code, which has become a part of the standard toolbox of unsafe code authors.
+Meanwhile, <a href="https://github.com/minirust/minirust">MiniRust</a> is our work-in-progress proposal for a precise specification of unsafe Rust, that I hope to integrate into an official Rust specification eventually.
+My long-term goal is to make unsafe Rust just as safe as safe Rust by means of formal verification based on rigorous foundations for all key components of the language.<br>
+On the Iris side, I am continuing development of its logical foundations.
+We are making Iris fit for specifying and verifying programming languages at scale using a more modular approach.
+The long-term goal is for Iris to be able to handle the full scale of complexities that arise when doing foundational verification of real languages.<br>
+For some more information, check out my <a href="https://www.ralfj.de/blog/categories/research.html">research blog</a>, my <a href="cv.pdf">CV</a>, and my <a href="research-statement.pdf">research statement</a>.</p>
<p>In my free time, I like to run internet services myself and work on free software.
This goes hand-in-hand with my pursuit of defending our privacy rights and our freedom in the digital world.
slug: Publications
---
-<!-- <h3>Under Submission</h3> -->
+<h2>Conference and journal papers</h2>
+
+<h3>2023</h3>
+
+<ul><li>
+ <b>Grove: a Separation-Logic Library for Verifying Distributed Systems</b><br/>
+ Upamanyu Sharma, Ralf Jung, Joseph Tassarotti, M. Frans Kaashoek, Nickolai Zeldovich <br>
+ <i>In <a href="https://sosp2023.mpi-sws.org/index.html">SOSP 2023</a></i><br>
+ [<a href="https://iris-project.org/pdfs/2023-sosp-grove.pdf">paper</a>] [<a href="https://arxiv.org/abs/2309.03046">extended version</a>]
+</li></ul>
+
+<ul><li>
+ <b>Verifying vMVCC, a high-performance transaction library using multi-version concurrency control</b><br/>
+ Yun-Sheng Chang, Ralf Jung, Upamanyu Sharma, Joseph Tassarotti, M. Frans Kaashoek, Nickolai Zeldovich <br>
+ <i>In <a href="https://www.usenix.org/conference/osdi23">OSDI 2023</a></i><br>
+ [<a href="https://pdos.csail.mit.edu/papers/vmvcc:osdi23.pdf">paper</a>] [<a href="https://pdos.csail.mit.edu/projects/vmvcc.html#vmvcc">paper website</a> (incl. Coq formalization)]
+</li></ul>
+
+<h3>2022</h3>
+
+<ul><li>
+ <b>Later Credits: Resourceful Reasoning for the Later Modality</b><br/>
+ Simon Spies, Lennard Gäher, Joseph Tassarotti, Ralf Jung, Robbert Krebbers, Lars Birkedal, Derek Dreyer <br>
+ <i>In <a href="https://icfp22.sigplan.org/">ICFP 2022</a></i><br>
+ [<a href="https://plv.mpi-sws.org/later-credits/paper-later-credits.pdf">paper</a>] [<a href="https://plv.mpi-sws.org/later-credits/">paper website</a> (incl. Coq formalization)]
+</li></ul>
+
+<ul><li>
+ <b>Simuliris: A separation logic framework for verifying concurrent program optimizations</b><br/>
+ Lennard Gäher, Michael Sammler, Simon Spies, Ralf Jung, Hoang-Hai Dang, Robbert Krebbers, Jeehoon Kang, Derek Dreyer<br>
+ <i>In <a href="https://popl22.sigplan.org/">POPL 2022</a>, <b>Distinguished Paper Award</b></i><br>
+ [<a href="https://iris-project.org/pdfs/2022-popl-simuliris.pdf">paper</a>]
+</li></ul>
+
+<h3>2021</h3>
+
+<ul><li>
+ <b>GhostCell: Separating Permissions from Data in Rust</b><br/>
+ Joshua Yanovski, Hoang-Hai Dang, Ralf Jung, Derek Dreyer<br>
+ <i>In <a href="https://icfp21.sigplan.org/">ICFP 2021</a></i><br>
+ [<a href="https://plv.mpi-sws.org/rustbelt/ghostcell/paper.pdf">paper</a>] [<a href="https://plv.mpi-sws.org/rustbelt/ghostcell/">paper website</a> (incl. Coq formalization)]
+</li></ul>
+
+<ul><li>
+ <b>GoJournal: a verified, concurrent, crash-safe journaling system</b><br/>
+ Tej Chajed, Joseph Tassarotti, Mark Theng, Ralf Jung, M. Frans Kaashoek, Nickolai Zeldovich<br>
+ <i>In <a href="https://www.usenix.org/conference/osdi21">OSDI 2021</a></i><br>
+ [<a href="perennial/gojournal.pdf">paper</a>]
+</li></ul>
+
+<ul><li>
+ <b>Safe Systems Programming in Rust: The Promise and the Challenge</b><br/>
+ Ralf Jung, Jacques-Henri Jourdan, Robbert Krebbers, Derek Dreyer<br>
+ <i>In Communications of the ACM (CACM)</i><br>
+ [<a href="https://cacm.acm.org/magazines/2021/4/251364-safe-systems-programming-in-rust/fulltext">article</a>]
+ [<a href="https://iris-project.org/pdfs/2021-rustbelt-cacm-final.pdf">pdf</a>]
+ [<a href="https://vimeo.com/514402648">video</a>]
+</li></ul>
+
+<h3>2020</h3>
+
+<ul><li>
+ <b>The Future is Ours: Prophecy Variables in Separation Logic</b><br/>
+ Ralf Jung, Rodolphe Lepigre, Gaurav Parthasarathy, Marianna Rapoport, Amin Timany, Derek Dreyer, Bart Jacobs<br>
+ <i>In <a href="https://popl20.sigplan.org/">POPL 2020</a></i><br>
+ [<a href="https://plv.mpi-sws.org/prophecies/paper.pdf">paper</a>] [<a href="https://plv.mpi-sws.org/prophecies/">paper website</a> (incl. appendix and Coq formalization)]
+</li></ul>
+
+<ul><li>
+ <b>Stacked Borrows: An Aliasing Model for Rust</b><br/>
+ Ralf Jung, Hoang-Hai Dang, Jeehoon Kang, Derek Dreyer<br>
+ <i>In <a href="https://popl20.sigplan.org/">POPL 2020</a></i><br>
+ [<a href="https://plv.mpi-sws.org/rustbelt/stacked-borrows/paper.pdf">paper</a>] [<a href="https://plv.mpi-sws.org/rustbelt/stacked-borrows/">paper website</a> (incl. appendix and Coq formalization)]
+</li></ul>
<h3>2018</h3>
<ul><li>
<b>Iris from the Ground Up: A Modular Foundation for Higher-Order Concurrent Separation Logic</b><br/>
Ralf Jung, Robbert Krebbers, Jacques-Henri Jourdan, Aleš Bizjak, Lars Birkedal, Derek Dreyer<br>
- In <a href="https://www.cambridge.org/core/journals/journal-of-functional-programming/issue/832F8756D0A835287494117BF9ED5E74">Journal of Functional Programming (JFP), Volume 28, e20, November 2018</a><br>
+ <i>In <a href="https://www.cambridge.org/core/journals/journal-of-functional-programming/issue/832F8756D0A835287494117BF9ED5E74">Journal of Functional Programming (JFP), Volume 28, e20, November 2018</a></i><br>
[<a href="https://www.mpi-sws.org/~dreyer/papers/iris-ground-up/paper.pdf">paper</a>] [<a href="https://iris-project.org">project website</a> (incl. appendix and Coq formalization)]
</li></ul>
<ul><li>
<b>Reconciling High-Level Optimizations and Low-Level Code in LLVM</b><br/>
Juneyoung Lee, Chung-Kil Hur, Ralf Jung, Zhengyang Liu, John Regehr, Nuno P. Lopes<br>
- In <a href="https://conf.researchr.org/track/splash-2018/splash-2018-OOPSLA">OOPSLA 2018</a><br>
+ <i>In <a href="https://conf.researchr.org/track/splash-2018/splash-2018-OOPSLA">OOPSLA 2018</a></i><br>
[<a href="twinsem/twinsem.pdf">paper</a>] [<a href="https://sf.snu.ac.kr/llvmtwin/">paper website</a>]
</li></ul>
<ul><li>
<b>MoSeL: A General, Extensible Modal Framework for Interactive Proofs in Separation Logic</b><br/>
Robbert Krebbers, Jacques-Henri Jourdan, Ralf Jung, Joseph Tassarotti, Jan-Oliver Kaiser, Amin Timany, Arthur Charguéraud, Derek Dreyer<br>
- In <a href="https://icfp18.sigplan.org/">ICFP 2018</a><br>
+ <i>In <a href="https://icfp18.sigplan.org/">ICFP 2018</a></i><br>
[<a href="https://iris-project.org/pdfs/2018-icfp-mosel-final.pdf">paper</a>] [<a href="https://iris-project.org/mosel/">paper website</a> (incl. appendix and Coq formalization)]
</li></ul>
<ul><li>
<b>RustBelt: Securing the Foundations of the Rust Programming Language</b><br/>
Ralf Jung, Jacques-Henri Jourdan, Robbert Krebbers, Derek Dreyer<br>
- In <a href="https://popl18.sigplan.org/">POPL 2018</a><br>
+ <i>In <a href="https://popl18.sigplan.org/">POPL 2018</a></i><br>
[<a href="https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf">paper</a>] [<a href="https://plv.mpi-sws.org/rustbelt/popl18/">paper website</a> (incl. appendix and Coq formalization)] [<a href="https://www.youtube.com/watch?v=Cy9NUVaiYUg" target="_blank">talk (YouTube)</a>] [<a href="https://plv.mpi-sws.org/rustbelt/popl18/talk.pdf">talk (slides)</a>]
</li></ul>
<ul><li>
<b>A Higher-Order Logic for Concurrent Termination-Preserving Refinement</b><br/>
Joseph Tassarotti, Ralf Jung, Robert Harper<br>
- In <a href="https://www.etaps.org/index.php/2017/esop">ESOP 2017</a><br>
+ <i>In <a href="https://www.etaps.org/index.php/2017/esop">ESOP 2017</a></i><br>
[<a href="https://iris-project.org/pdfs/2017-esop-refinement-final.pdf">paper</a>] [<a href="https://www.cs.cmu.edu/~jtassaro/papers/iris-refinement/index.html">paper website</a> (incl. Coq formalization)]
</li></ul>
<ul><li>
<b>Higher-Order Ghost State</b><br/>
Ralf Jung, Robbert Krebbers, Lars Birkedal, Derek Dreyer<br>
- In <a href="https://conf.researchr.org/home/icfp-2016">ICFP 2016</a><br>
+ <i>In <a href="https://conf.researchr.org/home/icfp-2016">ICFP 2016</a></i><br>
[<a href="https://iris-project.org/pdfs/2016-icfp-iris2-final.pdf">paper</a>] [<a href="https://iris-project.org">project website</a> (incl. appendix and Coq formalization)] [<a href="https://www.youtube.com/watch?v=vUxWU-qvGX0" target="_blank">talk (YouTube)</a>] [<a href="iris/talk-icfp2016.pdf">talk (slides)</a>]
</li></ul>
<h3>2015</h3>
-<!--<ul><li>
- <b>Unifying Worlds and Resources</b><br/>
- Ralf Jung, Derek Dreyer<br>
- In <a href="https://users-cs.au.dk/birke/hope-2015/">HOPE 2015</a><br>
- [<a href="https://www.youtube.com/watch?v=9Dyna88piek&list=PLnqUlCo055hX6SsmMr1AmW6quMjvdMPvK&index=10" target="_blank">talk (YouTube)</a>] [<a href="iris/talk-hope2015.pdf">talk (slides)</a>]
-</li></ul>-->
-
<ul><li>
<b>Iris: Monoids and Invariants as an Orthogonal Basis for Concurrent Reasoning</b><br/>
Ralf Jung, David Swasey, Filip Sieczkowski, Kasper Svendsen, Aaron Turon, Lars Birkedal, Derek Dreyer<br>
- In <a href="https://popl.mpi-sws.org/2015/">POPL 2015</a><br>
+ <i>In <a href="https://popl.mpi-sws.org/2015/">POPL 2015</a></i><br>
[<a href="https://iris-project.org/pdfs/2015-popl-iris1-final.pdf">paper</a>] [<a href="https://iris-project.org">project website</a> (incl. appendix and Coq formalization)] [<a href="iris/talk-popl2015.pdf">talk (slides)</a>]
</li></ul>
+<h2>Theses</h2>
+
+<h3>2020</h3>
+<ul><li>
+ <b>Understanding and Evolving the Rust Programming Language</b><br/>
+ Ralf Jung<br>
+ <i>PhD Thesis</i><br>
+ [<a href="thesis.html">website</a>]
+</li></ul>
+
+
+<h3>2019</h3>
+<ul><li>
+ <b>Higher-Order Ghost State</b><br/>
+ Ralf Jung<br>
+ <i>Master Thesis</i><br>
+ [<a href="https://iris-project.org/pdfs/2016-icfp-iris2-final.pdf">based on this paper</a>]
+</li></ul>
+
<h3>2013</h3>
<ul><li>
- <b>An Intermediate Language To Formally Justify Memory Access Reordering</b> (Bachelor's Thesis)<br/>
- Ralf Jung, Supervisor: Sebastian Hack, Advisor: Sigurd Schneider<br>
- [<a href="bachelor/bachelor.pdf">pdf</a>] [<a href="bachelor/bachelor-talk.pdf">slides</a>] [<a href="bachelor/bachelor.zip">coq</a>]
+ <b>An Intermediate Language To Formally Justify Memory Access Reordering</b><br/>
+ Ralf Jung<br>
+ <i>Bachelor Thesis</i><br>
+ [<a href="bachelor/bachelor.pdf">thesis</a>] [<a href="bachelor/bachelor-talk.pdf">slides</a>] [<a href="bachelor/bachelor.zip">coq</a>]
+</li></ul>
+
+<h2>Workshop Talks</h2>
+
+<h3>2020</h3>
+
+<ul><li>
+ <b>Stacked Borrows: An Aliasing Model for Rust</b><br/>
+ Ralf Jung<br>
+ <i>At <a href="https://agozillon.github.io/PRiML/">PRiML 2020</a> (invited talk)</i><br>
+ [<a href="rust/talk-priml2020.pdf">slides</a>]
+</li></ul>
+
+<h3>2019</h3>
+
+<ul><li>
+ <b>Logical Atomicity in Iris: the Good, the Bad, and the Ugly</b><br/>
+ Ralf Jung<br>
+ <i>At <a href="https://iris-project.org/workshop-2019/">Iris Workshop 2019</a></i><br>
+ [<a href="iris/talk-iris2019.pdf">slides</a>]
+</li></ul>
+
+<h3>2015</h3>
+
+<ul><li>
+ <b>Unifying Worlds and Resources</b><br/>
+ Ralf Jung, Derek Dreyer<br>
+ <i>At <a href="https://users-cs.au.dk/birke/hope-2015/">HOPE 2015</a>: 4th ACM SIGPLAN Workshop on Higher-Order Programming with Effects</i><br>
+ [<a href="iris/talk-hope2015.pdf">slides</a>] [<a href="https://www.youtube.com/watch?v=9Dyna88piek&list=PLnqUlCo055hX6SsmMr1AmW6quMjvdMPvK&index=10" target="_blank">YouTube</a>]
</li></ul>
-</p>
-../ralf/style.scss
\ No newline at end of file
+../personal/style.scss
\ No newline at end of file
--- /dev/null
+---
+title: Understanding and Evolving the Rust Programming Language
+slug: Thesis
+---
+
+<i>PhD Thesis, Defended in August 2020</i>
+
+
+<h3>Abstract</h3>
+
+<p><em>Rust</em> is a young systems programming language that aims to fill the gap between <em>high-level languages</em>—which provide strong static guarantees like memory and thread safety—and <em>low-level languages</em>—which give the programmer fine-grained control over data layout and memory management. This dissertation presents two projects establishing the first formal foundations for Rust, enabling us to better <em>understand</em> and <em>evolve</em> this important language: <em>RustBelt</em> and <em>Stacked Borrows</em>.</p>
+
+<p><em>RustBelt</em> is a formal model of Rust’s type system, together with a soundness proof establishing memory and thread safety. The model is designed to verify the safety of a number of intricate APIs from the Rust standard library, despite the fact that the implementations of these APIs use <em>unsafe</em> language features.</p>
+
+<p><em>Stacked Borrows</em> is a proposed extension of the Rust specification, which enables the compiler to use the strong aliasing information in Rust’s types to better analyze and optimize the code it is compiling. The adequacy of this specification is evaluated not only formally, but also by running real Rust code in an instrumented version of Rust’s <em>Miri</em> interpreter that implements the Stacked Borrows semantics.</p>
+
+<p>RustBelt is built on top of <em>Iris</em>, a language-agnostic framework, implemented in the Coq proof assistant, for building higher-order concurrent separation logics. This dissertation begins by giving an introduction to Iris, and explaining how Iris enables the derivation of complex high-level reasoning principles from a few simple ingredients. In RustBelt, this technique is exploited crucially to introduce the <em>lifetime logic</em>, which provides a novel separation-logic account of <em>borrowing</em>, a key distinguishing feature of the Rust type system.</p>
+
+<p style="font-weight:bold">
+This thesis has received an <a href="https://awards.acm.org/about/2020-doctoral-dissertation" style="font-weight:bold;">Honorable Mention for the ACM Doctoral Dissertation Award</a>,
+the <a href="https://sigplan.org/Awards/Dissertation/#2021_Ralf_Jung__Max_Planck_Institute_for_Software_Systems_and_Saarland_University" style="font-weight:bold;">ACM SIGPLAN John C. Reynolds Doctoral Dissertation Award</a> (as one of two recipients),
+an <a href="https://www.mpg.de/prizes/otto-hahn-medal" style="font-weight:bold;">Otto Hahn Medal</a>,
+the <a href="https://etaps.org/2021/doctoral-dissertation-award" style="font-weight:bold;">ETAPS Doctoral Dissertation Award</a>,
+and the Saarland University <a href="https://www.uni-saarland.de/universitaet/aktuell/news/artikel/nr/24072.html" style="font-weight:bold;">Dr. Eduard Martin-Preis</a>.
+</p>
+
+<h3>Download and references</h3>
+
+<ul>
+<li><a href="phd/thesis-screen.pdf">Screen-optimized version (PDF)</a> (updated 2020-12-08)</li>
+<li><a href="phd/thesis-print.pdf">Print-optimized version (PDF)</a> (official final version)</li>
+<li>DOI: <a href="http://dx.doi.org/10.22028/D291-31946">10.22028/D291-31946</a></li>
+</ul>
git diff --cached
git commit -m "site upload"
-rsync ./ mpi-contact:public_html/ -aP --exclude .git
+rsync ./ ralfj.de:/srv/research.ralfj.de/docroot/ -aP --exclude .git
#/bin/bash
cd "$(dirname "$0")"
-ralf/upload.sh
+personal/upload.sh
research/upload.sh
projects / web.git / commitdiff
