Making it work, and how it doesn't I love this line: Why does rationality work? In large part, because we do practical work to make it work. In addition to making the world less nebulous, we do this by applying our systems in nebulous ways. We fudge, applying tacit “know-how” to the way we implement the system. Mostly, this is a good thing, because it keeps everything going. (Forgive me if this what I’m about to rant about is covered in a later part; I’ve only read Part I so far.) However, there’s a risk that an incoherent understanding of this fact will be marshalled as an excuse for dysfunctional systems. For example, Microsoft a couple of years ago changed their terms of service to say Don’t publicly display or use the Services to share inappropriate content or material (involving, for example, nudity, bestiality, pornography, offensive language, graphic violence, or criminal activity). Seems reasonable at first, right? Except it’s for all their services, and does not distinguish between private Skype video calls to your significant other, and posting something publicly on, e.g., Xbox Live. Speaking of posting stuff on Xbox Live, Microsoft encourages you to post clips from the games you play, many of which are quite graphically violent. And does “shar[ing]… criminal activity” include using your Hotmail account to discuss a crime that someone else committed? Microsoft’s response was, bizarrely, to insist that they hadn’t actually changed the policy, only clarified it. In essence, though, their response to the criticisms was, “That’s not what we meant,” which is the usual response to someone pointing out that a policy is overly broad and has unreasonable consequences. As if “Don’t worry, we intend to enforce the policy selectively,” is supposed to be comforting.