32 – Who’s Afraid of AI?

The Logical Fallacy of Generalizing from fictional evidence

Genie Button Though Experiment

Wait but Why on AI Part 1 and Part 2

The Downfall meme we mentioned. This is Steven’s favorite version.

Yudkowsky vs Hanson – Great AI FOOM debate  and the Video

Sam Harris AI TED talk

Albion’s Seed – SSC

Redditor provides an incredible explanation of how being poor can make you bad with money

This entry was posted in Uncategorized. Bookmark the permalink.

16 Responses to 32 – Who’s Afraid of AI?

  1. Gallant Tailor says:

    Hi there. G.T. here. Thank you for spending so much time addressing my feedback on the show. You really didn’t have to do that, certainly not at such length. After all that, I feel like I have to respond in some way, so at risk of drawing out a conversation that is already long enough, I’m taking up Steven’s suggestion and posting here. I will post the original message in full below after sharing my thoughts.

    Obviously, I’m going to take exception to what was said in the episode. My general objection is that throughout your response to me, you interpreted what I was saying in a very extreme way. Neither I nor most conservatives believe that human nature is the most important consideration in every issue. The idea is that human nature imposes limits, not that nothing else matters or that human nature reduces the variability in human societies down to zero. My point was that a belief in the importance of human nature is one of the things that makes an ideology right-wing, and the more you believe that human nature matters, the more right-wing you are. There is a spectrum of views here, and very few people inhabit the most extreme end of it.

    Regarding some more specific points:

    You spent a lot of time addressing what I said in this section:

    “The leftist view seems to be that with enough social engineering, any kind of society is possible. The poor can be made wealthy, black communities can be raised to the level of white communities, same-sex unions can acquire the same respect as traditional marriages, etc.”

    The examples in the second sentence were actually only meant to be instances of what I was describing in the previous sentence, namely problems that the left believes can be solved through widespread social change. All that these two sentences were supposed to be was a characterization of the left-wing view of social progress. Frankly, I generated these examples just by thinking for five seconds about social changes that the left wants. If I thought that these three examples in this one sentence would be the focus of much of your response, I would have chosen them more carefully. I will grant you that if all you do is take the exact opposite of each sentence e.g. “The poor CANNOT be made wealthy!” the result is some pretty extreme claims, but I wasn’t thinking of it in those terms. What I WOULD say about the right-wing view on these issues is that a conservative will likely take a more limited perspective on the changes he expects society to make than a liberal will. Perhaps: “Tell your children not to bully the gay kids in school”, rather than “Destroy heteronormativity!”, or “Don’t discriminate on the basis of race.” rather than “We will not rest until 13% of congress is black!”, or “Help out with your church’s charity work.” rather than “War on Poverty!” Conservatives tend to be skeptical of the power of social change, and a belief in human nature is one of the reasons for that. That’s all was trying to say. I certainly didn’t mean to say that human nature makes it fundamentally impossible to address any of these things even a little. You would have to be very right-wing indeed to believe that, and it is certainly not my view. I wish you had instead focused more on the examples I gave in the next section regarding race and racism. Those are points I would actually stand by.

    I also take exception to your claim that I made a “pro-lynching” argument. I said two things about lynching which were: 1.) lynchings were not random, rather they were done in response to crimes, and 2.) many whites were lynched too, indicating that they were not entirely racially motivated. Saying that lynchings were not random and that they were not entirely racially motivated does not imply that they were a good thing. I suppose you might say that I have a view of lynching that is less negative than is typical, but I think it is misleading to characterize such a view as “pro-lynching”. I totally agree that extra-judicial killing is a terrible thing. You don’t need to tell me that. I never said otherwise.

    In addition to giving an extreme version of what I said, you also moderated your own views. For instance, Eneasz pointed out that if I think it’s reasonable to expect people not to murder each other, then I should consider it reasonable to expect people to “be decent to people who are a different skin color”. Okay, if your standard for not being racist is just not being wantonly mean to other races, then sure, that’s reasonable. But if you want people to show no racial preference at all in their choice of friends, neighborhoods, romantic partners, and the like, then I don’t think that’s going to happen. Honestly, it seems a bit like you’re motte-and-baily-ing me. When you speak to someone on my side, you “just want us to be nice to LGBT people.”, for instance, but when you speak to others on your side you are policing each other’s gendered pronouns and criticizing one another for accidentally saying something heteronormative. Those standards aren’t really equivalent.

    You also attributed to me a belief that biological factors are responsible for some of the social ills among certain racial or socioeconomic groups. This was not something I meant to imply, but it just so happens that I DO believe a moderate version of this. You are absolutely correct in saying that I did not give sufficient support for this belief in the original message, and I will happily provide it if asked (in another post though, this one is already too long). I just did not consider such an argument to be particularly germane to the point I was making there.

    Finally, I don’t want to drag out this discussion about Richard Spencer any more, but I will just point out that everything you said about Richard Spencer in this episode was already addressed in my original message. Just look at the second half of the relevant paragraph. Richard Spencer does NOT advocate for the deportation of all non-whites from the United States. If someone told you that, they are making it up.

    TL;DR: I’m not as extreme as the episode makes me out to be.

  2. Gallant Tailor says:

    My original message to The Bayesian Conspiracy:

    Hello again. I had some thoughts on your latest episode concerning your response to Albionic American at the very end, plus I wanted to make a few belated remarks on your response to me in “I did Nazi that coming”. I apologize in advance for the length of this message, but I could not find a way to cut it down and still say everything I wanted. I can understand if you don’t want to respond to all of this, or indeed any of this.


    I really enjoyed your response to Albionic American. I was particularly fascinated when Eneasz said that he genuinely had not considered that people might have the opinion expressed in that comment. So I feel the need to tell you, if you’re reading this, Eneasz, the idea that humans are fundamentally constrained (to use the term that was mentioned in the episode) is a core tenant of right-wing thought. Depending on who you ask it might even be the core tenant. I will acknowledge that this particular commenter did not give a full argument for his position, but he was gesturing towards something very important. So, since this idea seems to be a new one to you, I would like to lay it out in full.

    The right believes in something called human nature, which is to say basic instincts, desires, biases, etc. that all humans are born with and will always have to some extent. People may display different behaviors as a result of different material conditions or the presence or absence of different ideas, but the fundamentals of human nature are constant. As a result, any utopian scheme that requires us to fundamentally change our nature is bound to fail. If Socialism requires that greed be destroyed, Socialism will fail. If feminism requires that the distinction between male and female be destroyed, feminism will fail, and if world peace requires that in-group loyalty be destroyed, then world peace will fail. The leftist view seems to be that with enough social engineering, any kind of society is possible. The poor can be made wealthy, black communities can be raised to the level of white communities, same-sex unions can acquire the same respect as traditional marriages, etc. The right believes that if such things are desirable at all, we will at least need to see some evidence that we can get there.

    You brought up the idea of moral progress in the past to support the idea that further progress is possible. The right is generally skeptical of the idea of moral progress in general, at least partly because we believe that society has actually regressed morally on certain issues. Consider Jonathan Haidt’s moral foundations. If conservatives value things like loyalty, authority, and purity, then have we really progressed from their perspective? Are people more respectful of authority now than they were in the past? Are we more loyal to our nations or tribes? Are we more chaste in our sexual behavior? As far as I can tell, not so much. It would be difficult to find a conservative who would deny all moral progress, but the general feeling is that, to the extent that moral progress really is progress, it is a weak force at best. The right maintains that there are limits to how far moral progress can take us, the progress that has been made could fall apart if it isn’t maintained properly, and we have not actually made as much progress as liberals think.

    To give an example, the commenter specifically mentioned the alt-right’s view of race, and my politics are more or less in line with the alt-right, so I think I can further explain their position. The alt-right believes that on racial issues, what appears to have been progress is not actually progress at all. People are as racist as they have ever been, and those who claim not to be racist have simply not had enough exposure to other races to know any better. For instance:

    We don’t enslave blacks anymore! – Except that the abolition of slavery had more to do with technological advancement making slave labor obsolete than an increased compassion for blacks.
    We don’t lynch blacks at random for no reason anymore! – Except that lynchings were not random, they were done in response to crimes, and they were not targeted at blacks alone. About a third of lynching victims were white in a region where the population was two thirds white, one third black. Blacks were lynched disproportionately probably because they were disproportionately criminal as they still are today.
    We don’t segregate the races anymore! – Except that we still do, it’s just not enforced by law anymore. And self-segregation happens despite the government’s best efforts to foster diversity.
    We don’t force blacks to ride at the back of the bus in Alabama anymore! – And now whites in Alabama don’t ride the bus at all if they can help it.
    We have anti-discrimination laws now! – And that’s because our elites who make the laws are sheltered white people who live in homogeneous communities and know nothing of the realities of racial conflict on the ground. If the lower class whites who actually have to deal with diversity on a daily basis were making the laws, they would be very different.
    We are more diverse now! – And racial tensions are on the rise as a result.
    Our academics don’t publish pseudo-scientific papers on “the cranial capacity of the negroid” anymore! – Except that those findings are turning out to be not so pseudo-scientific after all (http://news.rutgers.edu/issue.2012-02-02.7844909214/article.2012-02-14.7247023894#.WNMBPvFie00).
    The media doesn’t use racist stereotypes anymore! – And our mainstream media is run by the same sheltered, naive, out of touch elites who run the government. Get online, where ordinary people communicate, and there are racist stereotypes everywhere.
    Interracial marriage is acceptable now! – But do people really accept it (https://www.washingtonpost.com/news/post-nation/wp/2016/08/19/people-say-they-approve-of-interracial-couples-but-studies-uncover-bias/?utm_term=.9009afc57152)? They’re certainly not doing it very much.
    It’s no longer socially acceptable to make racist remarks in public! – And there you are correct. The one thing the left has succeeded on when it comes to race is making it fashionable among middle-to-upper-class whites to espouse anti-racist ideas. But as far as victories go, this is a minor one. Anti-racism is a fashion trend that can only continue while these people still have 98% white neighborhoods that they can retreat to when the diversity they were advocating for actually arrives in their nations. People are only malleable up to a point. You can convince them to be tolerant of a group that’s far away and that they never see, but convincing people to tolerate an out-group that they see and interact with every day? I am not convinced that that’s possible. Certainly not on a large scale. If you don’t believe me consider this: are Europeans more or less racist now that more non-whites are entering their countries?

    And it’s not just race. On every issue where there seems to have been moral progress, the honest conservative will say that the progress is illusory, or that it came as a result of external factors, or that it was actually regress, or that it was genuine but small and will be limited in its future potential. You can call this view “pessimistic” or you can characterize human nature as “shitty” if that’s what you want, but to me it makes far more sense to hold people to normal human standards of morality that we know are reachable rather than to be disappointed that humans are not acting the way your pet utopia would require them to act.

    I imagine you disagree with this position, though. I don’t frequently hear this issue argued explicitly, so I’d be interested to hear your take.


    In your response to me, I thought you were quite fair overall and gave my concerns their due consideration. Thank you for that. I am however going to push back on one point you made and assert again that Richard Spencer does not advocate violence. Your argument that he does, frankly left me a bit confused. You correctly noted that Spencer does not publicly endorse any violent actions, so are you alleging that he privately endorses violence? If so, that is the sort of thing I would need evidence for. You spent a lot of time arguing that a peaceful ethnic cleansing is not possible, but even if you were correct on that, clearly Spencer does think it’s possible, so your argument isn’t really relevant to the question of what Spencer advocates. If someone supports an impossible policy, that does not license you to accuse him of supporting some alternate policy. Furthermore, your refutation of the idea of peaceful separation of the races focused solely on the present day US and solely on forced deportation as the mechanism. Most white nationalists, including Spencer, recognize that forcibly deporting every non-white from the US as it exists today would be inhumane and impractical. But there have been white nationalist strategies proposed other than forced deportation (I won’t go into them at length, but the main ones are secession and encouragement of self-deportation), and there are other white countries where deportation is more viable. There are even countries like Norway that are still so white that they could simply close their borders and the white nationalists would have everything they want. Spencer himself has always been agnostic on both the location of his dream ethno-state and the method by which it would be achieved, so your discussion of forced deportation from the US is not really relevant to him.

    Don’t take this the wrong way, but it seems like you guys are getting your information on white nationalism from Hollywood movies instead of actually listening to what white nationalists have to say. It kind of bothers me that you are willing to call Spencer and other people like him “douchebags” and “assholes” without really knowing much about them.

    You also mentioned in that episode that you would be interested to hear what comments of yours I have found “horribly backwards”. Upon further reflection, perhaps I shouldn’t have used the word “horribly”, and I apologize for that. On your main discussion topics you are actually pretty good at being neutral, better than most journalists in fact. The exceptions to that are the Nazi punching episode (which I have already discussed), the polyamory episodes (in which I thought you did a poor job representing the anti-polyamory side), the Trump reaction episode (in which you weren’t trying to be balanced), and the “Tinder, Actually” episode (which didn’t have a main topic). Sometimes your choice of topic strays toward left-wing talking points (e.g. “Kill all people”, “Kids these days”, “Animals”) even though within the episodes the topics are handled mostly fairly. But what I was mostly talking about when I said that some of the things you say bothered me were the side comments, not directly related to the main topic, that betray a left-wing perspective. Some examples:
    Eneasz opines that human society is pretty good at figuring out what is moral, Katrina jokes that maybe that is in doubt considering the election result, everyone laughs (because of course we’re all anti-Trump here, right?).
    Eneasz mentions that he uses emojis in work emails, Katrina suggests that his ability to do that and still be taken seriously is due to his being a man, Eneasz readily agrees (because even emojis are a feminist issue, apparently).
    Steven asks what the “manic pixie dream girl trope is, Eneasz describes it as a trope that is used by white male directors (because white people are so boring and un-creative, right?).
    Eneasz says “Old white men are the worst.” and “Fuck these white dudes ruining shit!” (because that’s not anti-white, right? Even though you would never say that about any other race).
    And so on. I haven’t kept notes, but there are plenty more. These are just the ones that stuck in my head for whatever reason. I don’t necessarily expect you to purge all comments like this from the show. I like that the podcast is informal and sounds like a normal conversation, and I don’t want you to feel like you have to constantly monitor what you say to avoid offense and remain neutral. I just wanted to let you know that some of the things you say might be a turn off for half of the country.


    Thank you for your time.

  3. Senjiu says:


    there’s a project where they’re gathering data on what humans would want self-driving cars to do in trolley-problem-like situations: http://moralmachine.mit.edu/hl/

    Have fun with it! 😛

  4. James says:

    I think the way you guys are talking about pain is what is making the math breakdown.

    You are measuring pain in painons. However pain is not a linear intiger. The model you are useing is similar to a linear no threshold modle. Whre half a painon is bad. And 2 painons are twice as bad as one.

    I would propose you should use a logrythmic threshold modle. So that below a point pain doesn’t count. So a mote of dust at .001 painon is imperceptable. And 2 painons are an order of magnitude worse than one.

    Further the subjective nature of pain makes this difficult. I would rather get paper cut twice at once then one after the other. And I might enjoy a paper cut from a loved one vs being supremely distressed is it came from Donald Trump.

    I don’t immediately see a way to encapsulate distress that elegantly captures levels of physical and mental distress in different situations and different people.

    I also object to your lack of marxists commodity analysis in your politics but I can’t so elegantly explain that.

    • The experiment posits that the amount of pain in the “very low” side of the equation be enough to be non-negligible. So use the least-painful thing that still DOES count.

      As for order of magnitudes… ultimately that doesn’t matter if you simply have *enough* of the smaller pains. Any small number, when multiplied by a large enough number, will generate a sum larger than some other arbitrarily large number. No matter how large a number you pick for your “very painful” side, one can find a multiplier that will bring the total amount on the “small pain” side to a higher total. The problem is with using numbers at all, in which case one must either choose to not use numbers, or accept what the numbers say.

    • Regarding marxist commodit analysis – I know nothing of that, so it will definitely be missing from my politics. Sorry.

  5. Noam says:

    First of all please allow me to apologize in advance for any grammar and spelling mistakes, as English is not my first language.
    After listening to this episode, I wondered about your approach to the idea of a race of super intelligent AI of humanity’s making inheriting our place in the future. I find that notion to be very comforting, as it describe a future in which, even if humanity fails to make peace among itself and go on to explore the universe, we have left a better being to go on in our stand.
    I find that many people have very low expectations of humanity’s ability not annihilate itself within the next few centuries, and so- if worse comes to worse- the idea of humanity leaving behind a consciousness that suppresses it, is quite fetching.

    • Senjiu says:

      I haven’t thought about that and I like it. It’s probably a lot easier for an AI with robots (as agents to affect the physical world) to survive a post-nuclear-war-world or a world with a runaway greenhouse effect (obviously it would be able to survive an incurable pandemic). What else is there? Ah, meteorite impact. Probably would survive that too if it isn’t located exactly below it.
      I think that’s all the doomsday scenarios I could think of.

      I’d also like some consciousness uploading. Or something of that sort. I imagine us using all kinds of implants at some point in the future (to improve memory, to directly interface with other pieces of technology, to restore functions of the brain that are declining due to illness or accidents. And the way I see it at some point more than 50%, more than 75%, more than 90% and eventually more than 99% of the brain of a person will be a machine. At no point in that transition would I say “okay, now the guy is dead and it’s a robot”, the personality would be the same and consciousness would stay too (from the perspective of the subject).
      I’m aware that that wouldn’t be an AI, it would be a human personality inside a computer. I think that’s maybe one scenario Elon Musk wants to head towards, so there are humans in the realm of AIs too, to keep them in check if necessary. (Otherwise I might not have understood what he’s aiming for..)

  6. Jonathan Doolin says:


    Thanks for reading my letter about voting. I was amused at how “voting” in an election got tied to “defecting” in the prisoner’s dilemma.

    I suppose, the framework you’re setting up is that “I know they’re voting irrationally for candidate A, therefore I will vote irrationally for candidate B.” If this were the case, I suppose that might be an example of defecting in the prisoner’s dilemma. I did not mean to make the case for “voting irrationally”.

    I suppose, voting on the basis of looks, hand-size, pithy slogans, smile, etc, might well be irrational motivations that are, in fact, more effective motivators at getting people out to vote. And if indeed those irrational elements are what inspire you to vote, then you may well be “defecting.”

    But, if you make an effort to vote on the basis of the platform that the candidate supports, and the candidate has a history of consistent and competent pursuit towards the goals and principles you agree with, then voting for that candidate is rational.

    That being said, there was recently a local election, and in fact, I did not vote, because I had no knowledge of the issues or the candidates. It was only the next day, when I heard someone bemoaning the low (2%) turnout, that I asked “Why should I vote if I have no idea, even who is running, and every seat is uncontested?” She told me it is still rational to go in and vote, (perhaps leaving the ballot blank), even in those cases, simply to be on record as a voter, because politicians will tend to focus their concerns on areas where people are interested enough in elections to vote.

    • Senjiu says:

      I considered leaving the ballot blank but then I looked up what that results in and.. basically it’s the opposite of what I want. It counts towards voter participation (unlike people staying at home) but it doesn’t influence how much every party got. So if I want the politicians to notice me being unhappy with the government I can’t not vote and can’t vote empty ballot. I decided to vote for a party that had next to no chances of getting into the parliament in that situation. If the category “other” gets a higher percentage all of the established parties lose a bit and that’s the clearest signal I can think of that says “I’m unhappy with the current situation.”.

  7. Edgar says:

    If you’ve not yet read “On the Impossibility of Supersized Machines” yet, I recommend it. I expect you’ll be amused.


  8. Emerich says:

    Since you guys (or at least one of you) are self-admittedly “leftish”, and you’re also rationalist minded, here’s a suggestion for a possible future podcast: what is “emergent order”, where and why does it happen, and what are its implications? And second: “What is Kenneth Arrows impossibility theorem, and does it have any implications for social and political organization?

    Have fun.

  9. Metastable says:


    Around 02:30 you said that Vernor Vinge coined the term “intelligence explosion” and that is incorrect – actually it was I. J. Good in 1965 (unless, you meant “singularity” then indeed Vinge was first to use that term in the context of superintelligence, for what I know).

    For interested listeners, more on the Three Major Singularity Schools here: https://youtu.be/mEt1Wfl1jvo or in text format here: https://intelligence.org/2007/09/30/three-major-singularity-schools/

    Around 35 minute you spoke about Elon Musk. He is engaged in OpenAI – https://openai.com/about/ and Neuralink – https://waitbutwhy.com/2017/04/neuralink.html and – https://neuralink.com/ He also invested couple years ago in DeepMind and Vicarious to keep an eye on what is going on in AI.

    Eneasz, you said that Elon was early founder of MIRI but I think you confused him with Peter Thiel (otherwise could you provide citation, please?). MIRI recently got some money from Musk but they got it indirectly through Future of Life Intitute: https://intelligence.org/2015/07/05/july-2015-newsletter/
    “MIRI has been awarded a $250,000 grant from the Future of Life Institute spanning three years to make headway on our research agenda. This will fund three workshops and several researcher-years of work on a number of open technical problems. MIRI has also been awarded a $49,310 FLI grant to fund strategy research at AI Impacts.”

    (Musk gave FLI $10 mln for research on beneficial AI: https://futureoflife.org/2015/10/12/elon-musk-donates-10m-to-keep-ai-beneficial/ )

    On what would be better to have first, full brain emulations or AGI I agree with Eliezer https://youtu.be/EUjc1WuyPT8?t=1h9m43s

    I think you may be interested in Luke Muehlhauser’s reply to Wait-but-Why’s posts on AI http://lukemuehlhauser.com/a-reply-to-wait-but-why-on-machine-superintelligence/

  10. Pingback: 79 – CyberChristianity | The Bayesian Conspiracy

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.