Podcast: Play in new window | Download
Subscribe: RSS
We talk with Tsvi from MIRI
Game-playing algorithm that pauses Tetris
“On The Origin of Circuits“, discussing a chip hardware evolution experiment
MIRI’s technical research agenda overview
Alignment for advanced machine learning systems paper from MIRI
Musk’s OpenAI
Paul Christiano’s about page, which links to his paper Tsvi mentioned
Logical induction paper from MIRI
Reason as Memetic Immune Disorder, by Phil Goetz (not Scott Alexander (yet))
The original “EA Has a Lying Problem” post. Lost of discussion in the comments, and also over at E-A.com
Eneasz made a comment somewhere in the episode to the effect that we should reward particularly effective altruists (such as those who save 5 other children rather than their own child) with hero worship and high praise. Perhaps this is a good idea, but for perverse reasons. What goal are we aiming at when we calculate our answer? What does that person want? What does that person deserve? What effect will this have on onlookers’ future behavior?
I hope that there are more effective ways to encourage persons to internalize the sort of norms and heuristics that would result in such behavior, though I am not sure what works. I was immediately reminded of Alfie Kohn’s book “Punished by Rewards” and Dan Pink’s book “Drive”, both of which describe some of the counterintuitive results found by researchers investigating motivation. Praise can become manipulative and persons who crave the approval of others too much become easy targets. That research is probably not actually relevant to the example of fire rescue, due to emergency factors at work there, but the same can be said about hero worship after the fact, at least with respect to that particular hero’s behavior. Maybe if stories about such persons get embedded in culture this could have more of an effect?
We should perhaps be grateful that they had the presence of mind to be able to save anyone at all, since under such chaotic and terrifying circumstances, someone with no training is basically operating on hunter-gatherer instinct rather than rational deliberation. Their amygdala is boss but lacks the software to deliberate.
In your discussion of the problems of accepting logical conclusions, I think you may have ended up with more of an anti-logic stance than I think is really justified.
That terrorists use logic (or more strongly: using logic made some people into terrorists) means use of logic doesn’t guarantee you a good answer (or even that you’re in the ballpark of one), but that lack of a guarantee isn’t a strong argument for abandoning logic. The takeaway “always be suspicious of logically derived conclusions” is solving the wrong problem. I think the standard explanation of the mistake is that “reversed stupidity is not intelligence.” To get the right answer, you have to get ALL the steps right, not just some of them.
When your computer breaks, deciding that you need a new one without an AMD processor or an ATI graphics card (when all you needed was a new power supply) is just as problematic a mistake as insisting your computer must work because it has an AMD processor and an ATI graphics card. The logical terrorists may have made the latter mistake, but that’s not a good reason to embrace the former.