If Anyone Builds It, Everyone Dies

... or "you can get ahead through telling people what they want to hear" or "people mistake dark personality traits for high agency", etc.

What gets me about 'rationalism' is how ahistorical it is. No reference to Korzybski or Teilhard de Chardin. There never was the movie Colossus the Forbin Project or the 1960s speculation that progress in computing would lead to accelerated progress in computing by an equation like

    dx    2
    -- = x 
    dt

all dreams that died when people realized that Simon's General Problem Solver wasn't useful, when SAT and P vs NP fomulated, the rise and fall of expert systems, logic giving way to databases, etc. It appeals though to the kind of young person who's family made a great fortune and wants to jump right into philanthropy or to the person who went to San Francisco because they felt marginalized someplace else and are looking for meaning living in the shadow of "Big Tech"

>if you goal is to get ahead in our attention economy.

If a person wants a reputation as smart and able to make accurate predictions, why would he focus most of his public statments on a prediction that if he is right, no one will find out definitively that he was right because everyone will be dead?

I wouldn't be surprised if Yudkowsky would agree that the initial creation of "Information Technology" was an almost direct stepping stone (in evolutionary terms) on the path to AI, which in turn (he argues and I agree) will lead us towards superintelligence. I suppose that Kurzweil might take this even further, arguing that the path was set as soon as an early hominin picked up a stick and intentionally drew an arrow towards berry bushes in the ground.