Max Tegmark
Artificial intelligence is evolving rapidly and could become one of the most powerful forces in our future. If we want to survive and thrive alongside superintelligent machines, we must align their goals with ours.
The ultimate risk of artificial intelligence is not malice, but extreme competence applied to misaligned goals.
The book frames humanity as a transition point between Life 2.0, which can redesign its software through learning, and Life 3.0, which can redesign both software and hardware.
Technical safety research must focus on making systems learn, adopt, and retain human values as they upgrade themselves.