Epoch 05 by Laia Pose

///////////////////////////////// ///////////////////////////////////////////// ////////////////////////// // /////////////////////////// ///////////////////////////////////////////// ///////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////// ///////////////////////////////////////////// //////////////////////////////////////////// ///////////////// ///////////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////// //////////////////////////////////////////////////////////////////////// //////////////////////////// ///////////////////////////////////////////// /////////////////////////////// /////////////////////////////////////////// ///////////////////////////////////////////// //////////////// //////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////// ///////////////////////////////////////////// ////////////////////////// ////////////////////////////// ///////////////////////////////////////////// ///////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////// ///////////////////////////////////////////// //////////////////////////////////////////// ///////////////// ///////////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////// ///////////////////////////////////////////// ////////////////////////// // ///////////////////////////////////////////// ///////////////////////////////////////////// /////////// ////////////////////////// ///////////////////////////////////////////// ///////////////////////////////// /////////////////// ///////////////////////////////////////////// //////////////////////////////////////// ////////////////////////////////// ///////////////////////////////////////////// ///////////////////////// /////////////////////////////////////// ///////////////////////////////////////////// //////////////////// ///// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////

/// ///

Epoch 05

What will machines do

Creating intelligence greater than our own

when they are smarter than us?

//////////////////////////////////////////// ///////////////////////////////////////////// ///////////////// /////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////// //////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////// ///////////////////////////////////////////// //////////////////////////////////// /////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// ////////////////////////////// ///////////////////////////////////////////// /////////////////////////////// /////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// // ///////////////////////////////////////////// /////////////////////////////// /////////////////////////// //////////////////////////////////////////// //////////////////////////////// ////////////////////////////// //////////////////////////////////////// //////////////////////////////// ////////////////////////////////// //////////////////////////////////// ////////////////////////////////////////////// //////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////// ////////////////// ///////////////////////////////////////////// /////////////////////////////////////////// ////////////////////// ///////////////////////////////////////////// /////////////////////////////////////// /////////////////////////////// ///////////////////////////////////////////// ////////////////////////////// ////////////////////////////// ///////////////////////////////////////////// /////////////////////////////// //////////////////////////////////// ///////////////////////////////////////////// ///////////////////////// //////////////////////////////////////////// ///////////////////////////////////////////// ///////////////// /////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////// //////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////// ///////////////////////////////////////////// ////////////////////////////////////

///////////////////////////// ///////////////////////////////////////////// /////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// / ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////// /////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////// /////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////////////////////// ///////////////////////////////////////////// //////////////////////////////////// /////////////////// ///////////////////////////////////////////// ///////////////////////////////////////// //////////////// ///////////////////////////////////////////// //////////////////////////////////////////// //////////// ///////////////////////////////////////////// ///////////////////////////////////////////// // ////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// /////////////////////////////// ///////////////////////////////////////////// ///////////////////////////// /// ///////////////////////////////////////////// ///////////////////////////////////////////// /////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////// //////////////// ///////////////////////////////////////////// /////////////////////////////////// //////// /////////////// ///////////////////////////////////////////// ////////////////////////////////////////// // ///////////////////// ///////////////////////////////////////////// /////////////////////////////////////// ///////////////////////////// ///////////////////////////////////////////// /////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// / ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////// ////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////

Epoch 05

</Creating greater than

intelligence our own

///// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////// //////////////////////////// ///////////////////////////////////////////// //////////////////////////////// ///// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////// //////////////////////////// ////////////////////////////////////////////////// /////////////////////////// ////////// ///////////////////////////////////////////// /////////////////////////// ////////////////////// /////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// /////////////// ///////////////////////////////////////////// /////////////////////////////// ///////////// //////////////////////// ///////////////////////////////////////////// //////////////////////////////// /// ////////////////////////////////// ///////////////////////////////////////////// ////////////////////////// //////////////////////////////////////////// ///////////////////////////////////////////// //////////////// ////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////////////////////////////////////// ////// /////////////////////////////// ///////////////////////////////////////////// /////////////////////// ///// //////////////////////////////// ///////////////////////////////////////////// //////////////////////////// ///////// ///////////////////////////////////////////// /////////////////////////////////// /////////////// ////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// /////////////////// ///////////////////////////////////////////// ///////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// /////////////////// ///////////////////////////////////////////// /////////////////////////////////////////

CopyrightÂŠ 2015 by Stanford University. Published by Stanford University Press, Redwood City, California. All rights reserved. No part of this book may be reproduced in any form without written permission from Stanford University. For more information about visit our Web site at http://www.sup.org/ Epoch 05: Creating intelligence greater than our own.â&#x20AC;&#x201D;1st ed. Printed in the United States of America

TABLE

OF CONTENTS//

Introduction

Achieving the computational capacity of the human brain

The six epochs of evolution

Preparing the defenses

Economics of the Singularity

Will humans be able to control computers that are smarter than us?

Strong AI

Setting a date for the Singularity

The future of Jobs

Conference at a glance

Map

{ //

intro

//// / / / / / / / / / / / / / /

///////

/>It is hard to imagine what went through the minds of the first groups of

////

humans who controlled fire. But it is unlikely that they were aware of the deep and far-reaching consequences that that achievement would have in the fate of all their descendants. The bringing of light inside the caves also brought an improvement on the chances of survival and an increase in their capacity to change the World around them; it brought an age of enlightenment which still extends to our days. Similarly to our ancestors millions of years ago, we humans of today who are pushing the boundaries of Artificial Intelligence cannot imagine all the dramatic changes that generations to come will see with the help of new artificial allies. The Singularity—the moment where we build an intelligence exceeding our own—will be another turning point in human history, the start of a new era; an era of true enlightenment, which will make previous decades and centuries look dim and dull in retrospect. There are obviously differences between the situation of the hominids who tamed fire and that of ourselves. But there are also strong similarities. They were the few thousand members of a species who was struggling for survival, their very existence constantly jeopardized by famine, illness, and predators. We are now thousands of millions, but famine and illness remain a problem—and of a larger order of magnitude—and new challenges such as pollution, wars or economical collapse pose severe threats to our collective survival. So after millions of years, we are definitely still struggling for survival. We need to ask ourselves if chances are that we need to go beyond human to put an end to all that human suffering and pain which has travelled with our species for hundreds, thousands, and millions of years. It is however sensible to raise voices of concern about the disruption that intelligent machines will cause in our society. Some will see them as a Pandora box, that will let the evils of the world loose by being opened. And it is indeed certain that the transition will not be an easy one. But, have we ever thought of how many caves were burnt down in the taming of fire? How many of our ancestors died when those forces which were stronger than them—and are still stronger than us—escaped beyond their control? Yet should they have given up, and sacrificed all the innovations and well-being that came with fire because of that? However, something sets us apart from our ancestors, and that is our constant exposure to change, individually and as a society, and our incipient but increasing capacity to predict it and its consequences. The Singularity will bring changes, and some of them will probably be seen as a change for the worse. However, it is in our hands to start thinking about that moment from long before it happens, from today, from now, and to be ready for it; to include people of all backgrounds and fields in the discussion; to think up to its most far-reaching consequences; to see as far as possible beyond the day it happens. Events like the one that is going to happen in the forthcoming days are thus essential to allow a smooth transition to a society where super-human intelligence is the rule and not the exception, and everyone can enjoy all the benefits of it. I want to believe that with these meetings we are contributing a few grains of sand to the foundations of this new society, as members of academia, industry or just free thinkers. And I also want to believe that, many centuries from now, our descendants will look at us with gratitude, in the same sense that we should be grateful for those who gave their time—and even their lives—to the taming of fire. Because, from their point of view, our ways of live will be closer to those of the people living in the caves than to their own.

/ / / /

{/Edgar Gonàlez Pellicer—PhD on Artificial Intellicenge

/{<0000_03

/ /

/< ACHIEVING THE COMPUTATIONAL \CAPACITY OF THE HUMAN BRAIN//

How does computational capacity compare to How does computational capacity compare to human memory capacity? It turns out that we arrive at similar time-frame estimates if we look at human memory requirements. The number of “chunks” of knowledge mastered by an expert in a domain is approximately 105 for a variety of domains. These chunks represent patterns (such as faces) as well as specific knowledge. For example, a world-class chess master is estimated to have mastered about 100,000 board positions. Shakespeare used 29,000 words but close to 100,000 meanings of those words. Development of expert systems in medicine indicate that humans can master about 100,000 concepts in a domain. If we estimate that this “professional” knowledge represents as little as 1 percent of the overall pattern and knowledge store of a human, we arrive at an estimate of 107 chunks. While human neurons are wondrous creations, we wouldn’t (and don’t) design computing circuits using the same slow methods. Despite the ingenuity of the designs evolved through natural selection, they are many orders of magnitude less capable than what we will be able to engineer. As we reverse engineer our bodies and brains, we will be in a position to create comparable times faster than our naturally evolved systems. Our electronic circuits are already more than one million times faster than a neuron’s electrochemical processes, and this speed is continuing to accelerate. Most of the complexity of a human neuron is devoted to maintaining its life-support functions, not its information-processing capabilities. Ultimately, we will be able to port our mental processes to a more suitable computational substrate. Then our minds won’t have to stay so small.

human

computationmemory capacity?

computing, vacuum tubes, discrete transitions, and integrated circuits) that have provided exponential growth to the price-performance and capabilities of computation. Each time a paradigm reached its limits, another paradigm took its place. We can already see the outlines of the sixth paradigm, which will bring computing into the molecular third dimension. Because computation underlies the foundations of everything we care about, form the economy to human intellect and creativity, we might well wonder: are there ultimate limits to the capacity of matter and energy to perform computation? If so, what are these limits, and how long will it take to reach them? Our human intelligence is based on computational processes that we are learning to understand. We will ultimately multiply our intellectual powers by applying and extending the methods of human intelligence using the vastly greater capacity of nonbiological computation. So to consider the ultimate limits of computation is really to ask: what is the destiny of our civilization? A common challenge to the ideas presented in this book is that these exponential trends must reach a limit, as exponential trends commonly do. When a species happens upon a new habitat, as in the famous example of rabbits in Australia, its numbers grow exponentially for a while. But it eventually reaches the limits of that environmentâ&#x20AC;&#x2122;s ability to support it. Surely the processing of information must have similar constrains. It turns out that, yes, there are limits to computation based on the laws of physics. But these still allow for a continuation of exponential growth until nonbiological intelligence is trillions of trillions of times more powerful than all of human civilization today, contemporary computers included.

How al capacity

does compare to to

Weâ&#x20AC;&#x2122;ve already had five paradigms (electromechanical calculators, relay-based

/{<0000_07

∈

�

⦶

� ⪖

⤢ ⩔

�

△ � ⩏

⍑ �

♢ �

� ∳

�

Epoch 1 Physics and Chemistry Information in atomic structures

Epoch 2 Biology

.{; biology

Information in DNA

//BR< Epoch 3 Brains Information in neural patterns

>ains evol_ve Epoch 4 Technology Information in hardware and software designs

#techno/logy evolves

SIX

//EPOCHS EVOLUTION\\

<01/ Epoch One: Physics and Chemistry.

limits the

Evolution is a process of creating patterns of increasing order, the emphasis in this section is on the concept of patterns. The evolution of patterns that constitutes the ultimate story of our world. Evolution works through indirection: each stage or epoch uses the information-processing methods of the previous epoch to create the next. Here we conceptualize the history of evolution—both biological and technological—as occurring in six epochs. As we will discuss, the Singularity will begin with Epoch Five and will spread from Earth to the rest of the universe in Epoch six.

the

/THE OF

wo rld. Shopenhauer.

Everyone —Arthur

takes

the

limits

his

own

vision

for

We can trace our origins to a state that represents information in its basic structures: patterns of matter and energy. Recent theories of quantum gravity hold that time and space are broken down into discrete quanta, essentially fragments of information. There is controversy as to whether matter and energy are ultimately digital or analog in nature, but regardless of the resolution of this issue, we do know that atomic structures store and represent discrete information. A few hundred thousand years after the Big Bang, atoms began to form, as electrons became trapped in orbits around nuclei consisting of protons and neutrons. The electrical structure of atoms made them “sticky”. Chemistry was born a few million years later as atoms came together to create relatively stable structures called molecules. Of all the elements, carbon proved to be the most versatile; it’s able to form bonds in four directions (versus one to three for most other elements), giving rise to complicated, information-rich, three-dimensional structures. The rules of our universe and the balance of the physical constants that govern the interaction of basic forces are so exquisitely, delicately, and exactly appropriate for the codification and evolution of information (resulting in increasing complexity) that one wonders how such an extraordinary unlikely situation came about. Where some see a divine hand, others see our hands—namely, the anthropic principle, which holds that only in a universe that allowed our own evolution would we be here to ask such questions. Recent theories of physics concerning multiple universes speculate that new universes are created on a regular basis, each with its own unique rules, but that most of these either die out quickly or else continue without the evolution of any interesting patterns (such as Earth-based biology has created) because their rules do not support the evolution of increasingly complex forms. It’s hard to imagine how we could test these theories of evolution applied to early cosmology, but it’s clear that the physical laws of our universe are precisely what they need to be to allow for the evolution of increasing levels of order and complexity. \02> Epoch Two: Biology and DNA. In the second epoch, starting several billion years ago, carbon-based compounds became more and more intricate until complex aggregations of molecules formed self-replicating mechanisms, and life originated. Ultimately, biological systems evolved a precise digital mechanism (DNA) to store information describing a larger society of molecules. This molecule and its supporting machinery of codons and ribosomes enabled a record to be kept of the evolutionary experiments of this second epoch.

/{<0000_011

189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&746576 4783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 4890 9824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 2774093 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876 ^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746576458 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//7889737465764587 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 48903 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 001010101 20347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576? //7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 10 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u383847474 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 4890347832 189756253748098^9435 #!122200 & 00 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6 Each epoch continues evolution/345 of information through paradigm65538393854748404949 shift 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 189756253748098^9435 #!122200 & the 003345^5@998654/ 0 47832438756&76534 277409361!a 101013836753 55858..>><<0))({^ 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347(That 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 to a further level of “indirection”. is, evolution, used the results of one8 243875 6&76 534 277409361!001 43465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& 803345^5@99||86543456~`89 764 3DNA-guided 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//7889 epoch to create **8*(*^&6^788w83e772}0347 the next.) For example, in the third epoch, evolution &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 organisms that could detect information with their own organs5654#432420)00 000<>98767656 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987produced 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3sensory 232143} 65(&&888** 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 and 89 523nervous 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 and process ans store that information in their own brains systems.

You know, things are going to be really different!... No, no, I mean really different! —Mark Miller (Computer Scientist) to Eric Drexler, around 1986.

_03< Epoch Three: Brains.

These were

61!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) #!576?//78897374657645873759576?//788973 made <<783243 possible b second-epoch mechanisms (DNA and0435 epigenetic 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//788973746576458737598243795 information of proteins89and fragments that control gene expression), which >98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 523 RNA 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000 7645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654/ (indirectly) enabled and defined||20347 third-epoch information-processing mechanisms 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 43$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 (the brains and nervous systems of organisms). The third epoch started with6&76 the534 277409361!00101010#$$5^ 1 657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 65 of early animals to98767656576?/ recognize patterns, which10still accounts for the 101010101 vast 7656576?//7889010101010101 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143}ability 65(&&888** 5654#432420)00 000<> /7889731 0010101010100 000101010 1 01010101 010010 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 0<>9876765657 6?//788973746576458737 59 our 82437~`89 764 3Ultimately, 232143} 65(&&888** 5654#432420)00 majority of the activity in brains. our own species000<>98767656576?/7656576?/ evolved the ability /7889/78897 3746576458737 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 0435 #!576? to create abstract mental models of the world we experience and to4323#$$5^2143465(&&888**5654#432420) contemplate

the rational of these models.952348\\||9034 We have the ability to redesign the#!122200 world& 003345^5@998654//345 0 4783 73759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 implications 76?//788973746#$$5^ 764587375982437 <<783243 189756253748098^9435 3759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420 in our own minds and to put these ideas into action. 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 000 1010000 00101111174657645873759824379523 4890 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003 Combining endowment rational 10and abstract thought our7656576?/ opposable 543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>the 98767656576?/ /7889731 of 0010101010100 000101010 1 01010101 010010 with 101010101 /78890101010101010101010000 001011111 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 thumb, 6&76 534 our 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 species ushered in the fourth epoch and4323#$$5^2143465(&&888**5654#432420) the next level of indirection:0435 #!576?//78897374657645873 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 the evolution of human-created technology. This started out with simple mecha-8 243875 6&76 534 277409361!00 76?//788973746576458 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 7 nisms and developed into elaborate automata (automated mechanical machines). 873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 465764 //78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437computational 952348\\||9034 <<783243 189756253748098^9435devices, #!122200 & tech003345^5@998654//345 0 47832438756 , with sophisticated and communication 24379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 27 7409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** nology was itself capable of sensing, storing, and evaluation elaborate patterns of information. To compare the rate of progress5654#432420)00 000<> 98767656 $5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 27740 of the biological evolution intelligence to#!122200 that of&technological evolution, consider277409361! that the101013836753 most advanced mammals have 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243of 189756253748098^9435 003345^5@998654//345 0 47832438756&76534 655383938547656576?/ /7889748404949 55858..>><<0))({^ %54$201 232143} 65(&&888** 5654#432420)00 6?//788973746576458737 59 82437~`89 764 3hundred 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?/ added 000<>9876765657 about one cubic inch of brain matter every thousand years, whereas we are roughly/7889/78897 doubling374657645873759824379523 the computational 4890347832 189756253748098^94 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 189756253748098^9435 #!122200 003345^5@998654/ /345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ capacity of computers every year. Of course, neither brain size nor&computer capacity is the sole determinant of intelligence, 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010 but#!576?/ they/78897374657645873759576?/ do represent enabling factors. 465(&&888**5654#432420) 0435 /78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 18975625 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 764 3 232143}on 65(&&888** 000<>9876765657 6?//7889737465764 key milestones of both biological evolution and human803345^5@99||86543456~`89 technological development a single5654#432420)00 graph plotting &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654/ / 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?/ /788973746576458737 59 82437~`89 764 3 232143} 65(&&888** both the x-axis (number of years ago) and the y-axis (the paradigm-shift time) on logarithmic scales, we find a reasonably5654#432420)00 000<>98767656 10 000101010 1 01010101 010010 101010101 765657 straight line (continual acceleration), with biological evolution leading to human-directed development. 6?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523directly 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 565 3824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//788 4323#$$5^21434 65(&&888**5654#432420) 0435 #!576?/ /78897374657645873759576?/ /788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/ 489034783 Looking ahead several decades, the Singularity will begin with the fifth epoch. It will result from the merger of/78897374657645873759824379523 the vast 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 76 knowledge embedded in our own with the vastly greater capacity, 8speed, and of our000<>9 tech-8767656576?//7889737 4697374 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523brains 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 243875 6&76 534knowledge-sharing 277409361!0010101 &888** ability 5654#432420)00 7644323#$$5^21434 65(&&888**5654#432420) /78897374657645873759576?/ /788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/ 4890347 nology. The 0435 fifth#!576?/ epoch will enable our human-machine civilization to transcend the human brain’s limitations of a/78897374657645873759824379523 mere 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 76 873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?/ /7889737 46973 hundred trillion extremely slow connections. #2!#$$5^&**8*(*^&6^78 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 will allow us to5654#432420)00 overcome age-old human/7889731 problems and vastly amplify1 human creativity. We 7656576?/ will preserve and 4 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 000<> 98767656576?/ 0010101010100 10 000101010 01010101 010010 101010101 /78890101010101010101010000 00101111174657 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) enhance the intelligence that evolution has 6&76 bestowed on us while overcoming the profound limitations of biological evolution.0435 #!576?//78897374657645873 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374 5654#432420)00 000<>98767656576?/7656576?/ /7889/78897 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523so 489034783243$##2!#$$5^&**|8*(*^ But the Singularity will374657645873759824379523 also amplify the ability to act on our destructive inclinations, its full story has not yet&6^788w83e7720347 been written.8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$201193039 6?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 In the aftermath of thee7720347 Singularity, intelligence, derived from its biological origins in 277409361!0010101 &888** 565 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 8w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@998654345 9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 human brains and its technological origins in human ingenuity, will begin to saturate 654// 00000 <>987 3} 65(&&888 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 0010111117465 the matter and energy in its midst. It will achieve this by reorganizing matter and 46973746#$$5^ 764587375982437 952348\\9034 <<783243 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361 !0010101375 ““982437952 382437952 energy to provide an optimal level of computation to spread out form its origin on Earth. 122200 & 0033

\04\ Epoch Four: Technology.

Ultimately

If we place

05// Epoch Five: The Merger of Human Technology with Human Intelligence.

The Singularity

{06: Epoch Six: The Universe Wakes Up

We currently understand the speed of light as a bounding factor on the transfer of

information. Circumventing limit has to be regarded as highly speculative, but there 45^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?/ /788973746576458737 59 82437~`89 764this 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?/ /788858..>><<0))({^ %54$2011930393 <u38 90101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 534subtle 277409361!0010101 &888** 5654#432 are hints that thise7720347 constraint may be able to be superseded. If there are 6&76 even 420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 34783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 2 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 ultimately #!576?//78897374657645873759576?/ /788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420) deviations, we will harness this superluminal ability. 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0) civilization the rest the universe with its creativity and intelli<u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** infuses 5654#432420)00 000<>of 98767656576?/ /7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7

Whether our

gence quickly or slowly depends on its immutability. In any event the “dumb” matter and

656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00 10101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{32 mechanisms of the universe will be transformed into exquisitely sublime forms of intel43$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 000 ligence, which will constitute the sixth epoch in the evolution of patterns of information. 00 <>987 3} 65(&

This is the ultimate destiny of the Singularity and of the universe.

&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u38 3847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576 ?//7889010 101010101010101000 0 0010111117465764587 3759824379523 48903478 3243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$## 2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!001010

7645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 48903 034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//7889737465764587375 361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 67656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345 8 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 3 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 73759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 34783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654# 432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 10100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e77 000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//78896 7656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 03345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** ^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 10101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^21 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ 4 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 6576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 8 243875 6&76 534 2774093

374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000< 0003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//7889737465 // 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 10101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374 54// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 10101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 759824379523 489034783243$# #2!#$$5^&**8*(*^&6^78 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 10101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 ?//788973746576458

32438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//7889737465764587 0)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 03345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101010 3345^5 @99||8 174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 3759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 4657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 0101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>987676565 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645 45873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576? 6&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//788973746576458737598

6576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$ 09361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 Merger of Technology and Human Intelligence 8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** ^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?/ 0010101010100 10 000101010 1 01010101 010010 The methods of biology (including human intelligence) are integrated into the/7889731 (exponentially expand0101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143 ing) human technology base 53748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?/ /7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ 4 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 6576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100

Epoch 5

54#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//7889737465764587375982437952 55383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& 89/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 3243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 64 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 001011111746576458 46#$$5^ 764587375982437 952348\\9034 <<783243 43$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89 783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 64 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645 3746#$$5^ 764587375982437 952348\\9034 <<783243 0<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# 9 76 7645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 3759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 4657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888**

_05/< technology masters the methods of biology (including human intelligence)

>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 93 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 765657 54#432420)00 56~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 48 8** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 57645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737

UNI-verse

83847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//788

Epoch 6

277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 8w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 The 89 523Universe 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 Wakes Up ))({^ %54$2011930393

Patterns of matter and energy in the universe become

saturated with intelligent processes and knowledge

/{<0000_013

</PREPARING DEFENSES;

//THE

Intelligence is inherently impossible to control. STRONG AI PROMISES TO CONTINUE THE EXPONENTIAL GAINS OF HUMAN CIVILIZATION BUT THE DANGERS IT PRESENTS ARE ALSO PROFOUND PRECISELY BECAUSE OF ITS AMPLIFICATION OF INTELLIGENCE.

01/ We don't have to look past today to see the intertwined promise and peril of technological advancement. Imagine describing the dangers (atomic and hydrogen bombs for one thing) that exist today to people who lived a couple of hundred years ago. They would think it mad to take such risks, But how many people now would really want to go back to the short, brutish, disease-filled, poetry-stricken, disaster-prone lives that 99 percent of the human race struggled through a couple of centuries ago? 02/ We may romanticize the past, but up until fairly recently most of humanity lived extremely fragile lives in which one all-too-common misfortune could spell disaster. Two hundred years ago life expectancy for females in the record holding country (Sweden) was roughly thirty-five years, very brief compared to the longest life expectancy today— almost eighty-five years, for Japanese women. Life expectancy for males was roughly thirty-three years, compared to the current seventy-nine years in the record-holding countries. It took half the day to prepare the evening meal, and hard labor characterized most human activity. There were no social safety nets. Substantial portions of our species still live in this precarious way, which is at least one reason to continue technological progress and tho the economic enhancement that accompanies it. Only technology, with its ability to provide orders of magnitude of improvement in capability and affordability, has the scale to confront problems such as poverty, disease, pollution, and the other overriding concerns of society today. 03/ People often through of three stages in considering the impact of future technology: ave and wonderment at its potential to overcome age-old problems; then a sense of dread at a new set of grave dangers that accompany these novel technologies; followed finally by the realization that the only viable and responsible path is to set a careful course that can realize the benefits while managing the dangers. 04/ Needles to say we have already experienced technology’s downside—for example, death and destruction from war. The crude technologies of the first industrial revolution have crowded out many of the species that existed on our planet a century ago. Our centralized technologies (such as buildings, cities, airplanes, and power plants) are demonstrably insecure. 05/ The “NBC” (nuclear, biological and chemical) technologies of warfare have all been used or been threatened to be used in our recent past. The far more powerful GNR technologies threaten us with new, profound local and existential risks. If we manage to get past the concerns about genetically altered designer pathogens, followed by self-replicating entities created through nanotechnology, we will encounter robots whose intelligence will rival and ultimately exceed our own. Such robots may make great assistants, but who’s to say that we can count on them to remain reliably friendly to mere biological humans?

R F

/ ;

/{<0000_015 \<||>.\\|00000}

//OF _SINGULARITY

< / 0 0 2 . .

“…human labor would no longer

_ECONOMICS /THE

earn most income.”

>>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> >>>>>>>>>>>>> > <>><<> >>>>>>>>>>> >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<< Stuffed into skyscrapers by the billion, brainy bugbots will ><>>><<< > >>>>>>><<<<<<<<<<< be the knowledge workers of the future ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> OUR GLOBAL ECONOMY WOULD STUPEFY A ROMAN MERCHANT AS MUCH AS THE ROMAN ECONOMY WOULD HAVE CON->>>>>>>>>>>>> >>>> FOUNDED A CAVEMAN. BUT WE WOULD BE SIMILARLY AMAZED TO SEE THE ECONOMY THAT AWAITS OUR GRANDCHILDREN,>>>>>>><<<<<<<<<<< ><>>><<< FOR I EXPECT IT TO FOLLOW A SOCIETAL DISCONTINUITY MORE DRAMATIC THAN THOSE BROUGHT ON BY THE AGRICULTURAL >>>>>>><<<<<<<<<<< AND INDUSTRIAL REVOLUTIONS. THE KEY, OF COURSE, IS TECHNOLOGY. ><>>><<< >>>>>>>>> >>>>>>>>>>>>> A revolutionary speedup in economic growth requires an unprecedented and remarkable enabling tool. Machine intelligence <>><<> on a human level, if not higher, would do nicely. Its arrival could produce a singularity--an overwhelming departure from> > > > > > > > > > > >>>>>>><<<<<<<<<<< prior trends, with uneven and dizzyingly rapid change thereafter. A future shock to end future shocks. ><>>><<< >>>>>>>>> Yes, this theory of mine is a social and economic one, and therefore not as unfailingly accurate or testable as one in the>>>>>>>>>>>>> >>>> physical sciences. Nevertheless, social scientists routinely make short-term forecasts that hit the mark, and economists>>>>>>><<<<<<<<<<< ><>>><<< often offer insightful forecasts about unprecedented situations. >>>>>>><<<<<<<<<<< So we will outline how economists view technological change. In so doing, we ><>>><<< >>>>>>>>> hope to explain why it’s reasonable to view past history as a series of abrupt, >>>>>>>>>>>>> >>> seemingly unheralded transitions from one economic era to another, transitions >>>>>>>>>>>>> >>>> marked by the sudden and drastic increase in the rate of economic growth. We >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> will then show why another singularity is perhaps just around the corner. Finally, >>>>>>>>>>>>> >>>> we will outline its possible consequences. >>>>>>><<<<<<<<<<< A complex device, like a tractor or a building, can have thousands of parts, and ><>>><<< each part can rely on dozens of technologies. Yet in most cases even a spectacular >>>>>>><<<<<<<<<<< gain in the quality of one part bestows at best only a small improvement on the ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >> whole. Keep improving a part in successive increments of equal degree and you’ll >>>>>>>>>> >>>>>>> get ever smaller gains to the whole. This is the law of diminishing returns, and >>< ><< ><>>><<< >> it applies not only to devices and organizations but to entire industries. Consider <>><<> >>>>>>>> your personal computer: every couple of years its power-to-cost ratio has doubled, >>>>>>>>>>> and yet as you go from one generation to the next, you probably notice only a >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> small improvement as you plug away on your word processors and spreadsheets. >>>>>>>>>>>>> It turns out that most of these small, innovative gains come not from research >>>>>>><<<<<<<<<<< labs but from hands-on builders and users. So the more a thing gets used, the ><>>><<< >>>>>>>>> more it tends to improve. It doesn’t matter whether that thing is a physical device, > > > > > > > such as a car, or a social organization, such as a corporation. >>>>>>>>>>>>> >>>>>>><<<<<<<<<<< If any large system of interacting parts tends to improve by smooth gradations, ><>>><<< >>>>>>>>> then we should expect systems of systems, with their larger number of compo>>>>>>>>>>>>> >>>> nents and interactions, to improve even more smoothly. By this reasoning, the >>>>>>><<<<<<<<<<< world economy should improve most smoothly of all. The world economy consists ><>>><<< >>>>>>>>> of the largest number of interacting parts of any man-made system, and every>>>>>>>>>>>>> >>>>>>><<<<<<<<<<< one not stranded on an uncharted island contributes to the improvements in all ><>>><<< >>>>>>>>> those parts by using them. Finally, in each economic era the question of whether >>>>>>>>>>>>> >>> growth speeds up or slows down depends on two competing factors. Deceleration >>>>>>><<<<<<<<<<< typically ensues as innovators exhaust the easy ideas—the low-hanging fruit. But ><>>><<< >>>>>>>>> acceleration also ensues as the economy, by getting larger, enables its members >>>>>>>>>>>>> >>>> >>>>>>><>> >>>>>>> to explore an ever-increasing number of innovations. >>< ><< ><>>><<< >> <>><<> >>>>>><<<<<<<<<<<< ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<< ><>>><<< >>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<<

/ 0 0

<<<<<>> >>< ><< >>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>> <<<<<>> >>< ><< >>>>>>>>> <<<<<>> >>< ><< >>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>< The world economy, which now doubles in 15 years or so, would soon double in >>>>>>>>>>>>> >>>>>>>>>>> somewhere from a week to a month. <<<<<>> >>< ><< We have the tools to measure the world’s economic product not only >>>>>>>>>>< <>><<> for today—it’s about US $50 trillion per year—but also for times long >>>>>>>>>>>>>>>>>> past. A few years ago Angus Maddison, an economic historian at the <<<<<>> >>< ><< University of Groningen, in the Netherlands, plotted a graph of world >>>>>>>>>>>>>>>>>> <<<<<>> >>< ><< economic product--basically everything of value produced glob>>>>>>>>>>< <>><<> ally: bananas, submarines, magazine articles, you name it. It shows >>>>>>>< <>><<> that from 1950 to 2003, growth was relatively steady. During that time, >>>>>>>>>>>>>>>>>> despite enormous technical change, no particular technology left much <<<<<>> >>< ><< of a fingerprint on the data; no short-term accelerations in growth could >>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>> be attributed to this or that technological development. Also, Maddison’s data <<<<<>> >>< ><< offer little support for the idea that innovation and growth have been accelerat>>>>>>>>>>> ing recently. Now look at the data for world product over the past 7,000 years, <<<<<>> >>< ><< estimated by Bradford DeLong, an economic historian at the Univer>>>>>>>>>>< <>><<> sity of California, Berkeley. The data here tell a somewhat different story. >>>>>>>>>>>>>>>< >>>>>>>>>>>>> For most of that time, growth proceeded at a relatively steady exponen>>>>>>>>>>> tial rate, with a doubling of output about every 900 years. But within the <<<<<>> >>< ><< past few centuries, something dramatic happened: output began dou>>>>>>>>>>< <>><<> bling faster and faster, approaching a new steady doubling time of about >>>>>>>>>>>>>>>>>> 15 years. That’s about 60 times as fast as in the previous seven millennia. <<<<<>> >>< ><< >>>>>>>>>>>>>>>We call this transition the Industrial Revolution, but that does not mean we <<<<<>> >>< ><<understand it well or even know precisely how and why it arose. But whatever the >>>>>>>>>>< <>><<> Industrial Revolution was, clearly it was an event worthy of the name ”singularity.” >>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>>If we look further back, we see what appears to be at least one previous singu<<<<<>> >>< ><<larity--the transition to an economy based on agriculture. And slow as economic >>>>>>>>>>< <>><<>growth during the agricultural era may seem in the aftermath of the Industrial >>>>>>>>>>>>>>>>>>Revolution, it was actually lightning fast compared with that of the economic era <<<<<>> >>< ><< that came before, which was based on hunting and gathering. >>>>>>>>>>>>>>>>> <<<<<>> >>< ><< >>>>>>>>>>< <>><<> >>>>>>>>< <>><<> ><<<<<<<<<<<<<<<>> >>>>>>>>>>>>>>>>>< In the roughly 2 million years our ancestors lived as hunters and gatherers, >>>>>> >>>>>>>>>>> the population rose from about 10 000 protohumans to about 4 million modern >>>>>>>>>>> humans. If, as we believe, the growth pattern during this era was fairly steady, then <<<<<>> >>< ><< >>>>>>>>>>< <>><<> the population must have doubled about every quarter million years, on average. >>>> Then, beginning about 10 000 years ago, a few of those 4 million humans began to <<<<<>> >>< ><< settle down and live as farmers. The resulting communities grew so fast that they >>>>>>>>>>< <>><<> quickly accounted for most of the world population. From that time on, the farming > > > > > > > population doubled about every 900 years—some 250 times as fast as before. >>>>>>>>>>>>> <<<<<>> >>< ><< Our understanding of the existence, nature, and relevance of these transitions >>>>>>>>>>< <>><<> clearly becomes more speculative the further back we look in time. There may >>>>>>>>>>>>>>>>>> well have been two earlier singularities that started eras of this sort, although our <<<<<>> >>< ><< ability to identify them and weigh their relevance is very speculative. I suggest an >>>>>>>>>>< <>><<> era defined by the growth of the brain from the emergence of animal life to the first >>>>>>>>> <<<<<>> >>< ><< protohumans and perhaps an earlier era defined by the growth of the universe from >>>>>>>>>>< <>><<> shortly after the big bang to the first animals. >>>>>>>>>>>>>>>>>> <<<<<>> >>< ><< >>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>> ><<<<<<<<<<<<<<<>> >>>>>>>>>>>>>>>>>< >>>>>>>>>>>>> <<<>> >>< ><< >>>>>>>>>>< <>><<> /{<0000_017 >>>>>>>>>>>>>>>>>> <<<<<>> >>< ><< >>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>> <<<<<>> >>< ><<

////

We could set the date for the Singulari â&#x20AC;&#x201D;representing a profound and disrupt human capabilityâ&#x20AC;&#x201D;as 2045. The nonbi created in that year will be one billion t than all human intelligence today.

ity tive transformation in iological intelligence times more powerful

\\ //

360º

//// / / / / / / / / / / / / / /

///////

360º

So we have perhaps five eras during which the thing whose growth is at issue—the universe, brains, the hunting economy, the farming economy, and the industrial economy—doubled in size at fixed intervals. Each era of growth before now, however, has eventually switched suddenly to a new era having a growth rate that was between 60 and 250 times as fast. Each switch was completed in much less time than it had taken the previous regime to double—from a few millennia for the agricultural revolution to a few centuries for the industrial one. These switches constituted singularities.

Whatever may have been the key innovations behind these transitions, it is clear

////

that they were far more potent than such familiar textbook examples of great innovations as fire, writing, computers, or plastics. Most innovations happen within a given growth era and do not change its basic nature, including its basic growth rate. A few exceedingly rare innovations, however, do suddenly change everything. One such innovation led to agriculture; another led to industry. Therefore, we must admit that another singularityâ&#x20AC;&#x201D;at least the third one, and perhaps the fifth, depending on how you countâ&#x20AC;&#x201D;could lie ahead. Furthermore, data on these previous apparently similar singularities are some of the few concrete guides available to what such a transition might look like. We would be fools if we confidently expected all patterns to continue. But it strikes me as pretty foolish to ignore the patterns we see. If a new transition were to show the same pattern as the past two, then growth would quickly speed up by between 60 and 250-fold. The world economy, which now doubles in 15 years or so, would soon double in somewhere from a week to a month. If the new transition were as gradual (in power-law terms) as the Industrial Revolution was, then within three years of a noticeable departure from typical fluctuations, it would begin to double annually, and within two more years, it might grow a million-fold. If the new transition were as rapid as the agricultural revolution seems to have been, change would be even more sudden. Though such growth may seem preposterous, consider that in the era of hunting and gathering, the economy doubled nine times; in the era of farming, it doubled seven times; and in the current era of industry, it has so far doubled 10 times. If, for some as yet unknown reason, the number of doublings is similar across these three eras, then we seem already overdue for another transition. If we instead compare our era with the era of brain growth, which doubled 16 times before humans appeared, we would expect the next transition by around 2075. What innovation could possibly induce so fabulous a speedup in economic growth? It is easier to say what could not. Because of diminishing returns, no change that improved just one small sector of the economy could do the trick. In advanced countries today, farming, mining, energy, communications, transportation, and construction each account for only a small percentage of economic activity. Even so extraordinary an innovation as radical nanotechnology would do no more than dramatically lower the cost of capital for manufacturing, which now makes up less than 10 percent of U.S. GDP. No, the next radical jump in economic growth seems more likely to come from something that has a profound effect on everything, because it addresses the one permanent shortage in our entire economy: human time and attention. They are by far the most productive components of todayâ&#x20AC;&#x2122;s economy. About twothirds of all income in the rich countries is paid directly for wages, and much of the remaining third represents indirect costs of labor. (For example, corporate income largely reflects earlier efforts by entrepreneurs.) So any innovation that could replace or dramatically improve human labor would be a very big deal.

/ / / /

/{<0000_021

/ /

much retrain.

Previous technological revolutions happened more slowly, so people had longer to â&#x20AC;&#x201D; Tom Standage

One of the pillars of the modern singularity hypothesis in its many forms is that intelligence is a general elixir, able to cure many if not all economic ailments. Typically, this belief is expressed in the form of an argument that the arrival of very intelligent machines will produce the next singularity. Some people hope this arrival will follow a new Einstein, who will discover a powerful general theory of intelligence applicable to those machines. Others envision an â&#x20AC;?intelligence explosionâ&#x20AC;? via a series of powerful design innovations, beginning with one that would make machines smart enough to help us quickly find a second innovation, allowing even smarter machines, and so on. A few even imagine innovations so unprecedentedly potent that a single machine embodying the first innovation could go through the entire innovation series by itself, unnoticed, within a week, and then take over the world. There are many views on how intelligence might arise in a machine. One argument holds that hardware is the critical limiting factor and predicts that human-level machine intelligence will come soon after we have computer hardware whose performance is comparable with that of the human brain.

/{<0000_023

>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< > >>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< >>>>> >>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>>>> >>>>>>>>>< <>><<> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>> >>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> ><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< > >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< >>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<<<<<<<>> >>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< > >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>> ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< >>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>> >>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> ><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< >>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<<<<<<<>> >>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< > >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>> ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< >>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> <<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>< <>><<>>>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> <<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<<<<<<>> >>< ><< >>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<<<<<<>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< >> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>>>><<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>> ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< > >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< >>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< > >>>>>><<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< <<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> > ><>>><<< >>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<>> ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< >>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>><<<<<<<<<<<<<<>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> ><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <> >>>>>>>>>>>>>> >>>>>>><<<<<<>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> < >>>>>>>>>>>>>>>>>>< <>><<>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< > >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> <>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< >>> >>>>>>><<<<<<>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>< <>><<><<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>

189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&* 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$2011930 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//7889737 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&* 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$2011930 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654# 432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&765 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 0347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576? //7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409 4657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@9986 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//78896 7656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 7 98767656576?/7656576?//7889/78897 374657645873759824379523 4890347832 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6553839385474840494 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 48903478 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 189 %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000 &6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 48 43465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 56576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 48 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e772034 IF 23#$$5^21434 HUMANS65(&&888**5654#432420) GO ON TO CREATE INTELLIGENCE, WILL IT PRESENT A 82437~`89 SIGNIFICANT DANGER US? SEVERAL 0435ARTIFICIAL #!576?//78897374657645873759576?/ /788973746576458774657645873759 764 3 232143} 65(&&888**TO 5654#432420)00 000<>98767656576?//7889737465764587 534 277409361!0010101375 ““982437952 3824379523122200 003345^5@998 / 00000 <>987 3} 65(&&888** 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&8 TECHNICAL LUMINARIES HAVE BEEN OPEN& AND CLEAR654/ WITH RESPECT TO THIS5654#432420)00 POSSIBILITY: ELON MUSK, THE FOUNDER 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//788901010101010101010 w83e7720347 8 243875 6&76 534 2774093 OF SPACEX, HAS EQUATED IT TO “SUMMONING THE DEMON”; STEPHEN HAWKING WARNS IT “COULD SPELL THE END OF

// WILL HUMANS BE ABLE TO CONTROL COMPUTERS THAT ARE SMARTER> THAN US {?;

THE HUMAN RACE”; AND BILL GATES AGREES WITH MUSK, PLACING HIMSELF IN THIS “CONCERNED” CAMP. 61!0010101 &888** 5654#432420)00 000<>9 8767656576?/ /7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//7

764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ 24379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 once 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?/ that the AI is switched on and gradually assumes more 46576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{ and6?/ more responsibility in running our65(&&888** brave, newfangled the while 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 /788973746576458737 59 82437~`89 764 3 232143} 5654#432420)00 world—all 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$20 000<> 98767656576?//7889731 0010101010100 10 000101010 improving 1 01010101 010010 101010101 7656576?/ /78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w upon its own initial design—what’s to stop it from realizing that human 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 43$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 8973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/ /78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034 existence is an inefficiency or perhaps even an annoyance? Perhaps the AI would 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$20 to get rid101010101 of us if,7656576?/ as Musk has suggested, it decided that the best way489034783243$##2!#$$5^&**8*(*^&6^788w to get 000<> 98767656576?//7889731 0010101010100 10 000101010want 1 01010101 010010 /78890101010101010101010000 00101111174657645873759824379523 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 0<>9876765657 /788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/765657 rid of spam email “is to get6?/ rid of humans” altogether. 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232 01010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 89 523 8 243875 value ine7720347 warning of48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 the dystopian potential of certain trends or 6&76 534 277409361!0010101 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//788973746576458

Their worry is

No doubt there is

technologies. George Orwell’s 1984, for example, will always stand as a warning 73759576?//78897374657645873759824379523824379523122200 & 003345^5@998654/ / 00000 67 6 565 76?/ /788973746#$$5^ 764587375982437 952348\\||9034 <<783243 against technologies or<>987 institutions that remind us of Big Brother. But could the189756253748098^9435 #!122200 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^ 5654#432420)00 000<>98767656576?/7656576?//7889/78897anxiety 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 about AI just be the unlucky fate of every new radical technology that 7645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7 promises a brighter future: to be accused as764a3 harbinger of doom? It certainly 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//78885 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** wouldn’t 5654#432420)00 000<> 98767656576?//7889731 0010101010100 000101010 1 01010101 010010 101010101 be unprecedented. Consider the fear10 that once surrounded another7656576?/ pow-/78890101010101010101010000 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||8 543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?/ /7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890 erful technology: recombinant DNA. “A self-improving agent must reason about &**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 the behavior of its smarter successors in abstract 764587375982437 terms. If it could predict<<783243 their 189756253748098^9435 #!12220 45873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654/ / 00000 <>987 67 6 565 76?//788973746#$$5^ 952348\\||9034 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 /788967656576?/ /78897374657645873759824379523 actions in000<>9877656576?/ detail, it would already be as smart as them.” 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^ 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 45873759576?//788973746576458774657645873759 82437~`89In 764 1972 3 232143}Paul 65(&&888** 000<>98767656576?/ /788973746576458 189756253748098^9435 Berg,5654#432420)00 a leading biochemist at Stanford University, created a#!122200 strand& 003345^5@998654//345 0 4783 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//788901010101010 DNA5654#432420)00 that contained the genes of a4657645873759824379523 virus potentially489034783243$##2!#$$5^&** implicated in cancer &6^788w83e7720347 8 243875 6&76 534 277409361!0010101 of &888** 000<>9 all 8767656576?/ /7889737 8*(*^&6^788w83e7720347 8 243875 759576?//78897374657645873759824379523824379523122200 and & 003345^5@998654/ / 00000 <>987 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & fused it with genes of67a6bacteria that dwells in the human intestine: E. coli. <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99| 7409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** This caused an uneasy stir. But the alarms really went off when researchers 0000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &88 created the first recombinant DNA, demonstrating the ability to clone the DNA 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?/ /78897374657645873759824379523824379523122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?/ %54$20119303 93 to <u38384747489> 00~`89 764 3 232143} 65(&&888* of any organism, and to mix/7889748404949 and match 55858..>><<0))({^ genes together in order bring about 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?/ 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} novel forms of life. The Columbia biochemist Erwin Chargaff (whose work was 65(&&888** 5654#432420)0 5873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 invaluable to0 Watson and Crick’s discovery of 65538393854748404949 the double helix)55858..>><<0))({^ fired off a letter to 1000003345^5@ 189756253748098^9435 #!122200 & 003345^5@998654/ /345 47832438756&76534 277409361! 101013836753 %54$2011930393 <u383847474 89> 0003 0010101010100 10 000101010 1 01010101 010010 101010101the 7656576?/ /78890101010101010101010000 e7720347 89 523 48||90 editor of Science titled, “On00101111174657645873759824379523 the Dangers of Genetic489034783243$##2!#$$5^&**8*(*^&6^788w83 Meddling,” stressing “the 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143 465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?/ /78897374657645873759824379523824379523122200 & 003345^5@998654/ / argued 00000 <>987 67 6it565 76?/the /788973746#$$5^ 764587375982437 awesome irreversibility of what is being contemplated.” He that was 547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 4 pinnacle of brashness to play around/7889/78897 with the374657645873759824379523 order and stability489034783243$# that nature #2!#$$5^&**8*(*^&6^788w83e7720347 had, 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?/ 3#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?/ /788973746576458774657645873759 82437~`89 764 3 he 232143} 65(&&888** “will 5654#432420)00 000<>98767656576?/ /7889737465764587 over millions of years, maintained. “The future,” concluded, curse us for 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&8 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?/ /7889731 0010101010100 10 000101010 1 01010101 010010 101010101 765657 it.” A couple of months later, Liebe Cavalieri, then a biochemist at Cornell and a 6?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 2774 member of the Sloan-Kettering Institute for Cancer &Research, took/this argument 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?/ /78897374657645873759824379523824379523122200 003345^5@998654/ 00000 <>987 67 6 565 76?//788973746#$$5^ 764587 383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3Times, 232143} 65(&&888** 000<>9877656576?/ /788967656576?/ to the pages of The New York relating 5654#432420)00 the catastrophic scenarios we could/7889737465764587375982437952 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 expect to suffer if recombinant DNA research were not halted. As a way to stymie the ambitions of the relevant scientists, Cavalieri went so far as to suggest “that the Nobel Committee announce that no awards will ever be given in this area.” Chargaff and Cavalieri represented a contingent of researchers who wished to see the use of recombinant DNA banned for the foreseeable future.

Two years after the Asilomar conference, the National Academy epidemic, Berg called for a worldwide moratorium on all such exper- of Sciences held a follow-up meeting to further discuss the ramifiimentation, which led, in 1975, to the organization of the Asilomar cations of recombinant DNA. The panel, which included Paul Berg, Conference in California. More than 100 scientists from over a dozen drew protests against the new technology. countries attended, along with lawyers, journalists, and government Despite Cavalieri’s warning, Berg and his biochemist colleagues officials. Attendees fiercely debated whether research into recom- won a Nobel Prize in 1980. And since then, innumerable experibinant DNA was “flirting with disaster” or safe, even during meal ments—many of them inconceivable in 1975—have been conducted breaks and over cocktails well into the wee hours. with recombinant DNA without any hazardous repercussions. The On the final day of the conference, it was agreed that research worried, distinguished naysayers ended up conclusively on the should continue but only under the most stringent of restrictions wrong side of history. By being—and continuing to be—extremely on potentially dangerous experiments. For these, airlocks and ven- careful, biologists have thus far managed to prevent the doom that tilation systems were mandatory to prevent some general virus or Chargaff and Cavalieri prophesied. bacteria from escaping. And just to be extra safe, the microbes being But there have been occasional slip-ups in our biosafety defenses. tampered with had to be genetically weakened so that they’d die It was only last year, for instance, that a stash of old vials containing outside the lab. Stipulations such as these went on to help form the smallpox virus, kept precariously in a cardboard box labeled “1954,” the biosafety guidelines of the National Institutes of Health, which was spotted in a U.S. Food and Drug Administration lab in the heart of have been updated numerous times in each decade, most recently in the National Institutes of Health. And a month earlier, the director of 2013. If you’re a researcher and you’re looking to do an experiment the Center for Disease Control revealed that lab workers had fudged involving recombinant DNA, you must register and receive approval the handling and storage of what turned out to be live anthrax, as well by a safety committee if the lab work will involve, among other things, as inadvertently mixing deadly H5N1 influenza with a benign strain of cloning potentially lethal DNA or RNA combinations, or using human the virus in a shipment to another lab. or animal pathogens as hosts of manipulated genes. The Asilomar These lapses in professionalism, wrote Deborah Cotton, a professor Conference allowed research to proceed carefully “under a yellow of medicine at Boston University, “strain credulity.” Until these sorts light,” as Berg later said. of mistakes can be more reliably prevented, she suggested that highrisk experiments ought to be suspended.

3456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 **8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 5873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 03 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 8** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 74657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746576458 3456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 **8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 5873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 03 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e772 9361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//7889737 654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 49 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 83243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 9756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ 0101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^ 89034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^21 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6553839385476 89034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 47 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`8976443 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788

78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ ^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//788973746576458737598 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83 //78897374657645873759576?//7889737 {3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78 478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 76?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^78 189756253748098^9435 #!122200 & 003345^5@998654//345 0 2143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//788901010101 1 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875

Seeing the potential for recombinant DNA research to cause an

0 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 ^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//7889737465 7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 58..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 00101111174657645873759824379523 4890 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753

0101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^ 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//788973746576 00 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 ^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//788973746576 32438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^ 5 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 ||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 27 * 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101010101 88** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 ** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 //7889/78897 374657645873759824379523 4890347832 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 0010111117465764 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 3345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737

952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 7 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764432 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3

7409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 9986543456~`89764 7375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655 23 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 7 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764432

/{<0000_027

Because of the nature of humans and organizations, even our most diligent precautions are always fallible. In light of our experience with biosafety, would we be able to prevent a superintelligent AI from going rogue and threatening us? If so, one of the important causes will likely be the ongoing work in how to create—and maintain—a “Friendly AI.” Nate Soares, a researcher at the Machine Intelligence Research Institute and previously at Microsoft and Google, specializes in this area. In his most recent paper “The Value Learning Problem,” he essentially makes the point that Aretha Franklin didn’t go far enough when she says, “R-E-S-P-E-C-T/Find out what it means to me.” It’s simply not enough, in other words, for an AI to understand what moral behavior is; it must also have a preference for it. But, as every parent knows, instilling this in a child can be maddening. What makes that irritating situation tolerable, however, is the child’s lack of power to do any real harm in the mean time—which may not be the case at all with a super-intelligence. “Given the enormity of the stakes and the difficulty of writing bug-free software,” writes Soares, “every available precaution must be taken when constructing super-intelligent systems.” And it’s critical that we don’t wait till the technology arrives to begin thinking of precautions, Soares says; even if our guidelines are vague and abstract, they’ll be better than nothing. “If this were the timeline of aviation, we would only just be get-

ting beyond the stage where a few inventors were tinkering with flapping-wing contraptions.” One such precaution is based on the hypothesis that it won’t be humans who directly create a super-intelligence; instead, we’ll create a human-level AI that then continuously improves on its own design, making itself far more ingenious than anything humans could engineer. Therefore the challenge is in making sure that when it comes specifically to refining its own intelligence, the AI goes about it wisely, making sure that it maintains our moral values even as it rewires its own “brain.” If you see a paradox lurking here, you’re right: It’s analogous to trying to imagine what you would do in a difficult situation if you were smarter than you are. This is why “a self-improving agent must reason about the behavior of its smarter successors in abstract terms,” explains Soares. “If it could predict their actions in detail, it would already be as smart as them.” As of now, Soares thinks we’re very far from an adequate theory of how an intelligence beyond ours will think. One of the main hurdles he highlights is programming the AI’s level of selftrust, given that it will never be able to come up with a mathematically certain proof that one decision is superior to another. Program in too much doubt, and it will never decide how to effectively modify itself; program in too much confidence, and it will execute poor decisions rather than searching for better ones. Coming up with the right balance between paralyzing insecurity

Who will be man's successor? To wihich the answer is: We are ourselves creating our own successors. Man will become to the machine what the horse and the dog are to man; the conclusion being that machines are, or are becoming animate. —Samuel Butler, 1863 letter, "Darwing among the machines"

/{<0000_029 \<||>.\\|00000}

///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////// // //////// ////////////////////////////// /////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////////////////////// //////////////// /////////////////////////////// ///// ///////// ///////////// //////////////////////////////// ///////////////////////////////////////////// /////////////// ////////// //// /////// ///// ////////////////////////////////////// //////////////////////////////// ////////////////// // /////////////////// ////////////////////////////// ///////// ///////////////////////////////////////////// ///////////////////// //// //////////////////// //////////////////////////////////////////////////////////////////// / ///////////////////////////////////////////// /////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////////////// /////////////////////////////// ///// ///////// ///////////// //////////////////////////////// ////////////////////////////////////// / ////////////////// / ////////// //// /////// ///// ////////////////////////////////////// //////////////////////////////// ////////////////// // /////////////////// //////// ////////////////////////////// ///////////////////////////////////////////// ///////////////////// //// //////////////////// /////////////////////////////////////////////// //// /////////////////////////////// ///// ///////// ///////////// //////////////////////////////// ///////////////////////////////////////////// ////////////////// / /////// /////// ///// ////////////////////////////////////// //////////////////////////////// ////////////////// // /////////////////// ///////////////////////////////////////////// /////////////////////////////////////// ///////////////////// //// //////////////////// //////////////////////////////////////////////////////////// ///////////////////////

and brash self-assurance is spectacularly complicated. And it may not be possible to task an AI with solving this problem itself, because it may require a deeper understanding of the theoretical problems than we currently have; it may even be unwise to try, because a super AI may reason that it’s in its own interest to deceive us. This is why figuring out how to make machines moral first, perhaps before allowing them to self-modify, “is of critical importance,” writes Soares. “For while all other precautions exist to prevent disaster, it is value learning which could enable success.” But to some other AI researchers, precautions are not just unnecessary, they’re preposterous. “These doomsday scenarios are logically incoherent at such a fundamental level that they can be dismissed as extremely implausible,” writes Richard Loosemore, in an article for the Institute for Ethics and Emerging Technologies. He called out the Machine Intelligence Research Institute specifically for peddling unfounded paranoia. Their restrictions would “require the AI to be so unstable that it could never reach the level of intelligence at which it would become dangerous,” writes Loosemore. One insight to glean from this dispute may be that the technology is still too far off for anything convincing to be said about it. As Loosemore declares, “If this were the timeline of aviation, rather than the timeline of AI, we would only just be getting beyond the stage where a few inventors were tinkering with flapping-wing contraptions.” On the other hand, the development of super-intelligence may be the paradigmatic example of where it’s better to be safe than sorry.

var=intelligence <explosion

0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783 *8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654# 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 2774093 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824 22200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^ 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 9 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 48903478 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 5 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^ 47 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654# 8767656576?//788973746576458 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6553 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 23 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 45873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^ 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//788973746 /78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//7889737465764 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654# 432420)00 0 1897562537480 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u38384747 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&* e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000 //7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 2774093 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824 22200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^ 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 9 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//78896 7656576?//78897374657645873759824379523 48903478 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 5 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 4890347832 189756253748098^9 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u38384747 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&* e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 18975 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^ 83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#4 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 2774093 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824 22200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^ 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 9 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 48903478 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 5 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^ 47 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654# 8767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8* ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//7 %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873 034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 2774093

61!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 9986543456~`89 3465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 565 >9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>987676 /7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^ 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//788973 6?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379 $##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!001010 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~ 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 3243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 764587375982437 952348\\9034 <<783243 43$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//7889737465 788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746576458737598243795 ##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!001010 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~ 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||90 5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 952348\\9034 <<783243 0<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/765 7 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^78 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(& 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 76 010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^ 83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 4890347832 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5 #!576?//788973746576458

73759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 76458737598243 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//7 759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 37465764 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 2774093 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e 903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@ <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 23 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010 657645873759824379523 4890 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6553 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||8 543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 00 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^

��≥≓≒≺⊄ �� ⊳⊸⋮⌐⋔⋘ ◀◃◯◻◾⋌⋍⤣ ⤥ � � ⫋ ⦶ �⤥ ⧈ ⥊�� ∪⊑◁●⊚⥯ ±↞��⧄��

//STRONG_AI //STRONG _AI/STRONG _AI// STRONG _ AI//STRONG _AI/ /STRONG _ AI/ /STRONG _ AI//STRONG _ AI//STRONG _ AI//STRONG _ AI//STRONG _ AI

///////// //// /////CONSIDER /////////////// //////NONBIO////// //////////////////////////////// STRONG AI PROMISES TO CONTINUE THE EXPONENTIAL GAINS OF HUMAN CIVILIZATION. (IF WE THE / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / ////////////// /////////////////////// LOGICAL INTELLIGENCE DERIVED FROM OUR HUMAN CIVILIZATION AS STILL HUMAN.) BUT THE DANGERS IT PRESENTS / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / ////////////////////// /////////////// ARE ALSO PROFOUND PRECISELY BECAUSE OF ITS AMPLIFICATION OF INTELLIGENCE. INTELLIGENCE IS INHERENTLY / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / ///////////////////////////// /////// IMPOSSIBLE TO CONTROL, SO THE VARIOUS STRATEGIES THAT HAVE BEEN DEVISED TO CONTROL NANOTECHNOLOGY / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / ///////////////////////////////////// WON’T WORK FRO STRONG AI. There have been discussions and proposals to guide ai development toward what eliezer yudkowsky calls “friendly ai”.these are useful for discussion, but it is infeasible today to devise strategies that will absolutely ensure that future ai embodies human ethics and values.

_//01 Protection from “Unfriendly” Strong AI Even as effective a mechanism as the broadcast architecture, however, won’t serve as protection against abuses of strong AI. The barriers provided by the broadcast architecture rely on the lack of intelligence in nanoengineered entities. By definition, however, intelligent entites have the cleverness to easily overcome such barriers. Eliezer Yudkowsky has extensively analyzed paradigms, architectures, and ethical rules that may help assure that once strong AI has the means of accessing and modifying its own design it remains friendly to biological humanity and supportive of its values. Given that self-improving strong AI cannot be recalled, Yudkowsky points out that we need to “get it right the first time,” and that its initial design must have “zero nonrecoverable errors.” Inherently there will be no absolute protection against strong AI. Although the argument is subtle I believe that maintaining an open free-market system for incremental scientific and technological progress, in which each step is subject to market acceptance, will provide the most constructive environment for technology to embody widespread human values. As I have pointed out, strong AI is emerging from many diverse efforts and will be deeply integrated into our civilization’s infrastructure. Indeed, it will be intimately embedded in our bodies and brains. As such, it will reflect our values because it will be us. Attempts to control these technologies via secretive government programs, along with inevitable underground development, would only foster an unstable environment in which the dangerous applications would be likely to become dominant.

/////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////// //////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///// //////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////////////////// //////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////////////// / ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////////////// ///////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////////////// ////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////////// ////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /// ////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////// /// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////// /////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////////////// ////////////////////

/{<0000_035

\< SETTING A DATE FOR {THE SINGULARITY\\ \ <</ A more modest but still profound threshold will be achieved much earlier.

In the early 2030s one thousand dollars’ worth of computation will buy about 1017 cps. Today we spend more than $1011 ($100 billion) on computation in a year, which will conservatively rise to $1012 ($1 trillion) by 2030. So we will be producing about 1026 to 1029 cps of nonbiological computation per year in the early 2030s. This is roughly equal to our estimate for the capacity of all living biological human intelligence. Even if just equal in capacity to our brains, this nonbiological portion of our intelligence will be more powerful because it will combine the pattern recognition powers of human intelligence with the memory and skill-sharing ability and memory accuracy of machines. The nonbiological portion will always operate at peak capacity, which is far from the case for biological humanity today; the 1026 cps represented by biological human civilization today is poorly utilized. This state of computation in the early 2030s will not represent the Singularity, however, because it does not yet correspond to a profound expansion of our intelligence. By the mid-2040s, however, that one thousand dollars’ worth of computation will be equal to 1026 cps, so the intelligence created per ><>>><<< year will be >>>>>>>>>>>>>>>>>>< about >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< <>><<> >>>>>>>>>>>>> >>>>>>>>> ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><< one billion times more powerful than all>>>>>>>>>>>>> human intelligence>>>>>>>>>>>>>>>>>>>>> today. >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< That will indeed represent a>>>>>>>>>>>>> profound change, and it is for that reason that we can>>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >><and ><< ><>>><<< >>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >> set the date for the Singularity—representing a profound disputed transfor>>>>>>>>>>>>> >>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< > mation in human>>>>>>><<<<<<<<<<<<<<<>> capability—as 2045. >>>>>>>>>>>>>>>>>>>>> >>< ><< ><>>><<< >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><< >>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >> Despite the clear predominance of nonbiological intelligence by the mid-2040s, >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< but ours will still be a human civilization. We will transcend biology, but fortunately >>>>>>>>>>>>> >>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >> we will not transcend our humanity. >>>>>>>>>>>>> >>>>>>>>>< <>><<> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>> _Returning to the limits of computation to physics, the estimates >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< according >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<<<<<<>> >>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> above >>>>>>>>>>>>>>>>>>>>>>>>> were expressed in therms of laptop-size computers because that is a >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><< >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>> >>>>>>><<<<<<<<<<<< familiar form factor today. By the second decade of this century, however, most computing will not be organized in such rectangular devices but will be highly distributed throughout the environment. Computing will be everywhere: In the walls, in our furniture, in our clothing, and in our bodies and brains. And, of course, human civilization will not be limited to computing with just a few pounds of matter. The amount of time required for our human civilization to achieve scales of computation—and intelligence—that go beyond our planet and into the universe may be a lot shorter than you might think.

><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>> >>>>>>>>>>>>> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> ><<>>< <>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> <<<>>><> <<> <>> >><<<<<<<<<<<<<<<>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>

>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< << >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> << >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> < ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> <> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> <<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>><

/{<0000_037

{ THE JOBS/>

FUTURE

\OF /

Robotics and artificial intelligence will permeate wide segments of our daily life </The vast majority of experts anticipate that robotics and artificial intelligence will permeate wide segments of daily life by 2025, with huge implications for a range of industries such as health care, transport and logistics, customer service, and home maintenance. But even as they are largely consistent in their predictions for the evolution of technology itself, they are deeply divided on how advances in AI and robotics will impact the economic and employment picture over the next decade. Some 1,896 experts responded to the following question about the economic impact of robotic advances and AI:

/{ Will networked, automated, artificial intelligence (AI) applications and robotic devices have displaced more jobs than they have created by 2025? Half of these experts (48%) envision a future in which robots and digital agents have displaced significant numbers of both blue- and white-collar workers—with many expressing concern that this will lead to vast increases in income inequality, masses of people who are effectively unemployable, and breakdowns in the social order. The other half of the experts who responded to this survey (52%) expect that technology will not displace more jobs than it creates by 2025. To be sure, this group anticipates that many jobs currently performed by humans will be substantially taken over by robots or digital agents by 2025. But they have faith that human ingenuity will create new jobs, industries, and ways to make a living, just as it has been doing since the dawn of the Industrial Revolution. These two groups also share certain hopes and concerns about the impact of technology on employment. For instance, many are concerned that our existing social structures—and especially our educational institutions—are not adequately preparing people for the skills that will be needed in the job market of the future. Conversely, others have hope that the coming changes will be an opportunity to reassess our society’s relationship to employment itself—by returning to a focus on small-scale or artisanal modes of production, or by giving people more time to spend on leisure, self-improvement, or time with loved ones.

/{<0000_039

I see the movement towards AI and robotics as evolutionary, in large part because it is such a sociological leap. The technology may be ready, but we are notâ&#x20AC;&#x201D;at least, not yet. â&#x20AC;&#x201D;Geoff Livingston

Key themes: reasons to be hopeful

01//Advances in technology may displace certain types of work, but historically they have been a net creator of jobs. 02//We will adapt to these changes by inventing entirely new types of work, and by taking advantage of uniquely human capabilities. 03//Technology will free us from day-to-day drudgery, and allow us to define our relationship with â&#x20AC;&#x153;workâ&#x20AC;? in a more positive and socially beneficial way. 04//Ultimately, we as a society control our own destiny through the choices we make.

/..The view from those who expect AI and robotics to have a positive or neutral impact on jobs by 2025 Vint Cerf, vice president and chief Internet evangelist for Google, said, "Historically, technology has created more jobs than it destroys and there is no reason to think otherwise in this case. Someone has to make and service all these devices.." Ben Shneiderman, professor of computer science at the University of Maryland, wrote, “Robots and AI make compelling stories for journalists, but they are a false vision of the major economic changes. Journalists lost their jobs because of changes to advertising, professors are threatened by MOOCs, and store salespeople are losing jobs to Internet sales people. Improved user interfaces, electronic delivery (videos, music, etc.), and more self-reliant customers reduce job needs. At the same time someone is building new websites, managing corporate social media plans, creating new products, etc. Improved user interfaces, novel services, and fresh ideas will create more jobs.” John Markoff, senior writer for the Science section of the New York Times, responded, “You didn’t allow the answer that I feel strongly is accurate—too hard to predict. There will be a vast displacement of labor over the next decade. That is true. But, if we had gone back 15 years who would have thought that ‘search engine optimization’ would be a significant job category?” Michael Glassman, associate professor at the Ohio State University, wrote, “I think AI will do a few more things, but people are going to be surprised how limited it is. There will be greater differentiation between what AI does and what humans do, but also much more realization that AI will not be able to engage the critical tasks that humans do.” Robert Cannon, Internet law and policy expert, predicts, “Everything that can be automated will be automated. Non-skilled jobs lacking in ‘human contribution’ will be replaced by automation when the economics are favorable. At the hardware store, the guy who used to cut keys has been replaced by a robot. In the law office, the clerks who used to prepare discovery have been replaced by software. IBM Watson is replacing researchers by reading every report ever written anywhere. This begs the question: What can the human contribute? The short answer is that if the job is one where that question cannot be answered positively, that job is not likely to exist.” Pamela Rutledge, PhD and director of the Media Psychology Research Center, responded, “There will be many things that machines can’t do, such as services that require thinking, creativity, synthesizing, problem-solving, and innovating… Advances in AI and robotics allow people to cognitively offload repetitive tasks and invest their attention and energy in things where humans can make a difference. We already have cars that talk to us, a phone we can talk to, robots that lift the elderly out of bed, and apps that remind us to call Mom. An app can dial Mom's number and even send flowers, but an app can't do that most human of all things: emotionally connect with her.”

>>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>< <>><<> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>>> >>>>>>><<<<<<>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>><<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>><>> >>>>>>><<<<<<<<<<<<<<<>> >>< ><< ><>>><<< >>>>>>>>>>>>>>>>>>< <>><<>

/{<0000_043

“We will reach the singularity in the time frame most experts predict. From that point on you’re going to see that the top species will no longer be humans, but machines.” —Louis A. Del Monte.

Landscape Lands </andscape {and //scape

189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&7465764 243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 4890347832 523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/ 3#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746576458 189756253748098^9435 #!12220 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 04 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#43 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576? //7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 2 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//788974840494 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 37465764587375982 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 0010111117465 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^78 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 76 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654 7 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374 w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&88 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$#

Throughout history, technology has been a job creator—not a job destroyer.

61!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//788973746576458737598243795 >98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000 7645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654/ 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 43$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1 657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 65 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 0<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 3746576458737 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//7889010101010101 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?

Advances in technology create new jobs and industries even as they displace some of the older ones

73759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 4783 3759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//788973746 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 000 1010000 00101111174657645873759824379523 4890 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003 543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 001011111 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00 76?//788973746576458 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543 56~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 001011111746576458737598 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420)0435 #!576?//78897374657645873759576?//7889737465764587375982437 56576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^7 7409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656 $5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 27740 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$201 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 4890347832 189756253748098^94 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@ 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010 465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 18975625 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//7889737465764 &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656 10 000101010 1 01010101 010010 101010101 765657 6?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 565 3824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65 **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//788 4323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 76 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4697374 7644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 4890347 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 76 873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973 #2!#$$5^&**8*(*^&6^78 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 4 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$201193039 6?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 565 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 8w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@998654345 9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 0010111117465 46973746#$$5^ 764587375982437 952348\\9034 <<783243 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361 !0010101375 ““982437952 382437952 122200 & 0033

There are certain jobs that only humans have the capacity to do

The technology will not advance enough in the next decade to substantially impact the job market

Our social, legal, and regulatory structures will minimize the impact on employment

45^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u38 90101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432 420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 34783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 2 277409361!00101010#$$5^ 1000003345^5@9986543456~`897644323#$$5^21434 65(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//788973746576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420) 3824379523122200 & 003345^5@998 654// 00000 <>987 3} 65(&&888** 5654#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0) <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7 656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00 10101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 73759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{32 43$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 3824379523122200 & 003345^5@998 654// 000 00 <>987 3} 65(&

45||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783 243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& /78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764432 00 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 10101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 8967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654# 432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 32420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 49 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//78896 7656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 24379523 4890347832 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 57645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 1013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 88w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 64587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 4#432420)00 000<>9876765657 6?//788973746576458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e772034 46576458774657645873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788 88** 5654#432420)00 000<>98767656576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> ##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 2774093

824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101&888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 79523824379523122200&003345^5@998654//00000 <>987 67 6 565 76?//788973746#$$5^ 764587375982437 952348\\||9034 <<783243 189756253748098^9435 #!122200&003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 6553839385476 788w83e772}0347 803345^5@99||86543456~`89 764 3 232143} 65(&&888** 5654#432420)00 0 189756253748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 27 6576?//7889731 0010101010100 10 000101010 1 01010101 010010 101010101 7656576?//78890101010101010101010000 00101111174657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$ 09361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143465(&&888**5654#432420) 0435 #!576?//78897374657645873759576?//78897374657645873759824379523824379523122200 & 003345^5@998654// 00000 <>987 67 6 565 119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>9877656576?//788967656576?//78897374657645873759824379523 489034783243$##2!#$$5^& **8*(*^&6^788w83e772}0347 803345^5@99||86543456~`89 764 3 435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 65538393854748404949 55858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 8*(*^&6^788w83 e7720347 89 523 48||9034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** ^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100 10 000101010 1 01010101 010010 0101 &888** 5654#432420)00 000<>9 8767656576?//7889737 4657645873759824379523 489034783243$##2!#$$5^&** 8*(*^&6^788w83e7720347 8 243875 6&76 534 277409361!00101010#$$5^ 1000003345^5@9986543456~`89764 4323#$$5^2143 53748098^9435 #!122200 & 003345^5@998654//345 0 47832438756&76534 277409361! 101013836753 655383938547656576?//7889748404949 55858..>><<0))({^ %54$20119303 93 <u38384747489> 00~`89 764 3 232143} 65(&&888** 5654#432420)00 458737 59 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?/7656576?//7889/78897 374657645873759824379523 489034783243$# #2!#$$5^&**8*(*^&6^788w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ 4 3 232143} 65(&&888** 5654#432420)00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 6576?/67 6 56576?//788858..>><<0))({^ %54$2011930393 <u383847474 89> 0003345^5 @99||86543456~`89 764 3 232143} 65(&74657645||873759 82437~`89 764 3 232143} 65(&&888** 5654#432420)00 000<> 98767656576?//7889731 0010101010100

277409361!0010101 &888** 5654#432420)00 000<>9 8767656576?//7889737 46973746#$$5^ 764587375982437 952348\\9034 <<783243 8w83e7720347 89 523 489034783243$##2!#$$5^&**|8*(*^ &6^788w83e7720347 8 243875 6&76 534 00 000<>98767656576?//78897374657645873759824379523 489034783243$##2!#$$5^&**8*(*^&6^788w83e7720347 89 523 48903478{3243$##2!#$$5^&**8*(*^&6^788w83e77 ||20347 8 243875 6&76 534 277409361!0010101375 ““982437952 ))({^ %54$2011930393

/{<0000_047

01//Impacts from automation have thus far impacted mostly blue-collar employment; the coming wave of innovation threatens to upend white-collar work as well. 02//Certain highly-skilled workers will succeed wildly in this new environmentâ&#x20AC;&#x201D;but far more may be displaced into lower paying service industry jobs at best, or permanent unemployment at worst. 03//Our educational system is not adequately preparing us for work of the future, and our political and economic institutions are poorly equipped to handle these hard choices.

Key themes: reasons to be concerned

/{<0000_049

human human human hu man h \\

/{<0000_051

â&#x20AC;&#x153;The jobs that the robots will leave for and knowledge. In other words, only with machines. " â&#x20AC;&#x201D; Howard Rheingold

humans will be those that require thought the best-educated humans will compete

//The view from those who expect AI and robotics to displace more jobs than they create by 2025 Jerry Michalski, founder of REX, the Relationship Economy expedition, sees the logic of the slow and unrelenting movement in the direction of more automation: “Automation is Voldemort: the terrifying force nobody is willing to name. Oh sure, we talk about it now and then, but usually in passing. We hardly dwell on the fact that someone trying to pick a career path that is not likely to be automated will have a very hard time making that choice. X-ray technician? Outsourced already, and automation in progress. The race between automation and human work is won by automation, and as long as we need fiat currency to pay the rent/ mortgage, humans will fall out of the system in droves as this shift takes place… The safe zones are services that require local human effort (gardening, painting, babysitting), distant human effort (editing, coaching, coordinating), and high-level thinking/relationship building. Everything else falls in the target-rich environment of automation.” Tom Standage, digital editor for The Economist, makes the point that the next wave of technology is likely to have a more profound impact than those that came before it: “Previous technological revolutions happened much more slowly, so people had longer to retrain, and [also] moved people from one kind of unskilled work to another. Robots and AI threaten to make even some kinds of skilled work obsolete (e.g., legal clerks). This will displace people into service roles, and the income gap between skilled workers whose jobs cannot be automated and everyone else will widen. This is a recipe for instability.” Mark Nall, a program manager for NASA, noted, “Unlike previous disruptions such as when farming machinery displaced farm workers but created factory jobs making the machines, robotics and AI are different. Due to their versatility and growing capabilities, not just a few economic sectors will be affected, but whole swaths will be. This is already being seen now in areas from robocalls to lights-out manufacturing. Economic efficiency will be the driver. The social consequence is that good-paying jobs will be increasingly scarce." Stowe Boyd, lead researcher at GigaOM Research, said, “As just one aspect of the rise of robots and AI, widespread use of autonomous cars and trucks will be the immediate end of taxi drivers and truck drivers; truck driver is the number-one occupation for men in the U.S.. Just as importantly, autonomous cars will radically decrease car ownership, which will impact the automotive industry. Perhaps 70% of cars in urban areas would go away. Autonomous robots and systems could impact up to 50% of jobs, according to recent analysis by Frey and Osborne at Oxford, leaving only jobs that require the ‘application of heuristics’ or creativity… An increasing proportion of the world’s population will be outside of the world of work—either living on the dole, or benefiting from the dramatically decreased costs of goods to eke out a subsistence lifestyle. The central question of 2025 will be: What are people for in a world that does not need their labor, and where only a minority are needed to guide the 'bot-based economy?” Alex Howard, a writer and editor based in Washington, D.C., said, “I expect that automation and AI will have had a substantial impact on white-collar jobs, particularly back-office functions in clinics, in law firms, like medical secretaries, transcriptionists, or paralegals. Governments will have to collaborate effectively with technology companies and academic institutions to provide massive retraining efforts over the next decade to prevent massive social disruption from these changes.” Bryan Alexander, technology consultant, futurist, and senior fellow at the National Institute for Technology in Liberal Education, wrote, “The education system is not well positioned to transform itself to help shape graduates who can ‘race against the machines.’ Not in time, and not at scale. Autodidacts will do well, as they always have done, but the broad masses of people are being prepared for the wrong economy.”

/{<0000_053

Displacement of workers from automation is already happening—and about to get much worse

The consequences for income inequality will be profound The educational system is doing a poor job of preparing the next generation of workers We will experience less drudgery and more leisure time

It will free us from the industrial age notion of what a “job” is.

/{<0000_055

innovative minds while you have some snacks.

everyone involved, and share ideas with other

Come to get the official welcome, meet and greet

Location: Main Building, Room 234 B

Opening Speech

07:00pm–08:00pm

innovative minds while you have some snacks.

Come to get the official welcome, meet and greet everyone involved, and share ideas with other

Location: Main Building Hall

Meet and Greet

03:00pm–06:00pm

Location: Clubhouse

Check-in and early Registration

Location: Main Building, Room 321

Nick Bostrom

AI implications

12:00pm–01:00pm

thrilling period in its history.

Humanity stands on the verge of the most transforming and

Location: Terman Engineerign Building, Room 154

Ray Kurzweil

The Singularity Is Near

11:00am–12:00pm

Location: Main Building, Room 234 B

Symposium on Singularity

09:00am–10:00am

>>020{ August_Thursday

<019/ August_Wednesday 12:00pm–02:00pm

Earth-originating intelligent life or permanently and drastically curtail its potential"

Introducing the concept of an existential risk, an “adverse outcome would annihilate

\\sch schedule sched at_ a> gla glance .}

/{<0000_059

Location:School of Education Room 450

Francis Fukuyama

The future of economics

03:00pm–04:00pm

Location: Clubhouse

Lunch break

01:00pm–02:00pm

Location: Main Building, Room 321

10 expert speakers will discuss how AI may influence human labor

The Future of Jobs

03:00pm–05:30pm

Location: Clubhouse

Eat and mingle

Lunch break

01:00pm–02:30pm

potentialities extrapolated for one reason--the survival of humankind.

as the balance tips in their favor? These questions are addressed rigorously, their

“singularity,” a point in time like no other. Will machines continue to serve us

of man. One day the AI will exceed our own intelligence. We will have reached the

Scientists are working relentlessly at improving AI technology for the benefit

Location: Physics Lecture Hall

Louis A. Del Monte

Will Artificial Intelligence serve us or replace us?

11:00pm–01:00pm

can currently identify cells in humans with 12 different types of cancer tumors.

The beginning of the human treatment with nanomedicine. DNA nanobots

Location: Main Building, Room 234 B

Dr Ido Bachelet

How technology can change our life expectancy

09:00am–10:00am

#021.. August_Friday

\\map map stan .>ford stanford campus map m .}

_09

</Campus Center

Clubhouse Bookstore School of Education Green Libray Art Galery Main Building Hall Conference room 234 Conference room 321 Physics Lecture Hall Research Center Cafeteria Terman Engineering

_18

</Library Quad

_21 Lassuen Mall

03 09 18 21 36 42 43 47 51 58 62 72

_36

_03 Duena St

_72

</Shuttle Stop

_58

_62

≺⊄��⊳⊸ ◯ ◻ ◾ ⋌ ⋍ ⤣ ⤥ � � ⫋ ⦶ �⤥ ⧈ ��∪⊑◁●⊚⥯⎬± ��

Santa Teresa Mall

_42 _43

Lomita Mall

Escondido Mall

</Main Quad

_51

_47

/{<0000_061

///// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////// //////////////////////////// ///////////////////////////////////////////// //////////////////////////////// ///// ///////////////////////////////////////////// ///////////////////////////////////////////// ///////// //////////////////////////// ////////////////////////////////////////////////// /////////////////////////// ////////// ///////////////////////////////////////////// /////////////////////////// ////////////////////// /////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////// /////////////// ///////////////////////////////////////////// /////////////////////////////// ///////////// //////////////////////// ///////////////////////////////////////////// //////////////////////////////// /// ////////////////////////////////// ///////////////////////////////////////////// ////////////////////////// //////////////////////////////////////////// ///////////////////////////////////////////// //////////////// ////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ///////////////////////////////////////////// ////////////////////////////////////////////////////// ////// /////////////////////////////// ///////////////////////////////////////////// /////////////////////// ///// //////////////////////////////// ///////////////////////////////////////////// //////////////////////////// ///////// ///////////////////////////////////////////// /////////////////////////////////// /////////////// ////////////////////// ///////////////////////////////////////////// ////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// /////////////////// ///////////////////////////////////////////// ///////////////////////////////////////// ////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// /////////////////// ///////////////////////////////////////////// /////////////////////////////////////////

{ COLOPHON COLOPHON >/

> /..

This book was designed by Laia Pose as a student project for the Typography 4 class at the Academy of Art University. It has been typeset using the typefaces DIN Next Lt Pro, DIN Next Slab Pro and Netto OT. It has been printed on Staples Photo Supreme Matte Paper Double Sided 61lb and bound by California Office Service in San Francisco, May 2015.

/{<0000_063

/////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// ///////////////// ///////////////////////////////////////////// //////////////////////////////////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// / ///////////////////////////// ///////////////////////////////////////////// //////////////////////////////// /////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// /////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// //////////////// / ///////////////////////////////////////////// ///////////////////////////////////////////// ////////////// /////////////////////////////////////////// ///////////////////////////////////////////// ////////////////// //////////// ///////////////////////////////////////////// ///////////////////////////////////////////// /// /////////////////////////// ///////////////////////////////////////////// ////////////////////////////////// /////////////////////////////////////////// ////////////////////////////////////////////////// ///////////// ///////////////// ///////////////////////////////////////////// /////////////////////////// //////////////// ////////////// ///////////////////////////////////////////// ///////////////////////////////////////////// / ///////////////////////////// ///////////////////////////////////////////// //////////////////////////////// /////////////////////////////////////////// /////////////////////////////// //////////////////////////////// /////////////////////////////////////////// //////////////////////////////// /////////////////////////////// ///////////////////////////////////////////// //////////////////////////////// ///////////////////////////// / ///////////////////////////////////////////// ////////////////////////////////////////////// ///////////// ///////////////// ///////////////////////////////////////////// //////////////////////////////////////////// ///////////////////////////////////////// ///////////////////////////////////////////// //////////////////// ////////// /////////////////////// ///////////////////////////////////////////// /////////////////////////// /// ///////////////////////////////////////////// ///////////////////////////////////////////// //////////// //////// ///////////////////////////////////////////// ///////////////////////////////////////////// /////// //////////////////// ///////////////////////////////////////////// /////////////////////////////////////////

/////////////////////////////////////// /////////////////////// ////////////// ///////////////////////////////////////////// //