Microsoft Word Kurzweil, Ray The Singularity Is Near doc

The Precautionary Principle.
As Bostrom, Freitas, and other observers including myself have pointed out, we cannot 
rely on trial-and-error approaches to deal with existential risks. There are competing interpretations of what has 

---

become known as the "precautionary principle." (If the consequences of an action are unknown but judged by some 
scientists to have even a small risk of being profoundly negative, it's better to not carry out the action than risk 
negative consequences.) But it's clear that we need to achieve the highest possible level of confidence in our strategies 
to combat such risks. This is one reason we're hearing increasingly strident voices demanding that we shut down the 
advance of technology, as a primary strategy to eliminate new existential risks before they occur. Relinquishment, 
however, is not the appropriate response and will only interfere with the profound benefits of these emerging 
technologies while actually increasing the likelihood of a disastrous outcome. Max More articulates the limitations of 
the precautionary principle and advocates replacing it with what he calls the "proactionary principle," which involves 
balancing the risks of action and inaction.
24
Before discussing how to respond to the new challenge of existential risks, it's worth reviewing a few more that 
have been postulated by Bostrom and others. 
The Smaller the Interaction, the Larger the Explosive Potential.
There has been recent controversy over the 
potential for future very high-energy particle accelerators to create a chain reaction of transformed energy states at a 
subatomic level. The result could be an exponentially spreading area of destruction, breaking apart all atoms in our 
galactic vicinity. A variety of such scenarios has been proposed, including the possibility of creating a black hole that 
would draw in our solar system. 
Analyses of these scenarios show them to be very unlikely, although not all physicists are sanguine about the 
danger.
25
The mathematics of these analyses appears to be sound, but we do not yet have a consensus on the formulas 
that describe this level of physical reality. If such dangers sound far-fetched, consider the possibility that we have 
indeed detected increasingly powerful explosive phenomena at diminishing scales of matter. 
Alfred Nobel discovered dynamite by probing chemical interactions of molecules. The atomic bomb, which is 
tens of thousands of times more powerful than dynamite, is based on nuclear interactions involving large atoms, which 
are much smaller scales of matter than large molecules. The hydrogen bomb, which is thousands of times more 
powerful than an atomic bomb, is based on interactions involving an even smaller scale: small atoms. Although this 
insight does not necessarily imply the existence of yet more powerful destructive chain reactions by manipulating 
subatomic particles, it does make the conjecture plausible. 
My own assessment of this danger is that we are unlikely simply to stumble across such a destructive event. 
Consider how unlikely it would be to accidentally produce an atomic bomb. Such a device requires a precise 
configuration of materials and actions, and the original required an extensive and precise engineering project to 
develop. Inadvertently creating a hydrogen bomb would be even less plausible. One would have to create the precise 
conditions of an atomic bomb in a particular arrangement with a hydrogen core and other elements. Stumbling across 
the exact conditions to create a new class of catastrophic chain reaction at a subatomic level appears to be even less 
likely. The consequences are sufficiently devastating, however, that the precautionary principle should lead us to take 
these possibilities seriously. This potential should be carefully analyzed prior to carrying out new classes of accelerator 
experiments. However, this risk is not high on my list of twenty-first-century concerns.