Microsoft Word Kurzweil, Ray The Singularity Is Near doc

 
Trying to Understand Our Own Thinking 
 
The Accelerating Pace of Research 
We are now approaching the knee of the curve (the period of rapid exponential growth) in the accelerating 
pace of understanding the human brain, but our attempts in this area have a long history. Our ability to 
reflect on and build models of our thinking is a unique attribute of our species. Early mental models were of 
necessity based on simply observing our external behavior (for example, Aristotle's analysis of the human 
ability to associate ideas, written 2,350 years ago).
19
At the beginning of the twentieth century we developed the tools to examine the physical processes 
inside
the brain. An early breakthrough was the measurement of the electrical output of nerve cells, 
developed in 1928 by neuroscience pioneer E. D. Adrian, which demonstrated that there were electrical 
processes taking place inside the brain.
20
As Adrian write, "I had arranged some electrodes on the optic 
nerve of a toad in connection with some experiment on the retina. The room was nearly dark and I was 
puzzled to hear repeated noises in the loudspeaker attached to the amplifier, noises indicating that a great 
deal of impulse activity was going on. It was not until I compared the noises with my own movements around 
the room that I realized I was in the field of vision of the toad's eye and that it was signaling what I was 
doing." 
Adrian's key insight from this experiment remains a cornerstone of neuroscience today: the frequency of 
the impulses from the sensory nerve is proportional to the intensity of the sensory phenomena being 
measured. Fr example, the higher the intensity of the light, the higher the frequency (pulses per second) of 
the neural impulses from the retina to the brain. It was a student of Adrian, Horace Barlow, who contributed 
another lasting insight, "trigger features" in neurons, with the discovery that the retinas of frogs and rabbits 
has single neurons that would trigger on "seeing" specific shapes, directions, or velocities. In other words, 
perception involves a series of stages, with each layer of neurons recognizing more sophisticated features of 
the image. 
In 1939 we began to develop an idea of how neurons perform: by accumulating (adding) their inputs 
and then producing a spike of membrane conductance (a sudden increase in the ability of the neuron's 
membrane to conduct a signal) an voltage along the neuron's axon (which connects to other neuron's via a 
synapse). A. L. Hodgkin and A. F. Huxley described their theory of the axon's "action potential" (voltage).
21
They also made an actual measurement of an action potential on an animal neural axon in 1952. They 
chose squid neurons because of their size and accessible anatomy. 
Building on Hodgkin and Huxley's insight W. S. McCulloch and W. Pitts developed in 1943 a simplified 
model of neural nets that motivated a half century of work on artificial (simulated) neural nets (using a 
computer program to simulate the way neurons work in the brain as a network). This model was further 
refined by Hodgkin and Huxley in 1952. Although we now realize that actual neurons are far more complex 
that these early models, the original concept has held up well. This basic neural-net model has a neural 
"weight" (representing the "strength" of the connection) for each synapse and a nonlinearity (firing threshold) 
in the neuron soma (cell body). 
As the sum of the weighted inputs to the neuron soma increases, there is relatively little response from 
the neuron until a critical threshold is reached, as which point the neuron rapidly increased the output of its 
axon and fires. Different neurons have different thresholds. Although recent research shows that the actual 
response is more complex than this, the McCulloch-Pitts and Hodgkin-Huxley models remain essentially 
valid. 
These insights led to an enormous amount of early work in creating artificial neural nets, in a field that 
became known as connectionism. This was perhaps the first self-organizing paradigm introduced to the field 
of computation. 
A key requirement for a self-organizing system is a nonlinearity: some means of creating outputs that 
are not simple weights sums of the inputs. The early neural-net models provided this nonlinearity in their 
replica of the neuron nucleus.
23
(The basic neural-net method is straightforward.)
24
Work initiated by Alan 
Turing on theoretical models of computation around the same time also showed that computation requires a 

---

nonlinearity. A system that simple creates weighted sums of its inputs cannot perform the essential 
requirements of computation. 
We now know that actual biological neurons have many other nonlinearities resulting from the 
electrochemical action of the synapses and the morphology (shape) of the dendrites. Different arrangements 
of biological neurons can perform computations, including subtracting, multiplying, averaging, filtering, 
normalizing, and thresholding signals, among other types of transformations. 
The ability of neurons to perform multiplication is important because it allowed the behavior of one 
network of neurons in the brain to be modulated (influenced) by the results of computations of another 
network. Experiments using electrophysiological measurements on monkeys provide evidence that the rate 
of signaling by neurons in the visual cortex when processing an image is increased or decreased by whether 
or not the monkey is paying attention to a particular area of that image.
25
Human fMRI studies have also 
shown that paying attention to a particular area of an image increases the responsiveness of the neurons 
processing that image in a cortical region called V5, which is responsible for motion detection.
26 
The connectionism movement experienced a setback in 1969 with the publication of the book