Microsoft Word Kurzweil, Ray The Singularity Is Near doc

Perceptrons
by MIT's Marvin Minsky and Seymour Papert.
27
It included a key theorem demonstrating that 
the most common (and simplest) type of neural net used at the time (called a Perceptron, pioneered by 
Cornell's Frank Rosenblatt), was unable to solve the simple problem of determining whether or not a line 
drawing was fully connected.
28
The neural-net movement had a resurgence in the 1980s using a method 
called "backpropagation," in which the strength of each simulated synapse was determined using a learning 
algorithm that adjusted the weight (the strength of the output of each of artificial neuron after each training 
trial so the network could "learn" to more correctly match the right answer. 
However, backpropagation is not a feasible model of training synaptic weight in an actual biological 
neural network, because backward connections to actually adjust the strength of the synaptic connections 
do not appear to exist in mammalian brains. In computers, however, this type of self-organizing system can 
solve a wide range of pattern-recognition problems, and the power of this simple model of self-organizing 
interconnected neurons has been demonstrated. 
Less well know is Hebb's second form of learning: a hypothesized loop in which he excitation of the 
neuron would feed back on itself (possibly through other layers), causing a reverberation (a continued 
reexcitation could be the source of short-term learning). He also suggested that this short-term reverberation 
could lead to long-term memories: "Let us assume then that the persistence or repetition of a reverberatory 
activity (or 'trace') tends to induce lasting cellular changes that add to its stability. The assumption can be 
precisely stated as follows: When an axon of cell A is near enough to excite a cell B and repeatedly or 
persistently take part in firing it, some growth process or metabolic change takes place in one or both cells 
such that A's efficiency, as one of the cell's firing B, is increased." 
Although Hebbian reverberatory memory is not as well established as Hebb's synaptic learning, 
instances have been recently discovered. For example, sets of excitatory neurons (ones that stimulate a 
synapse) and inhibitory neurons (ones that block a stimulus) begin an oscillation when certain visual 
patterns are presented.
29
And researchers at MIT and Lucent Technologies' Bell Labs have created an 
electronic integrated circuit, composed of transistors, that simulates the action of sixteen excitatory neurons 
and one inhibitory neuron to mimic the biological circuitry of the cerebral cortex.
30
These early models of neurons and neural information processing, although overly simplified and 
inaccurate in some respects, were remarkable, given the lack of data and tools when these theories were 
developed. 
Peering into the Brain 

---

We've been able to reduce drift and noise in our instruments to such an extent that we can see the tiniest 
motions of these molecules, through distances that are less than their own diameters....[T]hese kinds of 
experiments were just pipedreams 15 years ago. 
—S
TEVEN 
B
LOCK
,
P
ROFESSOR OF 
B
IOLOGICAL 
S
CIENCES AND OF 
A
PPLIED 
P
HYSICS
,
S
TANFORD 
U
NIVERSITY
Imagine that we were trying to reverse engineer a computer without knowing anything about it (the "black box" 
approach). We might start by placing arrays of magnetic sensors around the device. We would notice that during 
operations that updated a database, significant activity was taking place in a particular circuit board. We would be 
likely to take note that there was also action in the hard disk during these operations. (Indeed, listening to the hard disk 
has always been one crude window into what a computer is doing.) 
We might then theorize that the disk had something to do with the long-term memory that stores the databases and 
that the circuit board that is active during these operations was involved in transforming the data to be stored. This tells 
us approximately where and when the operations are taking place but relatively little about how these tasks are 
accomplished. 
If the computer's registers (temporary memory locations) were connected to front-panel lights (as was the case 
with early computers), we would see certain patterns of light flickering that indicated rapid changes in the states of 
these registers during periods when the computer was analyzing data but relatively slow changes when the computer 
was transmitting data. We might then theorize that these lights reflected changes in logic state during some kind of 
analytic behavior. Such insights would be accurate but crude and would fail to provide us with a theory of operation or 
any insights as to how information is actually coded or transformed. 
The hypothetical situation described above mirrors the sort of efforts that have been undertaken to scan and model 
the human brain with the crude tools that have historically been available. Most models based on contemporary brain-
scanning research (utilizing such methods as fMRI, MEG, and others discussed below) are only suggestive of the 
underlying mechanisms. Although these studies are valuable, their crude spatial and temporal resolution is not 
adequate for reverse engineering the salient features of the brain.