Neural Networks

A neural network also known as an artificial neural network provides a
unique computing architecture whose potential has only begun to be tapped. They
are used to address problems that are intractable or cumbersome with traditional
methods. These new computing architectures are radically different from the
computers that are widely used today. ANN\'s are massively parallel systems that
rely on dense arrangements of interconnections and surprisingly simple
processors (Cr95, Ga93).
Artificial neural networks take their name from the networks of nerve
cells in the brain. Although a great deal of biological detail is eliminated in
these computing models, the ANN\'s retain enough of the structure observed in the
brain to provide insight into how biological neural processing may work (He90).
Neural networks provide an effective approach for a broad spectrum of
applications. Neural networks excel at problems involving patterns, which
include pattern mapping, pattern completion, and pattern classification (He95).
Neural networks may be applied to translate images into keywords or even
translate financial data into financial predictions (Wo96).
Neural networks utilize a parallel processing structure that has large
numbers of processors and many interconnections between them. These processors
are much simpler than typical central processing units (He90). In a neural
network, each processor is linked to many of its neighbors so that there are
many more interconnections than processors. The power of the neural network
lies in the tremendous number of interconnections (Za93).
ANN\'s are generating much interest among engineers and scientists.
Artificial neural network models contribute to our understanding of biological
models. They also provide a novel type of parallel processing that has powerful
capabilities and potential for creative hardware implementations, meets the
demand for fast computing hardware, and provides the potential for solving
application problems (Wo96).
Neural networks excite our imagination and relentless desire to
understand the self, and in addition, equip us with an assemblage of unique
technological tools. But what has triggered the most interest in neural
networks is that models similar to biological nervous systems can actually be
made to do useful computations, and furthermore, the capabilities of the
resulting systems provide an effective approach to previously unsolved problems
Neural network architectures are strikingly different from traditional
single-processor computers. Traditional Von Neumann machines have a single CPU
that performs all of its computations in sequence (He90). A typical CPU is
capable of a hundred or more basic commands, including additions, subtractions,
loads, and shifts. The commands are executed one at a time, at successive steps
of a time clock. In contrast, a neural network processing unit may do only one,
or, at most, a few calculations. A summation function is performed on its
inputs and incremental changes are made to parameters associated with
interconnections. This simple structure nevertheless provides a neural network
with the capabilities to classify and recognize patterns, to perform pattern
mapping, and to be useful as a computing tool (Vo94).
The processing power of a neural network is measured mainly be the
number of interconnection updates per second. In contrast, Von Neumann machines
are benchmarked by the number of instructions that are performed per second, in
sequence, by a single processor (He90). Neural networks, during their learning
phase, adjust parameters associated with the interconnections between neurons.
Thus, the rate of learning is dependent on the rate of interconnection updates
Neural network architectures depart from typical parallel processing
architectures in some basic respects. First, the processors in a neural network
are massively interconnected. As a result, there are more interconnections than
there are processing units (Vo94). In fact, the number of interconnections
usually far exceeds the number of processing units. State-of-the-art parallel
processing architectures typically have a smaller ratio of interconnections to
processing units (Za93). In addition, parallel processing architectures tend to
incorporate processing units that are comparable in complexity to those of Von
Neumann machines (He90). Neural network architectures depart from this
organization scheme by containing simpler processing units, which are designed
for summation of many inputs and adjustment of interconnection parameters.
The two primary attractions that come from the computational viewpoint
of neural networks are learning and knowledge representation. A lot of
researchers feel that machine learning techniques will give the best hope for
eventually being able to perform difficult artificial intelligence tasks (Ga93).
Most neural networks learn from examples, just like children learn to
recognize dogs from examples of dogs (Wo96). Typically, a neural network is
presented with a training set consisting of a group of examples from which the
network can learn. These examples, known as training patterns, are represented
as vectors, and can be taken from such sources as images, speech signals, sensor
data, and diagnosis information (Cr95, Ga93).
The most common training scenarios utilize supervised learning, during