6 MLP神经网络.ppt

Multi-layer Perceptrons
Junying Zhang
contents
structure
universal theorem
MLP for classification
mechanism of MLP for classification
nonlinear mapping
binary coding of the areas
MLP for regression
learning algorithm of the MLP
back propagation learning algorithm
heuristics in learning process
XOR and Linear Separability Revisited
Remember that it is not possible to find weights that enable Single Layer Perceptrons to deal with non-linearly separable problems like XOR:
However, Multi-Layer Perceptrons (MLPs) are able to cope with non-linearly separable problems. Historically, the problem was that there were no learning algorithms for training MLPs. Actually, it is now quite straightforward.
Structure of an MLP
it posed of several layers
neurons within each layer are not connected
ith layer is only fully connected to the (i+1)jth layer
Signal is transmitted only in a feedforward manner
Structure of an MLP
Model of each neuron in includes
A nonlinear activation function  is nonlinear
The function is smooth  derivative
Generally, sigmoidal function
work contains one or more layers of hidden neurons that are not part of input or output of  enable to plex tasks
Expressive power of an MLP
Questions
How many hidden layers are needed?
How many units should be in a (the) hidden layer?
Answers
Komogorov’s mapping work existence theorem (universal theorem)
Komogorov’s mapping work existence theorem (universal theorem)
Any continuous function g(x) defined on the unit hypercube can be represented in the from
For properly chosen functions and
It is impractical
the functions and are not the simple weighted sums passed through nonlinearities favored in works
It tells us very little about how to find the nonlinear functions based on data — the central problem work based pattern recognition
those functions can be plex; they are not smooth
Komogorov’s mapping work existence theorem (universal theorem)
Any continuous function g(x) can be approximated to arbitra