2282

vector W that provides the best possible approximation of the function *g*(*X*) based

on the training input [*X*].

The standard or basic training method is the *Gradient Descent Method. *In this

method, weight changes move the weights in the direction where the error declines

most quickly. Training is carried out by assigning random initial weights to each of

the neurons (usually between 0.1 and 1.0) and then presenting sets of known input

and *target *(*output*) values to the network. The network estimates the output value

from the inputs, compares the model predicted output to the target value, and then

adjusts the weights in order to reduce the mean squared difference between the net-

work output and the target values. The complete inputoutput sets are often run

through the network for several iterations (or epochs) until either the mean square

error is reduced to a given level or reaches a minimum, or until the network has

been trained for a given number of iterations.

If we let *w*m represent the value of weight *w *after *m*th iteration in a neuron, then

wm

wm

(6)

1

where wm is the change in the weight *w *at the end of iteration *m*. It is calculated by

wm

e*d*m

(7)

where e is the user-specified parameter controlling the proportion by which the

weights are modified. The term *d*m is given by

∂*E*

n (

)

(8)

∂*w*m

1

where *N *is the total number of examples and *E *is the simulation output error.

In neural network model development, the first step is to design a specific network

architecture that includes a specific number of layers, each consisting of a certain

number of neurons. The size and structure of the network needs to match the nature

of the investigated phenomenon. Because it is usually not well known at the early

stage, the task is not easy and often involves a trial and errors approach. The new

network is then subjected to the training process. In that phase, neurons apply an

iterative process to the number of inputs (variables) to adjust the weights of the

network in order to optimally predict (in traditional terms one could say, find a fit

to) the sample data on which the training is performed. After learning from an exist-

ing data set, another new data set is used to validate or verify the performance of

the trained neural network. If the neural network performance is satisfactory in model

verification, it is capable in model predictions using other new data inputs.

One of the major advantages of neural networks is that, theoretically, they are

capable of approximating any continuous function (Haykin, 1999). The resulting

Integrated Publishing, Inc. |