Page - 67 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

3.3. Black-boxModeling node is also called a neuron, which is the fundamental unit in neural networks. Each node can have multiple inputs from the input of the network or other nodes, but it has only one output. The output could be either the sum of all inputs (e.g. nodes in the output layer), or the resultof somelocal functions (perceptron) [Bar98]. Thelocal functionofeachnodeiscalledtheactivationfunction,which could be either linear or nonlinear. There are different choices of activation functions (shown in figure 4.9), including the hyperbolic tangent function, softmax function, sinusoidal function, and sigmoid function [HDB+96]. It has been stated in [Hor91] that it is the archi- tectureof themultilayerneuralnetworknot theselectionofactivation functions that gives the neural networks the universal approximation ability, therefore the performance difference between different activa- tion functions is quite small. In the NN estimator and controller used in this dissertation, the hyperbolic tangent function (tanh) [HDB+96] is selected(figure 3.5d),which isdefinedas tanh(x) = sinh(x) cosh(x) = ex−e−x ex+e−x . (3.69) It is selectedbecause itsderivative iseasy tocalculate. The interconnection between any two nodes is called a synaptic link, or synapse, which can transfer the data from one node to another. Each synaptic link has its own weight, which can be considered as the strength of this link. One or more nodes with the same type of functions constitute a block, which is called one layer in neural net- works. Besides normal nodes, in each layer there is one extra node called the bias node, whose output is always one. It never receives any data from other nodes and only transmits the output to nodes in the next layer. The function of bias nodes is to shift activation func- tions todifferentdirectionssothat theresultednetworkisadaptive to different input-outputdynamics [HDB+96]. In principle, a neural network comprises at least two layers, one in- put layer and one output layer. In most cases, a neural network also contains one or more hidden layers, which lie between the input and the output layers. Hidden layers are important for the approximation ability of neural networks. An intuitive interpretation to the function of hidden layers is that the hidden layers receive the raw information 67