Page - 69 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

3.3. Black-boxModeling Accordingtohowdifferentnodesareinterconnected,neuralnetworks canbedividedintotwotypes. Thefirsttypeiscalledfeedforwardneu- ral networks (FNN), as the one shown in figure 3.4, where the data is sent fromtheinput layer,passingthroughhiddenlayers totheoutput layer. FNN assume that the output of the system being approximated is only determined by the input, and there is no interconnection be- tween nodes in the same layer or feedbacks from the following layers to the former layer (m+1-th layer tom-th layer). Incontrast toFNN,thereisanothertypeofneuralnetworks,calledre- current neural networks (RNN), where interconnections are allowed between any arbitrary nodes. The concept of RNN contains differ- ent possibilities, like the networks with feedbacks from hidden lay- ers to the input (context) layer (like the Elman networks [Elm90]), or asynchronous fully connected networks (like the Hopfield networks [Hop82]). Feedbacks within the RNN creating internal loops or states givethenetworktheabilitytoapproximatemorecomplicatedsystems with dynamic temporal behaviors. In principle, RNNs can be much more complex and powerful than FNNs. A RNN structure consisting ofasimple feedback loopisshowninfigure 3.6. The overall function of a neural network f(x) is the expansion of all activation functions. For the NN in figure 3.4, the overall function of thefirstoutputcanberepresentedas YNN,1=f1(uNN,1,uNN,2) =g3,1   3∑ i=1 w21,ig2,i   2∑ j=1 w1i,jg1,j(uNN,j)+w 1 i,3   +w21,4   , (3.70) wheregi,j is the activation function of j-th node in i-th layer, andwli,j represents the weight from j-th node in the l-th layer to the i-th node in the (l+1)-th layer (the bias node is labeled as the last node in each layer). If there is no interconnection between any two nodes, then the corresponding weight equals to zero. By adjusting the weight of each synaptic link or number of nodes in hidden layers, the behavior of the neural network is also changed. Adjustments of weights are repeated iteratively or recursively until the desired system dynamics 69