Page - 74 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

3. ModelingMicrowaveHeating whichistheaveragedquadraticerroroveralltrainingdatapair. The principle of gradient descent is used to solve the above learning task, where the update rule for the weight vector (or matrix) is repre- sentedas [HDB+96] wnew=wold+∆w =wold−η∂JL2 ∂w , (3.73) where η is a positive step size and∂JL2/∂w is the gradient vector of w. The weight vector (or matrix) can be defined into different forms depending on the application. For instance, in equation 3.82 weights of different layers are represented by different weight vectors. But in equation 4.21, all weights of the network are included in one weight vector. To calculate the gradient vector, all activation function with the neural network have to be differentiable. Therefore functions like the unit step function are not appropriate. Depending on different presentationsof thetrainingcost function, the learningprocesscanbe applied in twoways. 1. Batch learning: The cost function used in batch learning is exactly the same as equation 3.72, which learns from all data pairs within the set D. In other words, the calculation of ∆w requires informa- tionfromalldatapairs inD. 2. Incremental learning: in this mode of backpropagation, the cost function isgeneratedas [HDB+96] JIL(q) = 1 2 ( YNN(q)−Yd(q) )T( YNN(q)−Yd(q) ) , 1≤ q≤Q. (3.74) The cost function 3.74 consists of only one data pair in each up- date. Instead of learning from all training date pairs at once, the learning data pairs in incremental learning are presented one by oneto thenetwork. Inneuralnetworklearning,onecompleteupdateofweightsusingthe entire data set D is called an epoch [Hay98]. In batch learning, since theentiredataset is includedinthecostfunction(equation 3.72),each update is equivalent to one epoch, which means all weights are up- datedonlyonceperepoch. Inincremental learning,onlyonedatapair 74