Page - 127 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

4.2. IntelligentControl TD learning always converges to the optimal control policy with the appropriate learningrate (equation 4.46) andadditional assumptions [Bar98]. Therefore it ismorestable tobeappliedinthepractice. Empiricalknowledgebaseddiscretizationandoptimization Most lookup table based TD learning methods suffer the curse of di- mensionality [Sut96], which means the overall computational time and memory of the learning grows exponentially with the the dimen- sion of states and actions space. For example, it is assumed that the temperature vector consists of temperatures of 5 different locations and the control input vector contains inputs of 6 different heating sources. Each temperature value takes 3 possible states (temperature high/middle/low) and each control input variable takes 2 possible states (switch ON/OFF). The number of overall possible state-action pairsof theMDPis35×26= 15552,whichmeansthelookuptablehas to estimate and save 15552 differentQvalues. If the control period is 1 s, then it needs at least 15552 s to go through all state-action pairs, not even mentioning to get the accurate estimation of each individual state-actionvalues. As stated in [BM03], empirical knowledge based options and policies are great approaches to accelerate the learning and provide guaran- tees about the system performance during the learning. In order to make the TD learning controller more realistic in practice, empirical knowledgebasedoptimizationisemployedtofurthersimplifytheen- tire MDP and the correspondingQ(λ) learning controller. Since the complexity of the controller mainly depends on the state and the ac- tion spaces, the simplification focuses on the reduction of dimensions ofboth thestateandtheactionspaces. Recalling the control task represented in equation 4.30 which aims to controlNmeasured temperatures converge to the target temperature and meanwhile reduce the temperature window between Ymax and Ymin. To reduce the number of the state space, this control task can be modified into a simpler task such as controlYmax andYmin converge the target temperature. Because the modified control task is a neces- sary condition for the original control task. In other words, the over- all temperature distribution will be improved if both Ymax and Ymin 127