Page - 112 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

4. ControlSystemDesign Meanwhile, the name ’plant’ is used to denote the external environ- ment inRL. In most cases, states in a MDP are discrete and finite, but the above expression 4.31 can be extended into situations of infinite or contin- uous states [SK00]. An important property always assumed in MDPs is that the next stateS(k+1) and rewardR(k+1) depends only on the current stateS(k) and action U(k), which makes the MDP simi- lar to an ARX model with one-step delay. For plants where the above propertydoesnot fullyhold, it is still appropriate toconsider themas approximated MDPs, as long as the current state can provide a good basis forpredicting thenextstateandreward. Despite significant differences regarding the process of system mod- eling and the controller design, it was found that RLC is closely re- lated to conventional control methods. It has been proved in [SBW92] that RLC is essentially an optimal control approach, and the relation- ship between RLC and adaptive feedback control was well explained in [LV09]. More and more applications implement the principle of RL in conventional control frameworks, such as the RL based fuzzy controller [JLL00] [Lin03] and RL based online PID tuning algorithm [HB00]. Detailed information of RL refers to literatures as [Bar98], [Alp04]and[B+06]. The structure of a normal RLC system is shown in figure 4.8. In gen- eral, aRLCsystemconsistsof followingfourparts. RL Controller Action U(k) Plant (HEPHAISTOS) Reward R(k)State S(k) R(k+1) S(k+1) Figure4.8. Reinforcement learningcontroller. 112