Page - 113 - in Adaptive and Intelligent Temperature Control of Microwave Heating Systems with Multiple Sources

4.2. IntelligentControl Controlpolicypi(s,u) The control policy is a stochastic or deterministic rule by which the controller decides its future (next) actions based on the current state. It could be represented by a lookup table, a probability distribution or a function, mapping from individual states to control actions to be taken under each state. The control policy is the core of a RLC, which isequivalent tothecontrolsolutioninconventionalcontrollers. RewardfunctionR(k) The reward function is defined as the future rewards that the con- troller tries to maximize. It maps each control action to a scalar value, whichisthereward,evaluatingtheintermediatedesirabilityoftheac- tion. If the plant goes from a state of less (higher) value (the value of a state is determined by the value function such as introduced below) to a state of higher (less) value, the corresponding reward is positive (negative), indicating this transition is good (bad) for the plant and controller. The value of rewards could be finite predefined values or calculated results from a function, depending on different formula- tionsof therewardfunction. Valuefunction A value function specifies the long-term expected rewards for each state or state-action pair under a certain control policy. There are two types of value function defined. One is the state value functionVpi(s), which denotes the long-term expected reward starting from the state s using the control policy pi. The state value function is defined as (accordingto [Bar98]) Vpi(s) =E [G(k)|S(k) =s,pi ], (4.32) whereG(k) represents the long-term expected reward. The long-term expectedrewardcanbeformulatedaccordingtodifferentcriteria,and 113