Page - 185 - in Short-Term Load Forecasting by Artificial Intelligent Technologies

Energies2018,11, 2008 TheANNis trainedusingthebackpropagationalgorithm[24]. Thebackpropagationalgorithm is runoverall the trainingdata. This is calledanepoch.WhentraininganANN,manyepochsare performedwith a termination criterion suchas amaximumnumberof epochsor the error falling belowathreshold. Next,wedescribeaDNN.ADNNisessentialanANNwithmanyhiddenlayers. Thedifference is in the trainingprocess. Rather thantraining thenetworkusingonly thebackpropagationalgorithm, an initializationphase isdoneusing the contrastivedivergencealgorithm[25,26]. The contrastive divergence algorithm isperformedona restrictedBoltzmannmachine (RBM). Figure 4 illustrates aRBMwith fourvisiblenodesand threehiddennodes. Important tonote is thatunlike theANN, the arrowspoint in both directions. This is to indicate that the contrastive divergence algorithm updates theweightsbypropagatingtheerror inbothdirections. Figure4. ArestrictedBoltzmannmachinewith fourvisibleunits and threehiddenunits. Note the similaritywithasingle layerofaneuralnetwork. Similar to anANN, a RBMhas bias terms. However, since the error is propagated in both directions thereare twobias terms, band c. Thevisibleandhiddennodesare calculated fromone another [26]. Letvi represent the ithvisiblenode,wi theweightof the ithvisiblenode, c thebias term, n thenumberofvisiblenodes,andh thehiddennode. h=σ ( n ∑ i=1 wivi+c ) , (7) whichcanberewritten invectornotationforallhiddenunitsas h=σ(Wv+c). (8) Similarly, thevisiblenodecanbecalculated in termsof thehiddennodes. Lethj represent the jth hiddennode,wj theweightof the jthhiddennode,b thebias term,m thenumberofhiddennodes, andv thevisiblenode.Then v=σ ( m ∑ j=1 wjhj+b ) , (9) whichcanberewritten invectornotationforallvisibleunitsas v=σ ( WTh+b ) , (10) whereWT is the transposeofW. TrainingaRBMisdone in threephasesasdescribedinAlgorithm1for trainingvectorv0anda trainingrateε. Algorithm1isperformedoniterations (epochs)ofall inputvectors. 185