Page - 394 - in Short-Term Load Forecasting by Artificial Intelligent Technologies

Energies 2018,11, 242 Theactivationprobabilityof the jthhiddenunitcanbecomputedasfollowswhenavisiblevector v(v1, . . . ,vi, . . . ,vn) isgiven[32] p(hj=1|v)=σ(bj+ n ∑ i=1 viwij), (1) whereσ(·) is thesigmoidfunction,wij is theconnectionweightbetweenthe ithvisibleunitand jth hiddenunit, andbj is thebiasof the jthhiddenunit. Similarly,whenahiddenvectorh(h1, . . . ,hj, . . . ,hm) isknown, theactivationprobabilityof the ithvisibleunit canbecomputedas follows: p(vi=1|h)=σ(ai+ m ∑ j=1 hjwij), (2) where i=1,2, . . . ,n, and ai is thebiasof the ithvisibleunit. Hinton et al. [33] have proposed the contrastive divergence (CD) algorithm to optimize the RBM.TheCDalgorithmbasedRBM’s iterative learningprocedures forbinomialunitsare listedas follows[32]. Step1: Initialize thenumberofvisibleunitsn, thenumberofhiddenunitsm, thenumberof training dataN, theweightingmatrixW, thevisiblebiasvectora, thehiddenbiasvectorb andthe learningrate . Step2: Assignasamplex fromthetrainingdata tobe the initial statev0 of thevisible layer. Step3: Calculatep(h0j=1|v0)accordingtoEquation(1),andextracth0j∈{0,1} fromtheconditional distribution p(h0j=1|v0),where j=1,2, . . . ,m. Step4: Calculatep(v1i=1|h0)accordingtoEquation(2),andextractv1i∈{0,1} fromtheconditional distribution p(v1i=1|h0),where i=1,2, . . . ,n. Step5: Calculate p(h1j=1|v1)accordingtoEquation(1). Step6: Update theparametersaccordingto the followingequations: W=W+ (p(h0=1|v0)v0T−p(h1=1|v1)v1T), a= a+ (v0−v1), b=b+ (p(h0=1|v0)−p(h1=1|v1)). Step7: Assignanother sample fromthe trainingdata tobe the initial statev0 of thevisible layer, anditerateSteps3 to7until all theN trainingdatahavebeenused. 2.2.DeepBeliefNetwork Asaforementioned, theDBNasamiraculousdeepmodel isastackofRBMs[11,30,34,35]. Figure2 illustrates thearchitectureof theDBNwithkhiddenlayersandits layer-wisepre-trainingprocess. Theactivationof thekthhiddenlayerwithrespect to inputsamplex canbecomputedas Ak(x)=σ(bk+Wkσ(· · ·+W2σ(b1+W1x))) , (3) whereWu andbu (u=1,2, . . . ,k) are, respectively, theweightingmatricesandhiddenbiasvectorsof theuthRBM.Furthermore,σ is the logistic sigmoidfunctionσ(x)=1/(1+e−x). Inorder toobtainbetter featurerepresentation, theDBNutilizesdeeparchitectureandadopts the layer-wisepre-training tooptimize the inter-layerweightingmatrix [11]. The trainingalgorithmof the DBNwillbegiven in thenextsection indetail. 394