Page - 188 - in Short-Term Load Forecasting by Artificial Intelligent Technologies

Energies2018,11, 2008 dayswithhigher loads. Thesedaysare important, as theyarewhennaturalgas is themostexpensive, whichmeans thatpurchasinggason thespotmarketorhavingbought toomuchgascanbecostly. Unfortunately,RMSE ismagnitudedependent,meaningthat largersystemshave largerRMSE if the percent error is constant,whichmakes it apoormetric for comparing theperformanceof amodel acrossdifferentsystems. Anothercommonmetric forevaluatingforecasts ismeanabsolutepercenterror, MAPE=100 1 N N ∑ n=1 |sˆ(n)−s(n)| s(n) . (12) UnlikeRMSE,MAPE isunitlessandnotdependentonthemagnitudeof thesystem.Thismeans that it ismoreuseful forcomparing theperformanceofamethodbetweenoperatingareas. Itdoes, however,putsomeemphasisonthe lowestflowdays,which,ontopofbeingthe least importantdays to forecast correctly, areoften theeasiestdays to forecast. Assuch,MAPE isnot thebestmetric for lookingat theperformanceof themodelacrossall thedays inayear,butcanbeusedtodescribe the performanceonasubsetof similardays. Theerrormetricused in thispaper isweightedMAPE: WMAPE=100 N ∑ n=1 |sˆ(n)−s(n)| N ∑ n=1 s(n) (13) Thiserrormetricdoesnotemphasize the lowflowandless importantdayswhilebeingunitless andindependentof themagnitudeof thesystem.Thismeans that it is themosteffectiveerrormetric forcomparingtheperformanceofourmethodsover thecourseofa fullyear. Themeanandstandarddeviationof theperformanceof eachmodelover the62data sets are shown in Table 1. As expected, theDNNhas a lowermeanWMAPE than the linear regression andANNforecasters,meaning that generally, theDNNperformsbetter than the simplermodels. Additionally, the largeDNNmarginallyoutperformsthesmallDNNintermsofWMAPE.Bothresults are shown tobe statistically significant later in this section. In addition to themean, the standard deviationof theperformancesof the twoDNNarchitecturesaresmaller thanthatof theLRandANN. This isan important resultbecause itpoints toamoreconsistentperformanceacrossdifferentareasas wellasbetterperformanceoverall. Table1.Themeanandstandarddeviationof theperformanceof the fourmodelsonall62areas. LRWMAPE ANNWMAPE DNNWMAPE LargeDNNWMAPE Mean 6.41 6.41 5.78 5.58 StandardDeviation 2.49 2.83 2.11 2.09 Simplystatingthemeanperformancedoesnot tellusmuchwithout lookingat thedifferences in performance for each of the 62 areas individually, which is shown succinctly in Figures 6 and7. Figure 6a,b andFigure 7 arehistogramsof thedifference inperformanceonall 62 areasof twoforecastingmethods. Bypresenting theresults thisway,wecanvisualize thegeneraldifference in performanceforeachofthe62operatingareas.Additionally, t-testscanbeperformedonthehistograms todeterminethestatistical significanceof thedifference.Right-tailed t-testswereperformedonthe distributionsinFigure6a,b. Theresultingp-valuesare1.2×10−7 and6.4×10−4, respectively,meaning that theDNNperformedbetter, ingeneral, thantheANNorLR,andthat thedifferenceinperformance is statisticallysignificant inbothcases. 188