Seite - 256 - in Short-Term Load Forecasting by Artificial Intelligent Technologies

Energies2018,11, 1678 2.5.1.OLS—OrdinaryLeastSquaresRegression Ordinary least squares regression isasimplemodel type inwhich theoutput ismodeledwith thehyperplane thatminimizes thesquaredresidualsbetweenthe targetandtheoutputof themodel. Sometimes referred to asmultiple linear regression, it is a popular model due to its simplicity, computational speed,andthe fact that results canbeeasily interpreted. Becauseof its linearstructure, theOLSmodelunderperformswhenmodelingnonlinear input–outputrelationships. 2.5.2.MLP—MultilayerPerceptron Amultilayerperceptron isasimplekindofneuralnetwork.Neuralnetworkshavebeenapplied toproblems inmanyfields, includingheat loadforecasting,due to theirability tocapturecomplicated relationshipsbetween inputandoutput [7,9,11].Amultilayerperceptronhasat leastonehidden layer betweentheinputandoutput layersof themodelandanonlinearactivationfunctionallowsthemodel to capturenonlinear relationshipsbetween inputandoutput. Agoodcoverageofneuralnetwork models and themultilayerperceptroncanbe found in [20]. Weusedamultilayerperceptronwith onehiddenlayerandtherectifieractivationfunction: f(x)=max(0,x).Wehaveexperimentedwith addingmorehidden layers,but the increase in themodelaccuracywasnot largeenoughto justify the growth inmodelcomplexityandtheriskofoverfitting. Besides thesimplemultilayerperceptron,wehaveexperimentedwithamoreadvancedtypeof neuralnetwork. Recurrentneuralnetworkswith longshort-termmemory (LSTM)units [21]were implemented in an attempt to simplify the feature selection and leave it to themodel todiscover relevant lagsofheat loadandweatherdata.Our initialLSTMnetworksyieldedresults comparable to thesimplermodels includedin thiswork,but theirperformance tendedtobe inconsistent. Thebenefit ofsimplifiedfeatureselectionmayalsobeoutweighedbyamorecomplicatedmodelselectionand trainingprocedure. TheLSTMmodelingforheat loadforecastingrequiresmoreworkandwillbe left for futurework. 2.5.3. SVR—SupportVectorRegression Support vector regression is the application of support vectormachinemodels to regression problemsandwasfirst introducedin[22]. Supportvectorregressionhasacomputationaladvantage inveryhighdimensional featurespaces. Themodelonlydependsonasubsetof inputdata,because it minimizesacost function that is insensitive topointswithinacertaindistance fromtheprediction. Thecost function is less sensitive tosmall errorsandlesssensitive tovery largeerrorsandoutliers, comparedto thequadratic cost functionminimized in theordinary least squares regression. Toavoid overfitting, themodel isgovernedbyaregularizationparameterC, thatensures that theparametersof themodeldonotgrowuncontrollably. Thesmaller thevalueofC, theharder largemodelparameters arepenalized. Support vector regression is explained ingreat detail in [19,20]. By employing the so-called“kernel trick”, supportvectorregressioncanhandlenonlinear input–outputrelationships. A very popular kernel function is the radial basis function kernel (RBF),which has been proven effective in thisapplicationaswell. TheRBFkernel isgovernedbyakernelparameterγ. Thegreater thevalueofγ, themorepronethemodel is tooverfitting,but ifγ is chosentoosmall, themodelmay beunderfittingandfail tocaptureactual input–outputrelationships. Summing up, the threemachine learningmodelsOLS,MLP, and SVRwere chosen because theyhaveallbeensuccessfullyapplied toheat load forecasting in thepast. Usingwell-established algorithmsallowsus to focusonthemainresearchquestion:whetherconventionalheat loadforecasts canbeimprovedbyaddingnewtypesofdata. Eachof themodelshaveadvantagesanddisadvantages. Theadvantageof theOLSmodel is that it iscomputationallycheap,anditsestimatedparameterscarry aphysical interpretation. Thedisadvantageis that themodel is linearandfails tocapturenonlinearities in input–output relationships. Theadvantageof theMLPmodel is that it is capableofcapturingvery complexrelationshipsbetween inputandoutput.Adisadvantageofneuralnetworkmodels, suchas 256