Page - 305 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 442 0 10 20 30 0.00 0.05 0.10 0.15 0.20 0.25 0.30 GMM1 102 103 104 105 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Entropy(GMM1) (60%) CELB CEUB CEALB CEAUB MEUB −15 −10 −5 0 5 10 150.00 0.05 0.10 0.15 0.20 GMM2 102 103 104 105 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Entropy(GMM2) (57%) CELB CEUB CEALB CEAUB MEUB −3 −2 −1 0 1 2 30.00 0.05 0.10 0.15 0.20 0.25 0.30 GMM3 102 103 104 105 0.8 1.0 1.2 1.4 1.6 1.8 2.0 Entropy(GMM3) (39%) CELB CEUB CEALB CEAUB MEUB −5 −4 −3 −2 −1 0 1 20.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 GMM4 102 103 104 105 0.6 0.8 1.0 1.2 1.4 1.6 1.8 Entropy(GMM4) (56%) CELB CEUB CEALB CEAUB MEUB −4 −3 −2 −1 0 1 2 3 40.00 0.05 0.10 0.15 0.20 0.25 GMM5 102 103 104 105 1.3 1.4 1.5 1.6 1.7 1.8 1.9 Entropy(GMM5) (0%) CELB CEUB CEALB CEAUB MEUB −2 −1 0 1 20.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 GMM6 102 103 104 105 0.8 1.0 1.2 1.4 1.6 Entropy(GMM6) (0%) CELB CEUB CEALB CEAUB MEUB Figure3.Lowerandupperboundsonthedifferential entropyofGaussianmixturemodels.Onthe left ofeachsubfigure is thesimulatedGMMsignal.Ontherightofeachsubfigure is theestimationof its differentialentropy.Note thatasubsetof theboundscoincidewitheachother inseveral cases. 0 10 20 30 0.00 0.05 0.10 0.15 0.20 0.25 0.30 GMM1 −15−10 −5 0 5 10 150.00 0.05 0.10 0.15 0.20 GMM2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 α 3 4 5 6 7 8 alphadivergence True VR(L) VR(U) −3 −2 −1 0 1 2 30.00 0.05 0.10 0.15 0.20 0.25 0.30 GMM3 −5 −4 −3 −2 −1 0 1 20.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 GMM4 −1.0−0.5 0.0 0.5 1.0 1.5 2.0 α 0 5 10 15 20 alphadivergence True VR(L) VR(U) Figure4.TwopairsofGaussianMixtureModelsandtheirα-divergencesagainstdifferentvaluesofα. The“true”valueofDα isestimatedbyMCusing104 randomsamples.VR(L)andVR(U)denote the variationreducedlowerandupperbounds, respectively. Therangeofα is selectedforeachpair for aclearvisualization. 305