Page - 294 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 442 expectgoodqualityboundswithasmallgapwhenthemixturecomponentsaresimilarasmeasured byKLdivergence. 2.3. CaseStudies In the following,we instantiate theproposedmethodforseveralprominentcasesonthemixture ofexponential familydistributions. 2.3.1. TheCaseofExponentialMixtureModels Anexponentialdistributionhasdensity p(x;λ)=λexp(−λx)definedonX=[0,∞) forλ>0. ItsCDFisΦ(x;λ)=1−exp(−λx). Anytwocomponentsw1p(x;λ1)andw2p(x;λ2) (withλ1 =λ2) haveaunique intersectionpoint x = log(w1λ1)− log(w2λ2) λ1−λ2 (16) ifx ≥0;otherwise theydonot intersect. Thebasic formulas toevaluate theboundsare Ci,j(a,b)= log ( λ′jw ′ j ) Mi(a,b)+wiλ′j [( a+ 1 λi ) e−λia− ( b+ 1 λi ) e−λib ] , (17) Mi(a,b)=−wi ( e−λia−e−λib ) . (18) 2.3.2. TheCaseofRayleighMixtureModels ARayleighdistributionhasdensity p(x;σ)= x σ2 exp ( − x22σ2 ) , definedonX=[0,∞) forσ> 0. ItsCDFisΦ(x;σ)=1−exp ( − x22σ2 ) .Anytwocomponentsw1p(x;σ1)andw2p(x;σ2) (withσ1 =σ2) must intersectatx0=0andcanhaveatmostoneother intersectionpoint x = √√√√logw1σ22 w2σ21 / ( 1 2σ21 − 1 2σ22 ) (19) if thesquareroot iswelldefinedandx >0.Wehave Ci,j(a,b)= log w′j (σ′j)2 Mi(a,b)+ wi 2(σ′j)2 [ (a2+2σ2i )e − a2 2σ2i −(b2+2σ2i )e − b2 2σ2i ] −wi ∫ b a x σ2i exp ( − x 2 2σ2i ) logxdx, (20) Mi(a,b)=−wi ( e − a2 2σ2i −e − b2 2σ2i ) . (21) The last terminEquation (20)doesnothaveasimple closed form(it requires theexponential integral,Ei).Oneneedanumerical integrator tocompute it. 2.3.3. TheCaseofGaussianMixtureModels The Gaussian density p(x;μ,σ) = 1√ 2πσ e−(x−μ)2/(2σ2) has supportX = R and parameters μ ∈R and σ > 0. ItsCDF isΦ(x;μ,σ) = 12 [ 1+erf(x−μ√ 2σ ) ] , where erf is theGauss error function. The intersection point x of two componentsw1p(x;μ1,σ1) andw2p(x;μ2,σ2) can be obtained by solving thequadratic equation log(w1p(x;μ1,σ1)) = log(w2p(x;μ2,σ2)),whichgives atmost two 294