Page - 300 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 442 m(x)αm′(x)1−α/m(x) is likely tohaveasmallvariancewhenxvaries insideaslab Ir, especiallywhen α is close to1.Wewill thereforeboundthis ratio termin ∫ Ir m(x)αm′(x)1−αdx= ∫ Ir m(x) ( m(x)αm′(x)1−α m(x) ) dx= k ∑ i=1 ∫ Ir wipi(x) ( m′(x) m(x) )1−α dx. (47) No matter α < 1 or α > 1, the function x1−α must be monotonic onR+. In each slab Ir, (m′(x)/m(x))1−α rangesbetweenthese twofunctions:( c′ ν′(r)p ′ ν′(r)(x) cδ(r)pδ(r)(x) )1−α and ( c′ δ′(r)p ′ δ′(r)(x) cν(r)pν(r)(x) )1−α , (48) where cν(r)pν(r)(x), cδ(r)pδ(r)(x), c′ν′(r)p ′ ν′(r)(x)and c ′ δ′(r)p ′ δ′(r)(x)aredefinedinEquations (45)and(46). Similar to thedefinitionofAαi,j(I) inEquation(37),wedefine Bαi,j,l(I)= ∫ I wipi(x) ( c′lp ′ l(x) cjpj(x) )1−α dx. (49) Thereforewehave, Lα(m :m′)=minS, Uα(m :m′)=maxS, S= { ∑ r=1 k ∑ i=1 Bαi,δ(r),ν′(r)(Ir), ∑ r=1 k ∑ i=1 Bαi,ν(r),δ′(r)(Ir) } . (50) Theremainingproblemis toevaluateBαi,j,l(I) inEquation(49). Similar toSection4.1,assuming thecomponentsare in thesameexponential familywithrespect to thenaturalparametersθ,weget Bαi,j,l(I)=wi c′1−αl c1−αj exp ( F(θ¯)−F(θi)−(1−α)F(θ′l)+(1−α)F(θj) )∫ I p(x; θ¯)dx. (51) If θ¯= θi+(1−α)θ′l−(1−α)θj is in thenaturalparameterspace,Bαi,j,l(I)canbecomputedfromthe CDFof p(x; θ¯); otherwiseBαi,j,l(I) canbenumerically integrated by its definition inEquation (49). Thecomputationalcomplexity is thesameas thebounds inSection4.2, i.e.,O( (k+k′)). Wehave introducedthreepairsofdeterministic lowerandupperboundsthatenclose the true valueofα-divergencebetweenunivariatemixturemodels. Thus thegapbetweentheupperandlower boundsprovides the additive approximation factor of the bounds. We conclude by emphasizing that the presentedmethodology canbe easily generalized to other divergences [35,40] relying on Hellinger-type integralsHα,β(p : q)= ∫ p(x)αq(x)βdx like theγ-divergence [44]aswell asentropy measures [45]. 5. LowerBoundsof the f-Divergence The f-divergencebetweentwodistributionsm(x)andm′(x) (notnecessarilymixtures) isdefined fora convexgenerator f by: Df(m :m′)= ∫ m(x)f ( m′(x) m(x) ) dx. If f(x)=− logx, thenDf(m :m′)=KL(m :m′). 300