Page - 298 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 442 4.1. BasicBounds Forapairofgivenm(x)andm′(x),weonlyneedtoderiveboundsofHα(m :m′) inEquation(31) so thatLα(m :m′)≤Hα(m :m′)≤Uα(m :m′). Then theα-divergenceDα(m :m′) canbebounded byalinear transformationof therange [Lα(m :m′),Uα(m :m′)]. In thefollowingwealwaysassume without lossofgeneralityα≥1/2.OtherwisewecanboundDα(m :m′)byconsideringequivalently theboundsofD1−α(m′ :m). Recall that ineachelementaryslab Ir,wehave max { kw (r)p (r)(x),wδ(r)pδ(r)(x) } ≤m(x)≤ kwδ(r)pδ(r)(x). (32) Notice thatkw (r)p (r)(x),wδ(r)pδ(r)(x), andkwδ(r)pδ(r)(x)areall singlecomponentdistributions up toascalingcoefficient. Thegeneral thinking is toboundthemulti-componentmixturem(x)by singlecomponentdistributions ineachelementary interval, so that the integral inEquation(31)canbe computedinapiecewisemanner. For theconvenienceofnotation,werewriteEquation(32)as cν(r)pν(r)(x)≤m(x)≤ cδ(r)pδ(r)(x), (33) where cν(r)pν(r)(x) := kw (r)p (r)(x) or wδ(r)pδ(r)(x), cδ(r)pδ(r)(x) := kwδ(r)pδ(r)(x). (34) (35) If1/2≤α<1, thenbothxα andx1−α aremonotonically increasingonR+. Thereforewehave Aαν(r),ν′(r)(Ir)≤ ∫ Ir m(x)αm′(x)1−αdx≤Aαδ(r),δ′(r)(Ir), (36) where Aαi,j(I)= ∫ I (cipi(x)) α ( c′jp ′ j(x) )1−α dx, (37) and Idenotesan interval I=(a,b)⊂R. Theothercaseα>1 is similarbynoting thatxα andx1−α are monotonically increasinganddecreasingonR+, respectively. Inconclusion,weobtain the following boundsofHα(m :m′): If 1/2≤α<1, Lα(m :m′)= ∑ r=1 Aαν(r),ν′(r)(Ir), Uα(m :m ′)= ∑ r=1 Aαδ(r),δ′(r)(Ir); (38) ifα>1, Lα(m :m′)= ∑ r=1 Aαν(r),δ′(r)(Ir), Uα(m :m ′)= ∑ r=1 Aαδ(r),ν′(r)(Ir). (39) The remaining problem is to compute the definite integral Aαi,j(I) in the above equations. Here we assume all mixture components are in the same exponential family so that pi(x)= p(x;θi)= h(x)exp ( θ i t(x)−F(θi) ) ,whereh(x) isabasemeasure, t(x) isavectorofsufficient statistics, andthe functionF isknownas thecumulantgeneratingfunction. Then it is straightforward fromEquation(37) that Aαi,j(I)= c α i (c ′ j) 1−α ∫ I h(x)exp (( αθi+(1−α)θ′j ) t(x)−αF(θi)−(1−α)F(θ′j) ) dx. (40) 298