Page - 276 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 407 Given pandq inPμ, letusdefine κ(α)=− log (∫ T p 1−α 2 q 1+α 2 dμ ) , forα∈ [−1,1], whichcanbeusedtoexpress theRényidivergenceas D(α)R (p‖ q)= 4 1−α2κ(α), forα∈ (−1,1). The functionκ(α),whichdependson pand q, canbedefinedas theuniquenon-negative real number forwhich ∫ T exp (1−α 2 ln(p)+ 1+α 2 ln(q)+κ(α) ) dμ=1. (8) Thefunctionκ(α)makes theroleofanormalizingterm.ThegeneralizationofRényidivergence, whichwepropose, is based on the interpretation of κ(α) given in (8). Instead of the exponential function,weconsideraϕ-function in (8). Fixanyϕ-functionϕ:R→ (0,∞). Givenany pandq inPμ,we takeκ(α)= κ(α;p,q)≥0so that∫ T ϕ (1−α 2 ϕ−1(p)+ 1+α 2 ϕ−1(q)+κ(α)u0 ) dμ=1, (9) or, inotherwords, the terminside the integral isaprobabilitydistribution inPμ. Theexistenceand uniquenessofκ(α)asdefinedin(9) isguaranteedbycondition(a3’). Wedefineageneralizationof theRényidivergenceoforderα∈ (−1,1)as D(α)ϕ (p‖ q)= 41−α2κ(α). (10) Forα=±1, thisgeneralization isdefinedasa limit: D(−1)ϕ (p‖ q)= lim α↓−1 D(α)ϕ (p‖ q), (11) D(1)ϕ (p‖ q)= lim α↑1 D(α)ϕ (p‖ q). (12) Thecasesα=±1arerelated toageneralizationof theKullback–Leiblerdivergence, theso-called ϕ-divergence,whichwas introducedby the authors in [12]. The ϕ-divergence is givenby (Itwas pointed out to us by an anonymous referee that this form of divergence is a special case of the (ρ,τ)-divergenceforρ=ϕ−1 and f = ρ−1 (seeSection3.5 in[19])apart fromaconformalfactor,which is thedenominatorof (13)): Dϕ(p‖ q)= ∫ T ϕ−1(p)−ϕ−1(q) (ϕ−1)′(p) dμ∫ T u0 (ϕ−1)′(p)dμ . (13) Undersomeconditions, the limit in (11)or (12) isfinite-valuedandconverges to theϕ-divergence: D(−1)ϕ (p‖ q)=D(1)ϕ (q‖ p)=Dϕ(p‖ q)<∞. (14) Toshow(14),wemakeuseof the followingresult. Lemma2. Assumethat ϕ(·) is continuouslydifferentiable. If forα0,α1∈R, the expression∫ T ϕ (1−α 2 ϕ−1(p)+ 1+α 2 ϕ−1(q) ) dμ<∞ (15) 276