Page - 271 - in Differential Geometrical Theory of Statistics

entropy Article GeometryInducedbyaGeneralizationof RényiDivergence DavidC.deSouza1,†,RuiF.Vigelis 2,*,† andCharlesC.Cavalcante3,† 1 InstitutoFederaldoCeará,CampusMaracanaú,Fortaleza61939-140,Brazil;davidcs@ifce.edu.br 2 ComputerEngineeringSchool,CampusSobral,FederalUniversityofCeará,Sobral62010-560,Brazil 3 DepartmentofTeleinformaticsEngineering,FederalUniversityofCeará,Fortaleza60455-900,Brazil; charles@ufc.br * Correspondence: rfvigelis@ufc.br † Theseauthorscontributedequally to thiswork. AcademicEditors: FrédéricBarbarescoandFrankNielsen Received: 6September2016;Accepted: 11November2016;Published: 17November2016 Abstract: In thispaper,weproposeageneralizationofRényidivergence,andthenweinvestigate its inducedgeometry. Thisgeneralization isgivenintermsofaϕ-function, thesamefunctionthat isused in thedefinitionofnon-parametric ϕ-families. Thepropertiesof ϕ-functionsproved tobe crucial in thegeneralizationofRényidivergence.Assumingappropriateconditions,weverify that thegeneralizedRényidivergence reduces, in a limiting case, to the ϕ-divergence. Ingeneralized statisticalmanifold, theϕ-divergence inducesapairofdualconnectionsD(−1) andD(1).Weshow that the familyofconnectionsD(α) inducedbythegeneralizationofRényidivergencesatisfies the relationD(α) = 1−α2 D (−1)+ 1+α2 D (1),withα∈ [−1,1]. Keywords: Rényi divergence; ϕ-function; ϕ-divergence; ϕ-family; statistical manifold; informationgeometry 1. Introduction Informationgeometry, thestudyofstatisticalmodelsequippedwithadifferentiablestructure,was pioneeredbytheworkofRao[1], andgainedmaturitywith theworkofAmariandmanyothers [2–4]. Ithasbeensuccessfullyapplied inmanydifferentareas, suchasstatistical inference,machine learning, signalprocessingoroptimization[4,5]. Inappropriatestatisticalmodels, thedifferentiablestructure is inducedbya(statistical)divergence. TheKullback–Leiblerdivergence inducesaRiemannianmetric, called theFisher–Raometric, andapairofdualconnections, theexponentialandmixtureconnections. A statisticalmodel endowedwith the Fisher–Raometric is called a (classical) statisticalmanifold. Amarialsoconsideredafamilyofα-divergences that inducea familyofα-connections. Muchresearchinrecentyearshasfocusedonthegeometryofnon-standardstatisticalmodels[6–8]. Thesemodelsaredefinedin termsofadeformedexponential (alsocalledφ-exponential). Inparticular, κ-exponential models and q-exponential families are investigated in [9,10]. Non-parametric (or infinite-dimensional) ϕ-families were introduced by the authors in [11,12], which generalize exponential families inthenon-parametricsetting[13–16]. Basedonthesimilaritybetweenexponential and ϕ-families,wedefined theso-called ϕ-divergence,with respect towhich theKullback–Leibler divergence is a particular case. Statistical models equipped with a geometric structure induced by ϕ-divergences, which are called generalized statistical manifolds, are investigated in [17,18]. Withrespect to theseconnections,parametricϕ-familiesareduallyflat. The ϕ-divergence is intrinsically related to the (ρ,τ)-model of Zhang, which was proposed in[19,20], extendedto the infinite-dimensionsetting in [21],andexplainedinmoredetails in [22,23]. Entropy2016,18, 407 271 www.mdpi.com/journal/entropy