Page - 330 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 421 3.DivergencesandGoodness-of-Fit Theemphasisof thissectionis the importanceof theboundaryof theextendedmultinomialwhen understandingthe linksbetweeninformationgeometricdivergencesandfamiliesofgoodness-of-fit statistics. For completeness, a set ofwell-knownresults linking thePower-Divergence familyand informationgeometry in themanifoldsenseare surveyed inSections3.1–3.3. Theextension to the extendedmultinomialfamilyisdiscussedinSection3.4,wherewemakeclearhowtheglobalbehaviour ofdivergences isdominatedbyboundaryeffects. Thiscomplements theusual localanalysis,which linksdivergenceswith theFisher information, [8]. Perhaps thekeypoint is that, sincecounts in the datacanbezero, informationgeometricstructuresshouldalsoallowprobabilities tobezero.Hence, closuresofexponential familiesseemtobethecorrectgeometricobject toworkon. 3.1. ThePower-DivergenceFamily The results of Section2 concern theboundarybehaviourof two importantmembersof a rich classofgoodness-of-fit statistics.Animportantunifyingframeworkwhichencompasses theseand other importantstatisticscanbefoundin[5] (page16)with theso-calledPower-Divergencestatistics. Thesearedefined, for−∞<λ<∞, by 2NIλ (n N :π ) := 2 λ(λ+1) k ∑ i=0 ni [( ni Nπi )λ −1 ] , (11) with thecasesλ=−1,0beingdefinedbytakingtheappropriate limit togive lim λ→−1 2NIλ (n N :π ) =2 k ∑ i=0 Nπi log(Nπi/ni) , lim λ→0 2NIλ (n N :π ) =2 k ∑ i=0 ni log(ni/Nπi) . Importantspecial casesareshowninTable1 (whosefirstcolumnisdescribedbelowinSection3.3), andwealsonote thecaseλ=2/3,whichReadandCressie recommend[5] (page79)asareasonably robust statisticwithaneasily calculable criticalvalue for smallN. Ina sense, it lies “between” the Pearsonχ2 anddeviancestatistics,whichwecomparedinSection2. Table1.Special casesof thePower-Divergencestatistics. α:=1+2λ λ Formula Name 3 1 ∑ki=0 (ni−Nπi)2 Nπi Pearsonχ 2 7/3 2/3 95∑ k i=0ni [( ni Nπi )2 3 −1 ] Read–Cressie 1 0 2∑ki=0ni log(ni/Nπi) Twice log-likelihood(deviance) 0 −12 4∑ki=0 (√ ni− √ Nπi )2 Freeman–TukeyorHellinger −1 −1 2∑ki=0Nπi log(Nπi/ni) Twicemodifiedlog-likelihood −3 −2 ∑ki=0 (ni−Nπi) 2 ni Neymanχ 2 Thispaper isprimarilyconcernedwith thesparsecasewheremanyof theni countsarezero,and wearealso interested in lettingprobabilities,πi, becomingarbitrarilysmall,orevenzero. 3.2. LiteratureReview Before we look at this, we briefly review the literature on the geometry of goodness-of-fit statistics.Agoodsource for thehistoricaldevelopments (in thediscretecontext) canbefoundin[5] (pages131–153)and [7]. Important examples include theanalysisof contingency tables, log-linear, 330