Page - 326 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 421 Section2givesformalproofsoftworesults,Theorems1and2,whichwereannouncedin[2]. These resultsexplore thesamplingperformanceofstandardgoodness-of-fitstatistics—Wald,Pearson’sχ2, scoreanddeviance—inthesparsesetting. Inparticular, theylookat thecasewherethedatageneration process is“close to theboundary”of theparameterspacewhereoneormorecellprobabilitiesvanish. This complements results inmuchof the literature,where the centre of theparameter space—i.e., theuniformdistribution—isoften the focusofattention. Section 3 starts with a review of the links between Information Geometry (IG) [3] and goodness-of-fit testing. Inparticular, it looksat thepower familyofCressieandRead[4,5] in termsof thegeometric theoryofdivergences. In thecaseof regularexponential families, these linkshavebeen well-exploredinthe literature[6],ashas thecorrespondingsamplingbehaviour[7].What isnovelhere istheexplorationofthegeometrywithrespecttotheclosureoftheexponentialfamily; i.e., theextended multinomialmodel—akeytool inCIG.Weillustratehowtheboundarycandominate thestatistical properties inways thataresurprisingcomparedtostandard—andevenhigh-order—analyses,which areasymptotic insamplesize. Through simulation experiments, Section 4 explores the consequences of working in the sparsemultinomial setting, with the design of the numerical experiments being inspired by the informationgeometry. 2. SamplingDistributions intheSparseCase Oneof thefirstmajor impacts that informationgeometryhadonstatisticalpracticewas through the geometric analysis of higher order asymptotic theory (e.g., [8,9]). Geometric interpretations and invariant expressions of terms in thehigher order corrections to approximations of sampling distributionsareagoodexample, [8] (Chapter4).Geometric termsareusedtocorrect forskewnessand otherhigherordermoment (cumulant) issues in thesamplingdistributions.However, thesecorrection termsgrowvery largenear theboundary[1,10]. Since this regionplaysakeyrole inmodelling in the sparsesetting—themaximumlikelihoodestimator (MLE) oftenbeingontheboundary—extensions to theclassical theoryareneeded. Thispaper, togetherwith [2], start suchadevelopment. Thiswork is related to similar ideas in categorical, (hierarchical) log–linear, and graphicalmodels [1,11–13]. Asstated in [13], “their statisticalpropertiesundersparsesettingsarestill verypoorlyunderstood. Asaresult, analysisof suchdataremainsexceptionallydifficult”. In this sectionweshowwhy theWald—equivalently, thePearsonχ2 andscore statistics—are unworkablewhennear theboundaryof theextendedmultinomialmodel,but that thedeviancehasa simple,accurate,andtractablesamplingdistribution—evenformoderatesamplesizes.Wealsoshow howthehighermomentsof thedevianceareeasilycomputable, inprincipleallowingforhigherorder adjustments.However,wealsomakesomeobservationsabout theappropriatenessof theseclassical adjustments inSection4. First,wedefine somenotation, consistentwith that of [2]. With i rangingover{0,1,...,k}, let n=(ni)∼Multinomial (N,(πi)),wherehereeachπi> 0. In this context, theWald,Pearson’sχ2, andscorestatisticsall coincide, their commonvalue,W, being W := k ∑ i=0 (πi−ni/N)2 πi ≡ 1 N2 k ∑ i=0 n2i πi −1. Definingπ(α) :=∑iπαi ,wenote the inequality, foreachm≥1, π(−m)−(k+1)m+1≥0, inwhichequalityholdsifandonlyifπi≡1/(k+1)—i.e., iff(πi) isuniform.Wethenhavethefollowing theorem,whichestablishesthat thestatisticW isunworkableasπmin :=min(πi)→0forfixedkandN. 326