Page - 327 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 421 Theorem1. Fork>1andN≥6, thefirst threemomentsofWare: E(W)= k N , Var(W)= { π(−1)−(k+1)2 } +2k(N−1) N3 andE[{W−E(W)}3]givenby { π(−2)−(k+1)3 } −(3k+25−22N) { π(−1)−(k+1)2 } +g(k,N) N5 , whereg(k,N)=4(N−1)k(k+2N−5)>0. Inparticular, forfixedkandN,asπmin→0 Var(W)→∞andγ(W)→+∞, whereγ(W) :=E[{W−E(W)}3]/{Var(W)}3/2. Adetailedproof is foundinAppendixA,andwegivehereanoutlineof its important features. Themachinerydeveloped iscapableofdeliveringmuchmore thanaproofofTheorem1.As indicated there, it provides a generic way to explicitly compute arbitrary moments or mixed moments of multinomialcounts,andcould inprinciplebe implementedbycomputeralgebra.Overall, thereare fourstages. First, akeyrecurrencerelation isestablished; secondly, it is exploitedtodelivermoments of a single cell count. Third,mixedmoments of anyorder arederived from those of lower order, exploitingacertainfunctionaldependence. Finally, resultsarecombinedtofindthefirst threemoments ofW, highermomentsbeingsimilarlyobtainable. The practical implication of Theorem 1 is that standard first (and higher-order) asymptotic approximations to thesamplingdistributionof theWald,χ2, andscorestatisticsbreakdownwhen thedata generationprocess is “close to” the boundary,where at least one cell probability is zero. This result isqualitativelysimilar toresults in [10],whichshowshowasymptoticapproximations to thedistributionof themaximumlikelihoodestimate fail; forexample, in thecaseof logistic regression, whentheboundary isclose in termsofdistancesasdefinedbytheFisher information. Unlike statistics considered in Theorem 1, the deviance has a workable distribution in the same limit: that is, for fixed N and k as we approach the boundary of the probability simplex. Insharpcontrast to that theorem,wesee theverystableandworkablebehaviourof thek-asymptotic approximation to thedistributionof thedeviance, inwhich thenumberofcells increaseswithout limit. DefinethedevianceDvia D/2 = ∑{0≤i≤k:ni>0}ni log(ni/N)− k ∑ i=0 ni log(πi) = ∑{0≤i≤k:ni>0}ni log(ni/μi), whereμi :=E(ni)=Nπi. Wewill exploit the characterisation that themultinomial randomvector (ni)has thesamedistributionasavectorof independentPoissonrandomvariablesconditionedon their sum. Specifically, let theelementsof (n∗i )be independentlydistributedasPoissonPo(μi). Then, N∗ :=∑ki=0n∗i ∼Po(N),while (ni) :=(n∗i |N∗=N)∼ Multinomial(N,(πi)). Definethevector S∗ := ( N∗ D∗/2 ) = k ∑ i=0 ( n∗i n∗i log(n ∗ i/μi) ) , 327