Page - 262 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 277 Thisentailsusing(15) that∇Dˆ(pφ˜,pφT)=0. Comparing theprovedresultwith thenotationconsideredat thebeginningof theproof,wehave proved that the limit of the subsequence (φN1◦N0(k))k is a stationarypoint of theobjective function. Therefore, thefinal step is todeduce thesameresultontheoriginal convergentsubsequence (φN0(k))k. This is simplydueto the fact that (φN1◦N0(k))k isasubsequenceof theconvergentsequence (φN0(k))k, hence theyhavethesamelimit. WhenassumptionACisdropped, similararguments to thoseused in theproofofProposition2b. areemployed. Theoptimalitycondition in (11) implies : −∇Dψ(φk+1,φk)∈∂Dˆϕ(pφk+1,pφT) ∀k. Functionφ → Dˆϕ(pφ,pφT) is continuous,hence its subgradient isoutersemicontinuousand: limsup φk+1→φ∞ ∂Dˆϕ(pφk+1,pφT)⊂∂Dˆϕ(pφ˜,pφT). (16) Bydefinitionof the limsup: limsup φ→φ∞ ∂Dˆϕ(pφ,pφT)= { u|∃φk→φ∞,∃uk→uwithuk∈∂Dˆϕ(pφk,pφT) } . Inour scenario,φ= φk+1,φk= φk+1,u= 0anduk=∇1Dψ(φk+1,φk). Wehaveprovedabove in this proof that∇1Dψ(φ˜,φ¯) = 0 using only the convergence of (Dˆϕ(pφk,pφT))k, inequality (13) and the properties of Dψ. Assumption AC was not needed. Hence, uk → 0. This proves that u=0∈ limsupφk+1→φ∞∂Dˆϕ(pφnk+1,pφT). Finally,usingthe inclusion(16),wegetourresult: 0∈∂Dˆϕ(pφ˜,pφT), whichends theproof. Theproofof thepreviousproposition isverysimilar to theproofofProposition2. Thekey idea is touse thesequenceofconditionaldensitieshi(x|φk) insteadof thesequenceφk. According to the application,onemaybe interestedonly inProposition1or inPropositions2–4. Ifone is interested in theparameters,Propositions2 to4shouldbeused, sinceweneedastable limitof (φk)k. Ifweare only interested inminimizinganerrorcriterion Dˆϕ(pφ,pφT)betweentheestimateddistributionand the trueone,Proposition1shouldbesufficient. 4.CaseStudies 4.1.AnAlgorithmWithTheoreticallyGlobal InfimumAttainment Wepresent avariant of algorithm(11)whichensures theoretically the convergence to aglobal infimumof theobjective function Dˆϕ(pφ,pφT) as soonas there exists a convergent subsequenceof (φk)k. The idea is thesameasTheorem3.2.4 in [18].Defineφk+1 by: φk+1=arginf φ Dˆϕ(pφ,pφT)+βkDψ(φ,φ k). Theproofofconvergence isverysimpleanddoesnotdependonthedifferentiabilityofanyof the twofunctions Dˆϕ orDψ.WeonlyassumeA1andA2tobeverified. Let (φN(k))kbeaconvergent subsequence. Let φ∞ be its limit. This is guaranteed by the compactness ofΦ0 and the fact that thewhole sequence (φk)k resides inΦ0 (seeProposition1b). Suppose also that the sequence (βk)k converges to0askgoes to infinity. 262