Page - 260 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 277 seemssimpler,but it isnotverifiedinmanymixturemodels (seeSection4.2 foracounterExample). Proposition 3. Assume thatA1,A2 andA3 are verified, then {φk+1−φk}→ 0. Thus, byProposition 2 (according towhetherACisverifiedornot), any limitpointof the sequenceφk is a (generalized) stationarypoint of Dˆϕ(.|φT). Proof. By contradiction, let us suppose that φk+1−φk does not converge to 0. There exists a subsequencesuchthat‖φN0(k)+1−φN0(k)‖> ε, ∀k≥ k0. Since(φk)kbelongstothecompactsetΦ0, there exists a convergent subsequence (φN1◦N0(k))k such thatφN1◦N0(k)→ φ¯. The sequence (φN1◦N0(k)+1)k belongs to thecompactsetΦ0; therefore,wecanextracta furthersubsequence (φN2◦N1◦N0(k)+1)k such thatφN2◦N1◦N0(k)+1→ φ˜. Besides φˆ = φ˜. Finallysince thesequence (φN1◦N0(k))k is convergent, a further subsequencealsoconvergestothesamelimit φ¯.Wehaveprovedtheexistenceofasubsequenceof(φk)k suchthatφN(k)+1−φN(k)doesnotconverge to0andsuchthatφN(k)+1→ φ˜,φN(k)→ φ¯with φ¯ = φ˜. The real sequence (Dˆϕ(pφk,pφT))k converges as proved in Proposition 1c. As a result, both sequences Dˆϕ(pφN(k)+1,pφT)and Dˆϕ(pφN(k),pφT)converge to thesamelimitbeingsubsequencesof the sameconvergentsequence. In theproofofProposition1,wecandeduce the following inequality: Dˆ(pφk+1,pφT)+Dψ(φ k+1,φk)≤ Dˆ(pφk,pφT), (13) whichisalsoverifiedforanysubstitutionofkbyN(k). Bypassingtothelimitonk,wegetDψ(φ˜,φ¯)≤0. However, thedistance-like functionDψ isnonnegative, so that itbecomeszero.UsingassumptionA3, Dψ(φ˜,φ¯)=0 implies that φ˜= φ¯. Thiscontradicts thehypothesis thatφk+1−φkdoesnotconverge to0. Thesecondpartof theProposition isadirect resultofProposition2. Corollary 1. Under assumptions of Proposition 3, the set of accumulation points of (φk)k is a connected compact set.Moreover, ifφ → Dˆ(pφ,pφT) is strictly convex in theneighborhoodof a limitpoint of the sequence (φk)k, then thewhole sequence (φk)k converges toa localminimumof Dˆ(pφ,pφT). Proof. Since the sequence (φ)k is bounded andverifies φk+1−φk → 0, thenTheorem28.1 in [17] implies that thesetofaccumulationpointsof (φk)k isaconnectedcompactset. It isnotemptysinceΦ0 iscompact. Theremainingof theproof isadirect resultofTheorem3.3.1 from[18]. Thestrictconcavity of theobjective functionaroundanaccumulationpoint is replacedherebythestrict convexityof the estimateddivergence. Proposition3andCorollary1describewhatwemayhopetogetof thesequenceφk. Convergence of thewholesequence isboundbya local convexityassumption in theneighborhoodofa limitpoint. Althoughsimple, thisassumptionremainsdifficult tobecheckedsincewedonotknowwheremight bethe limitpoints. Inaddition,assumptionA3isveryrestrictive,andisnotverifiedinmixturemodels. Propositions2and3weredevelopedfor the likelihoodfunction in thepaperofTseng[2]. Similar results forageneral classof functionsreplacing Dˆϕ andDψ whichmaynotbedifferentiable (butstill continuous)arepresented in [3]. In these results, assumptionA3 isessential. Although in [18] this problemisavoided, theirapproachdemands that the log-likelihoodhas−∞ limitas‖φ‖→∞. This is simplynotverifiedformixturemodels.Wepresentasimilarmethodtotheone in[18]basedonthe ideaofTseng[2]ofusingthesetΦ0which isvalid formixtures.Welose,however, theguaranteeof consecutivedecreaseof thesequence (φk)k. Proposition4. AssumeA1,ACandA2verified.Any limitpointof the sequence (φk)k is a stationarypoint ofφ→ Dˆ(pφ,pφT). IfACisdropped, then0belongs to the subgradient ofφ → Dˆ(pφ,pφT) calculatedat the limitpoint. Proof. If (φk)k converges to, say,φ∞, thentheresult falls simplyfromProposition2. 260