Seite - 258 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 277 Westart theconvergencepropertiesbyprovingthat theobjective function Dˆϕ(pφ|pφT)decreases alongside the the sequence (φk)k, and give a possible set of conditions for the existence of the sequence (φk)k. Proposition1. (a)Assumethat the sequence (φk)k iswell defined inΦ, then Dˆϕ(pφk+1|pφT)≤ Dˆϕ(pφk|pφT), and (b)∀k,φk ∈Φ0. (c) AssumeA0 andA2 are verified, then the sequence (φk)k is defined and bounded. Moreover, the sequence (Dˆϕ(pφk|pφT))k converges. Proof. Weprove (a).Wehavebydefinitionof thearginf: Dˆϕ(pφk+1,pφT)+Dψ(φ k+1,φk)≤ Dˆϕ(pφk,pφT)+Dψ(φk,φk). Weuse the fact thatDψ(φk,φk)=0for theright-handsideandthatDψ(φk+1,φk)≥0for the left-hand sideof theprevious inequality.Hence, Dˆϕ(pφk+1,pφT)≤ Dˆϕ(pφk,pφT). Weprove (b)using thedecreasingpropertypreviouslyproved in (a). Wehaveby recurrence ∀k,Dˆϕ(pφk+1,pφT)≤ Dˆϕ(pφk,pφT)≤···≤ Dˆϕ(pφ0,pφT). Theresult followsdirectlybydefinitionofΦ0. Weprove(c)byinductiononk. Fork=0,clearlyφ0 iswelldefinedsincewechoose it. Thechoice of the initialpointφ0 of thesequencemayinfluence theconvergenceof thesequence. See theExample of theGaussianmixture inSection4.2. Suppose, for some k≥ 0, thatφk exists. Weprove that the infimumisattained inΦ0. Letφ∈Φbeanyvectoratwhich thevalueof theoptimizedfunctionhasa value less than itsvalueatφk, i.e., Dˆϕ(pφ,pφT)+Dψ(φ,φ k)≤ Dˆϕ(pφk,pφT)+Dψ(φk,φk).Wehave: Dˆϕ(pφ,pφT) ≤ Dˆϕ(pφ,pφT)+Dψ(φ,φk) ≤ Dˆϕ(pφk,pφT)+Dψ(φk,φk) ≤ Dˆϕ(pφk,pφT) ≤ Dˆϕ(pφ0,pφT). Thefirst line followsfromthenonnegativityofDψ. As Dˆϕ(pφ,pφT)≤ Dˆϕ(pφ0,pφT), thenφ∈Φ0. Thus, the infimumcanbe calculated for vectors inΦ0 instead ofΦ. SinceΦ0 is compact and the optimizedfunction is lowersemicontinuous(thesumof twolowersemicontinuousfunctions), then the infimumexistsandisattained inΦ0.Wemaynowdefineφk+1 tobeavectorwhosecorresponding value isequal to the infimum. Convergenceof the sequence (Dˆϕ(pφk,pφT))k comes fromthe fact that it isnon increasingand bounded. It isnon increasingbyvirtueof (a). Boundedness comes fromthe lower semicontinuity ofφ → Dˆϕ(pφ,pφT). Indeed,∀k,Dˆϕ(pφk,pφT)≥ infφ∈Φ0 Dˆϕ(pφ,pφT). The infimumofaproper lower semicontinuous function on a compact set exists and is attained on this set. Hence, the quantity infφ∈Φ0 Dˆϕ(pφ,pφT)existsandisfinite. Thisends theproof. Compactness inpart (c) canbe replacedby inf-compactness of functionφ → Dˆϕ(pφ|pφT)and continuityofDψ withrespect to itsfirstargument. Theconvergenceof thesequence (Dˆϕ(φk|φT))k is an interestingproperty, since, ingeneral, there isno theoreticalguarantee,or it isdifficult toprove that thewholesequence (φk)k converges. Itmayalsocontinuetofluctuatearoundaminimum.The decreaseof the error criterion Dˆϕ(φk|φT)between two iterationshelpsusdecidewhen to stop the iterativeprocedure. Proposition2. SupposeA1verified,Φ0 is closedand{φk+1−φk}→0. (a) IfACisverified, thenany limitpointof (φk)k is a stationarypointofφ → Dˆϕ(pφ|pφT); (b) IfACisdropped, thenany limitpoint of (φk)k is a“generalized”stationarypoint ofφ → Dˆϕ(pφ|pφT), i.e., zerobelongs to the subgradientofφ → Dˆϕ(pφ|pφT) calculatedat the limitpoint. 258