Page - 448 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 110 notconsiderhere,butwhichwillbediscussed inmoredetail inupcomingwork. Forexample,one canhaveasituationwith twolanguages inwhichaparameter isentailedbythevaluesof twoother parameters,butentailed to twodifferentvalues in the twolanguages. In thiscase, theproposalabove need tobemodified, because this entailedparameter should contribute to theHammingdistance betweenthe twolanguages. Insuchasituationtheentailedparametershould increase, rather than spoil, theefficiencyof thecode.Keepingentailedparameterscanbeusedforerror-correctingpurposes, ascontributingtoerrordetection. Theroleofentailmentofparameterswasconsideredin[8], in the useofspinglassmodels for languagechange,where theentailmentrelationsappearascouplingsat thevertices (interaction terms)betweendifferent Ising/Pottsmodelsonthesameunderlyinggraphof language interactions. Inupcomingwork,nowinpreparation,wewilldiscusshowtreatingdifferent formsofentailmentofparameters in thecodingtheorysettingdescribedhererelatedto the treatment ofentailmentrelations in thespinglassmodelof [8]. 3. EntropyandComplexityforLanguageFamilies 3.1.WhytheAsymptoticBound? In theexamplesdiscussedabovewecomparedthepositionof thecodepointassociatedtoagiven setof languages tocertaincurves in thespaceofcodeparameters. Inparticular,we focusedonthe asymptoticboundcurveandtheGilbert–Varshamovcurve. It shouldbepointedout that these two curveshaveaverydifferentnature. Theasymptoticbound is theonlycurve that separates regions in thespaceofparameters that correspondtocodepointswithentirelydifferentbehavior.Asshownin[13,24], codepoints in thearea belowtheasymptoticboundarerealizedwith infinitemultiplicityandfilldensely theregion,while codepoints that lieabovetheasymptoticboundare isolatedandrealizedwithfinitemultiplicity. TheGilbert–Varshamovcurve, by contrast, is related to the statistical behavior of sufficiently randomcodes (aswerecall inSection3.2below),butdoesnotseparate tworegionswithsignificantly differentbehavior in thespaceofcodepoints. Thus, in this respect, theasymptoticboundisamore natural curve toconsider thantheGilbert–Varshamovcurve. Thus,aheuristic interpretationof thepositionofcodesobtainedfromgroupsof languages,with respect to theasymptoticboundcanbeunderstoodas follows. Thepositionofacodepointaboveor belowtheasymptoticboundreflectsaverydifferentbehaviorof thecorrespondingcodewithrespect tohoweasily“deformable” it is. Thesporadic codes that lieabove theasymptoticboundare rigid objects, in contrast to thedeformable objects below the asymptotic bound. In termsofproperties of the distribution of syntactic parameterswithin a set of languages, this different nature of the associatedcodecanbeseenasameasureof thedegreeof“deformability”of theparameterdistribution: in languages that belong to the samehistorical linguistic families, the parameter distributionhas evolvedhistoricallyalongwith thedevelopmentof the family’sphylogenetic tree,andoneexpects that correspondingly the codeparameterswill indicate a higher degree of “deformability” of the correspondingcode. Ifagroupof languages ischosenthatbelongtoverydifferenthistorical families, on thecontrary,oneexpects that thedistributionofsyntacticparameterswillnotnecessarily leadany longer toacodethathas thesamekindofdeformabilityproperty: codepointsabovetheasymptotic boundmayberealizablebythis typeof languagegroups. There is no similar interpretation for the position of the code point with respect to the Gilbert–Varshamovline.Aninterpretationof thatpositioncanbesought in termsofShannonentropy, aswediscussbelow. Summarizing: themainconceptualdistinctionbetweentheGilbert–Varshamov line and the asymptotic bound is that the GV line represents only a statistical phenomenon, as wereviewbelow,while theasymptoticboundrepresents a true separationbetween twoclassesof structurallydifferentcodes, in thesenseexplainedabove. 448