Page - 442 - in Differential Geometrical Theory of Statistics

Entropy2016,18, 110 In thismodel,weonlyconsiderbinaryparameterswithvalues±1 (here identifiedwith letters0 or1 inF2)andweignoreparameters inaneutralizedstate following implicationsacrossparameters, as in thedatasetsof [3,4]. Theentailmentofparameters, that is, thephenomenonbywhichaparticular valueofoneparameter (butnot thecomplementaryvalue) rendersanotherparameter irrelevant,was addressed ingreaterdetail in [22].Wefirstdiscussaversionofourcodingtheorymodel thatdoesnot incorporateentailment.Wewill thencomment inSection2.7belowonhowthemodelcanbemodified to incorporate thisphenomenon. The idea thatnatural languagescanbedescribed,at the levelof their coregrammatical structures, in termsofastringofbinarycharacters (codewords)wasalreadyusedextensively in [23]. 2.1. CodeParameters In the theory of error-correcting codes, one assigns two main parameters to a code C, the transmission rate and the relative minimum distance. More precisely, a binary code C ⊂ Fn2 is an [n,k,d]2-code if thenumberofcodewords is#C=2k, that is, k= log2#C, (1) wherekneednotbean integer,andtheminimalHammingdistancebetweencodewords is d= min L1 =L2∈C dH(L1,L2), (2) where theHammingdistance isgivenby dH(L1,L2)= n ∑ i=1 |xi−yi|, forL1=(xi)ni=1 andL2=(yi) n i=1 inC. The transmissionrateof thecodeC isgivenby R= k n . (3) OnedenotesbyδH(L1,L2) therelativeHammingdistance δH(L1,L2)= 1 n n ∑ i=1 |xi−yi|, andonedefines therelativeminimumdistanceof thecodeCas δ= d n = min L1 =L2∈C δH(L1,L2). (4) Incodingtheory,onewouldlike toconstructcodes thatsimultaneouslyoptimizebothparameters (δ,R): a largervalueofR representsa faster transmissionrate (betterencoding),anda largervalueof δ represents the fact thatcodewordsaresufficientlysparse in theambientspaceFn2 (betterdecoding, withbettererror-correctingcapability). Constraintsonthisoptimizationproblemareexpressed in the formofbounds in thespaceof (δ,R)parameters, see [12,13]. Inoursetting, theRparametermeasures theratiobetweenthe logarithmicsizeof thenumberof languagesencompassingthegivenfamilyandthetotalnumberofparameters,orequivalentlyhow densely thegiven languagefamily is in theambientconfigurationspaceFn2 ofparameterpossibilities. The parameter δ is theminimum, over all pairs of languages in the given family, of the relative Hammingdistanceusedin thePCMmethodof [3,4]. 442