Page - 156 - in Document Image Processing

J. Imaging 2018,4, 39 Thefieldofclassifiercombinationcanbegroupedintodifferentcategories [35]basedonthestage atwhich theprocess isapplied, typeof information(classifieroutput)beingfusedandthenumber andtypeofclassifiersbeingcombined. Basedontheoperating levelof theclassifiers, classifiercombinationcanbedoneat the feature level.Multiple featurescanbe joinedtoprovideanewfeaturesetwhichprovidemore information about theclasses. Butwith the increase indimensionalityof thedata, trainingbecomesexpensive. Theclassifieroutputsafter theextractionoftheindividual featuresetscanbecombinedtoprovide better insightsat thedecision level.Decision level combinationtechniquesarepopularas it cannot needanyunderstandingof the ideasbehindthe featuregenerationandclassificationalgorithms. Feature level combination is performed by concatenating the feature sets in all possible combinationsandpassing it throughthebaseclassifier,MLPin thiscase.Apart fromthat, all theother combinationprocessesworkedoutoperateat thedecision level. Classifier combination can also be classified by the outputs of the classifiers used in the combination. Three typesofclassifieroutputsareusuallyconsidered[36]: • Type I (Abstract level): This is the lowest level in a sense that the classifierprovides the least amountof informationonthis level. Classifieroutput isasingleclass label informingthedecision of theclassifier. • Type II (Rank level): Classifier output on the rank level is an ordered sequence of candidate classes, theso-calledn-best list. Thecandidateclassesareorderedfromthemost likelyclassat the front and the least likely class index featuring at the last of the list. There are no confidence scores attached to the class labels on rank level and the relative positioning provides the required information. • TypeIII (Measurement level): Inadditionto theorderedn-best listsofcandidateclassesonthe rank level, classifieroutputon themeasurement levelhas confidencevaluesassigned toeach entryof then-best list. Theseconfidences,orscores,aregenerallyrealnumbersgeneratedusing the internalalgorithmfor theclassifier. Thissoft-decision informationat themeasurement level thusprovidesmore informationthantheother levels. In thispaper,TypeII (ranklevel)andTypeIII (measurement level) combinationproceduresare workedoutbecause theyallowthe inculcationofagreaterdegreeofsoft-decision informationfrom theclassifiersandfinduse inmostpracticalapplications. The focus of this paper is to explore the classifier combination techniques on a fixed set of classifiers. Thepurposeof thecombinationalgorithmis to learn thebehaviourof theseclassifiersand produceanefficientcombinationfunctionbasedontheclassifieroutputs.Hence,weusenon-ensemble classifier combinationswhich try to combineheterogeneous classifiers complementing eachother. Theadvantageofcomplementaryclassifiers is thateachclassifiercanconcentrateon itsownsmall sub problemandtogether thesingle largerproblemisbetterunderstoodandsolved. Theheterogeneous classifiers,here, aregeneratedby training thesameclassifierwithdifferent featuresetsandtuning themtooptimalvaluesof theirparameters. Thisproceduredoesawaywiththeneedfornormalization of the confidence scores provided bydifferent classifierswhich donot tend to followa common standardanddependof thealgorithm. Forexample, in theMLPclassifierusedhere, the last layerhas eachnodecontainingafinalscore foroneclass. Thesescorescanthenbeusedfor therankleveland decisionlevelcombinationalongwiththemaximumbeingchosenfor the individualclassifierdecision. In the next sub-section, the set ofmajor classification algorithms evaluated in this paper are categorized into two approaches based on how the combination process is implemented. In the first approach, rule based combinationpractices are demonstrated that apply a given function to combine theclassifierconfidences intoasinglesetofoutputscores. Thesecondapproachemploys anotherclassifier, called the ‘secondary’ classifier thatoperateson theoutputsof thebaseclassifier andautomaticallyaccount for thestrengthsof theparticipants. Theclassificationalgorithmis trained on these confidence valueswith output classes same as the original pattern recognition problem. 156