<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="83"/><title>Hybrid Named Entity Recognition System for South and South East Asian Languages</title><pubinfo>Proceedings of the IJCNLP-08 Workshop on NER for South and South East Asian Languages,pages 83-88, Hyderabad, India, January 2008. ©2008 Asian Federation of Natural Language Processing</pubinfo><author surname="P" givenname="Praveen"><org  name="International Institute of Information Technology" country="India" city="Hyderabad"/></author><author surname="V" givenname="Ravi Kiran"><org  name="International Institute of Information Technology" country="India" city="Hyderabad"/></author></firstpageheader><frontmatter><p><b>A Hybrid Named Entity Recognition Sy</b></p><p><b>Asian Languages</b></p><p><b>Praveen Kumar P</b></p><p>Language Technologies Research Centre International Institute of Information Technology - Hyderabad praveen_p@students.iiit.ac.in</p><p><b>Ravi Kiran V</b></p><p>Language Technologies Research Centre International Institute of Information Technology - Hyderabad ravikiranv@students.iiit.ac.in</p></frontmatter><abstract>This paper is submitted for the contest NERSSEAL-2008. Building a statistical based Named entity Recognition (NER) system requires huge data set. A rule based system needs linguistic analysis to formu­late rules. Enriching the language specific rules can give better results than the statis­tical methods of named entity recognition. A Hybrid model proved to be better in identifying Named Entities (NE) in Indian Language where the task of identifying named entities is far more complicated compared to English because of variation in the lexical and grammatical features of Indian languages. </abstract></header><body><section number="1" title="Introduction"><p>Named Entities (NE) are phrases that contain per­son, organization, location, number, time, measure etc. Named Entity Recognition is the task of identi­fying and classifying the Named Entities into pre­define categories such as person, organization, lo­cation, etc in the text.</p><p>NER has several applications. Some of them are Machine Translation (MT), Question-Answering System, Information Retrieval (IR), and Cross-lingual Information Retrieval.</p><p>The tag set used in the NER-SSEA contest has12 categories. This is 4 more than the CONLL-2003 shared task on NER tag-set. The use of finer tag-set aims at improving Machine Translation (MT). Annotated data for Hindi, Bengali, Oriya, Telugu and Urdu languages was provided to the contestants.</p><p>Significant work in the field of NER was done in English, European languages but not in Indian languages. There are many rule-based, HMM based; Conditional Random Fields (CRF) based NER systems. MEMM were used to identify the NE in Hindi (Kumar and Bhattacharyya, 2006). Many techniques were used in CoNLL-2002 shared task on NER which aimed at developing a language independent NER system.</p></section><section number="2" title="Issues: Indian Languages"><p>The task of NER in Indian Languages is a difficult task when compared to English. Some features that make the task difficult are</p><subsection number="2.1" title="No Capitalization"><p>Capitalization is an important feature used by the English NER systems to identify the NE. The ab­sence of the lexical features such as capitalization in Indian languages scripts makes it difficult to identify the NE.</p></subsection><subsection number="2.2" title="Agglutinative nature"><p>Some of the Indian language such as Telugu is ag­glutinative in nature. Telugu allows polyagglutina­tion, the unique feature to being able to add multi­ple suffixes to words to denote more complex words.</p><p>Ex: "hyderabadlonunci" = hyderabad+ lo + nunchi</p></subsection><subsection number="2.3" title="Ambiguities"><p>There can be ambiguity among the names of per­sons, locations and organizations such as <i>Washing­ton </i>can be either a person name as well as location name.</p></subsection><subsection number="2.4" title="Proper-noun &amp; common noun Ambiguity"><p>In India the common-nouns often occur as the per­son names. For instance <i>Akash </i>which can mean <i>'sky' </i>is also name of a person.</p><page local="2" global="84"/></subsection><subsection number="2.5" title="Free-word order"><p>Some of the Indian languages such as Telugu are free word order languages. The heuristics such as position of the word in the sentence can not be used as a feature to identify NE in these languages.</p></subsection></section><section number="3" title="Approaches"><p>A NER system can be either a Rule based or statis­tical or hybrid. A Rule-based system needs linguis­tic analysis to formulate the rules. A statistical NER system needs annotated corpus. A hybrid sys­tem is generally a rule based system on top of sta­tistical system.</p><p>For the NER-SSEAL contest we developed CRF based and HMM based hybrid system.</p><subsection number="3.1" title="Hidden Markov Model"><p>We used a second order Markov model for Named entity tagging. The tags are represented by the states, words by the output. Transition probabilities depend on the states. Output probabilities depend on the most recent category. For a given sentence w1^wT of length T. t1,t2.. tT are elements of the tag-set. We calculate</p><doubt alpha="61.4" length="44" tooSmall="False" monospace="0.0">Argmax[IVP(ti|ti-i,ti-2)P(wi|ti)](P(tT+i|tT)</doubt><p>This gives the tags for the words. We use linear interpolation of unigrams, bigrams and trigrams for transition probability smoothing and suffix trees for emission probability smoothing.</p><subsubsection number="3.1.1" title="HMM based hybrid model"><p>In the first phase HMM models are trained on the training corpus and are used to tag the test data. The first layer is purely statistical method of solv­ing and the second layer is pure rule based method of solving. In order to extend the tool for any other Indian language we need to formulate rules in the second layer. In the first layers HMM models are training from the annotated training corpus. The annotation follows as: Every word in the corpus if belongs to any Named entity class is marked with the corresponding class name. And the one's which don't fall into any of the named entity class fall into the class of words that are not named entities. The models obtained by training the annotated training corpus are used to tag the test data. In the first layer the class boundaries may not be identi­fied correctly. This problem of correctly identifying the class boundaries and nesting is solved in the second layer.</p><p>In the second layer, the chunk information of the test corpus is used to identify the correct bounda­ries of the named entities identified from the first layer. It's a type of validation of result from the first layer. Simultaneously, few rules for every class of named entities are used in order to identify nesting of named entities in the chunks and to identify the unidentified named entities from the first layer output. For Telugu these rules include suffixes with which Named Entities can be identi­fied</p></subsubsection></subsection><subsection number="3.2" title="Conditional Random Fields"><p>Conditional Random Fields (CRFs) are undirected graphical models, a special case of which corre­sponds to conditionally-trained finite state ma­chines. CRFs are used for labeling sequential data.</p><p>In the special case in which the output nodes of the graphical model are linked by edges in a linear chain, CRFs make a first-order Markov independ­ence assumption, and thus can be understood as conditionally-trained finite state machines (FSMs).</p><p>Let <i>o = (o, o2, o3, o4,... oT ) </i>be some observed input data sequence, such as a sequence of words in text in a document,(the values on n input nodes of the graphical model). Let S be a set of FSM states, each of which is associated with a label, l ? £.Let <i>s = (s1,s2,s3 ,s4 sT) </i>be some sequence of states, (the values on T output nodes). By the Hammersley- Clifford theorem, CRFs define the conditional probability of a state sequence given an input sequence to be:</p><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">I7</doubt><doubt alpha="41.7" length="36" tooSmall="False" monospace="0.0">P{S O) =—*SXp(£ £h fL{if-1,st, o,i))</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">^ r=Lk</doubt><p>where <i>Zo </i>is a normalization factor over all state sequences is an arbitrary feature function over its arguments, and ?k is a learned weight for each fea­ture function. A feature function may, for example, be defined to have value 0 or 1. Higher ? weights make their corresponding FSM transitions more likely. CRFs define the conditional probability of a label sequence based on the total probability over the state sequences,</p><p><i>nj</i><b><i>\0) </i></b><i>= Lr</i><b><i>.Hs</i></b><i>)=A*\°) </i>where <b><i>l(s) </i></b>is the sequence of labels corresponding to the labels of the states in sequence <b>s.</b></p><page local="3" global="85"/><p>z0 = <i>V </i>expf     <b><i>hfkist-hst^i) ,</i></b></p><doubt alpha="18.8" length="16" tooSmall="False" monospace="0.0">s€sr      \{=1 /</doubt><p>Note that the normalization factor, Z0, (also known in statistical physics as the partition function) is the sum of the scores of all possible states.</p><p>And that the number of state sequences is expo­nential in the input sequence length T. In arbitrar­ily structured CRF's calculating the normalization factor in closed form is intractable, but in liner-chain-structure CRFs, the probability that a par­ticular transition was taken between two CRF states at a particular position in the input can be calculated by dynamic programming.</p><subsubsection number="3.2.1" title="CRF based model"><p>CRF models were used to perform the initial tag­ging. The features for the Hindi and Telugu models include the Root, number and gender of the word from the morphological analyzer. From our previ­ous experiments it is observed that the system per­forms better with the suffix and the prefix as fea­tures. So the first 4, first 3, first 2 and the 1st letter of the word (prefix) and the last 4, 3, 2, 1 letters of the word (suffix) are used as features.</p><p>The word is a Named Entity depends on the POS tag. So the POS tag is used as a feature. The chunk information is important to identify the Named entities with more than one word. So the chunk information is also included in the feature list.</p><p>The resources for the rest of the three languages (Oriya, Urdu and Bengali) are limited. Since we couldn't find the morphological analyzer for these languages, the first 4,3,2,1 letters and the last 4,3,2,1 letters are used as features.</p><p>The word being classified as a named entity also depends on the previous and next words. So these are used as features for all the languages</p></subsubsection></subsection></section><section number="4" title="Evaluation"><p>Precision, Recall and F-measure are used as metric to evaluate the system. These are calculated for Nested (both nested and largest possible NE match), Maximal (largest possible NE match) and Lexicon matches</p><p>Nested matches (n): The largest possible as well as the nested NE</p><p>Maximal matches (m): The largest possible NE matched with reference data.</p><p>Lexical item (l): The lexical item inside the NE are</p><doubt alpha="100.0" length="7" tooSmall="False" monospace="0.0">matched</doubt></section><section number="5" title="Results"><p>Pm, Pn ,Pl are the precision of maximal, nested, lex­ical matches respectively. Rm, Rn, Rl are the recall of maximal, nested, lexical matches respectively. Similarly Fm, Fn, Fl are the F-measure of maximal, nested, lexical matches.</p><p>The precision, recall, F-measure of five lan­guages for CRF system is given in Tablel. Table 2 has the lexical F-measure for each category. Simi­larly Table3 and Table4 give the precision, recall and F-measure for the five languages and the lexi­cal F-measure for each category of HMM based system.</p><p>The performance of the NER system for five languages using a CRF based system is shown in Table-1.</p><table caption="Table 1: Performance of NER system for five languages (CRF)" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>F-Measure</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Language</p></td><td class="cell"><p>Pm</p></td><td class="cell"><p>Pn</p></td><td class="cell"><p>Pl</p></td><td class="cell"><p>Rm</p></td><td class="cell"><p>Rn</p></td><td class="cell"><p>Rl</p></td><td class="cell"><p>Fm</p></td><td class="cell"><p>Fn</p></td><td class="cell"><p>Fl</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Bengali</p></td><td class="cell"><p>61.28</p></td><td class="cell"><p>61.45</p></td><td class="cell"><p>66.36</p></td><td class="cell"><p>21.18</p></td><td class="cell"><p>20.54</p></td><td class="cell"><p>24.43</p></td><td class="cell"><p>31.48</p></td><td class="cell"><p>30.79</p></td><td class="cell"><p><b><i>35.71</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>69.45</p></td><td class="cell"><p>72.53</p></td><td class="cell"><p>73.30</p></td><td class="cell"><p>30.38</p></td><td class="cell"><p>29.12</p></td><td class="cell"><p>27.97</p></td><td class="cell"><p><b><i>42.27</i></b></p></td><td class="cell"><p><b><i>41.56</i></b></p></td><td class="cell"><p><b><i>40.49</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Oriya</p></td><td class="cell"><p>37.27</p></td><td class="cell"><p>38.65</p></td><td class="cell"><p>64.20</p></td><td class="cell"><p>19.56</p></td><td class="cell"><p>16.19</p></td><td class="cell"><p>25.75</p></td><td class="cell"><p>25.66</p></td><td class="cell"><p>22.82</p></td><td class="cell"><p><b><i>36.76</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Telugu</p></td><td class="cell"><p>33.50</p></td><td class="cell"><p>36.18</p></td><td class="cell"><p>61.98</p></td><td class="cell"><p>15.90</p></td><td class="cell"><p>11.13</p></td><td class="cell"><p>36.10</p></td><td class="cell"><p>21.56</p></td><td class="cell"><p>17.02</p></td><td class="cell"><p><b><i>45.62</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Urdu</p></td><td class="cell"><p>45.55</p></td><td class="cell"><p>46.11</p></td><td class="cell"><p>52.35</p></td><td class="cell"><p>26.08</p></td><td class="cell"><p>24.24</p></td><td class="cell"><p>30.13</p></td><td class="cell"><p>33.17</p></td><td class="cell"><p>31.78</p></td><td class="cell"><p><b><i>38.25</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>m: Maximal n: Nested l: lexical</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="4" global="86"/><p><b><u>NP: Not present in reference data</u></b><page local="5" global="87"/></p><table caption="Table 2: Class specific F-Measure for nested lexical match (CRF)"></table><table caption="Table 4: Class specific F-measure for nested lexical match (HMM)"></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Bengali</p></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>Oriya</p></td><td class="cell"><p>Telugu</p></td><td class="cell"><p>Urdu</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEP</p></td><td class="cell"><p>33.06</p></td><td class="cell"><p><b><i>42.31</i></b></p></td><td class="cell"><p><b><i>51.50</i></b></p></td><td class="cell"><p>15.70</p></td><td class="cell"><p>11.72</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NED</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p><b><i>42.85</i></b></p></td><td class="cell"><p>01.32</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>04.76</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEO</p></td><td class="cell"><p>11.94</p></td><td class="cell"><p>34.83</p></td><td class="cell"><p>12.52</p></td><td class="cell"><p>02.94</p></td><td class="cell"><p>20.92</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEA</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>36.36</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEB</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETP</p></td><td class="cell"><p>29.62</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>18.03</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETO</p></td><td class="cell"><p>28.96</p></td><td class="cell"><p>08.13</p></td><td class="cell"><p>03.33</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEL</p></td><td class="cell"><p>34.41</p></td><td class="cell"><p><b><i>61.08</i></b></p></td><td class="cell"><p><b><i>46.73</i></b></p></td><td class="cell"><p>12.26</p></td><td class="cell"><p><b><i>54.59</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETI</p></td><td class="cell"><p><b><i>63.86</i></b></p></td><td class="cell"><p><b><i>70.37</i></b></p></td><td class="cell"><p>35.22</p></td><td class="cell"><p><b><i>90.49</i></b></p></td><td class="cell"><p><b><i>62.22</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEN</p></td><td class="cell"><p><b><i>75.34</i></b></p></td><td class="cell"><p><b><i>74.07</i></b></p></td><td class="cell"><p>21.03</p></td><td class="cell"><p>26.32</p></td><td class="cell"><p>13.44</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEM</p></td><td class="cell"><p><b><i>46.96</i></b></p></td><td class="cell"><p><b><i>58.33</i></b></p></td><td class="cell"><p>14.19</p></td><td class="cell"><p><b><i>42.01</i></b></p></td><td class="cell"><p><b><i>77.72</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETE</p></td><td class="cell"><p>12.54</p></td><td class="cell"><p>13.85</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>08.63</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 3: Performance of NER system for five languages (HMM)" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Measure</p></td><td class="cell"><p></p></td><td class="cell"><p>Precision</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>Recall</p></td><td class="cell"><p></p></td><td class="cell"><p>F-Measure</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Language</p></td><td class="cell"><p>Pm</p></td><td class="cell"><p>Pn</p></td><td class="cell"><p>Pl</p></td><td class="cell"><p>Rm</p></td><td class="cell"><p>Rn</p></td><td class="cell"><p>Rl</p></td><td class="cell"><p>Fm</p></td><td class="cell"><p>Fn</p></td><td class="cell"><p>Fl</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Bengali</p></td><td class="cell"><p>50.66</p></td><td class="cell"><p>50.78</p></td><td class="cell"><p>58.00</p></td><td class="cell"><p>25.03</p></td><td class="cell"><p>24.26</p></td><td class="cell"><p>30.26</p></td><td class="cell"><p>33.50</p></td><td class="cell"><p>32.83</p></td><td class="cell"><p><b><i>39.77</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>69.89</p></td><td class="cell"><p>73.37</p></td><td class="cell"><p>73.59</p></td><td class="cell"><p>36.90</p></td><td class="cell"><p>35.75</p></td><td class="cell"><p>34.34</p></td><td class="cell"><p><b><i>48.30</i></b></p></td><td class="cell"><p><b><i>47.16</i></b></p></td><td class="cell"><p><b><i>46.84</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Oriya</p></td><td class="cell"><p>33.10</p></td><td class="cell"><p>34.70</p></td><td class="cell"><p>60.98</p></td><td class="cell"><p>24.63</p></td><td class="cell"><p>20.61</p></td><td class="cell"><p>36.72</p></td><td class="cell"><p>28.24</p></td><td class="cell"><p>25.86</p></td><td class="cell"><p><b><i>45.84</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Telugu</p></td><td class="cell"><p>15.61</p></td><td class="cell"><p>49.67</p></td><td class="cell"><p>62.00</p></td><td class="cell"><p>11.64</p></td><td class="cell"><p>24.00</p></td><td class="cell"><p>37.30</p></td><td class="cell"><p>13.33</p></td><td class="cell"><p>32.37</p></td><td class="cell"><p><b><i>46.58</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Urdu</p></td><td class="cell"><p>42.81</p></td><td class="cell"><p>47.14</p></td><td class="cell"><p>56.21</p></td><td class="cell"><p>29.37</p></td><td class="cell"><p>29.69</p></td><td class="cell"><p>37.15</p></td><td class="cell"><p>34.48</p></td><td class="cell"><p>36.83</p></td><td class="cell"><p><b><i>44.73</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>m: Maximal n: Nested l: lexical</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Bengali</p></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>Oriya</p></td><td class="cell"><p>Telugu</p></td><td class="cell"><p>Urdu</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEP</p></td><td class="cell"><p><b><i>38.10</i></b></p></td><td class="cell"><p><b><i>53.19</i></b></p></td><td class="cell"><p><b><i>63.04</i></b></p></td><td class="cell"><p>23.14</p></td><td class="cell"><p>34.96</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NED</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p><b><i>52.94</i></b></p></td><td class="cell"><p>08.75</p></td><td class="cell"><p>06.18</p></td><td class="cell"><p><b><i>49.18</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEO</p></td><td class="cell"><p>05.05</p></td><td class="cell"><p><b><i>40.42</i></b></p></td><td class="cell"><p>28.52</p></td><td class="cell"><p>04.28</p></td><td class="cell"><p>31.53</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEA</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>25.00</p></td><td class="cell"><p>10.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>04.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEB</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETP</p></td><td class="cell"><p>36.25</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>19.92</p></td><td class="cell"><p>00.00</p></td><td class="cell"><p>09.09</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETO</p></td><td class="cell"><p>07.44</p></td><td class="cell"><p>16.39</p></td><td class="cell"><p>09.09</p></td><td class="cell"><p>05.85</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEL</p></td><td class="cell"><p><b><i>49.35</i></b></p></td><td class="cell"><p><b><i>72.03</i></b></p></td><td class="cell"><p><b><i>50.09</i></b></p></td><td class="cell"><p>29.26</p></td><td class="cell"><p><b><i>58.59</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETI</p></td><td class="cell"><p><b><i>50.81</i></b></p></td><td class="cell"><p><b><i>62.56</i></b></p></td><td class="cell"><p><b><i>46.30</i></b></p></td><td class="cell"><p><b><i>70.75</i></b></p></td><td class="cell"><p><b><i>53.98</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEN</p></td><td class="cell"><p><b><i>66.66</i></b></p></td><td class="cell"><p><b><i>81.96</i></b></p></td><td class="cell"><p>30.43</p></td><td class="cell"><p><b><i>86.29</i></b></p></td><td class="cell"><p>23.63</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEM</p></td><td class="cell"><p><b><i>62.98</i></b></p></td><td class="cell"><p><b><i>54.44</i></b></p></td><td class="cell"><p>20.68</p></td><td class="cell"><p>35.44</p></td><td class="cell"><p><b><i>82.64</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NETE</p></td><td class="cell"><p>12.56</p></td><td class="cell"><p>17.43</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>11.67</p></td><td class="cell"><p>00.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Table-2 shows the performance for specific classes of named entities. Table-3 presents the results for the HMM based system and Table-4 gives the class specific performance of the HMM based system.</p></section><section number="6" title="Error Analysis"><p>In both HMM, CRF based system the pos-tag and the chunk information are being used. NEs are generally the noun chunks. The pos-tagger and the chunker that we used had low accuracy. These errors in the POS-Tag contributed signifi­cantly to errors in NER.</p><p>In Telugu the F-measure for the maximal named entities is low for both the CRF, HMM models. This is because the test data had a large number of TIME named entities which are 5-6 words long. These entities further had nested named entities. Both the models are able to iden­tify the nested named entities. We chose not to consider the Time entities as a maximal entity since it was not tagged as a maximal NE as in some places. Considering it as a maximal NE the F-measure of the system increased signifi­cantly to over 30 for both HMM and CRF based systems.</p><p>It is also observed that many NE's were re­trieved correctly but were wrongly classified. Working with fewer tag-set will help to increase the performance of the system but this is not suggested.</p></section><section number="7" title="Conclusion"><p>The overall performance of the HMM model based hybrid system is better than the CRF model for all the languages. The performance of HMM based system is less that that of CRF. We obtained a decent Lexical F-measure of 39.77, 46.84, 45.84, 46.58, 44.73for Bengali, Hindi, Oriya, Telugu and Urdu using rules over HMM model. HMM based model has a better F-measure for NEP, NEL, NEO classes when com­pared to CRF model</p></section><references><p>CRF++: http://crfpp.sourceforge.net</p><p>P. Avinesh, G. Karthik. 2007. <i>Parts-of-Speech Tag­ging and Chunking using Conditional Random</i> <i>Fields and Transformation Based learning.</i><i> </i>Pro­ceedings of SPSAL workshop IJCNLP 07</p><p>Thorsen Brants. 2000. <i>TnT: a statistical Part-of-Speech Tagger. </i>Proceeding of sixth conference on Applied Natural Language Processing.</p><p>N. Kumar and Pushpak Bhattacharyya. 2006. <i>NER in</i> <i>Hindi using MEMM.</i><i></i></p><p>J. Lafferty, A. McCullam, F. Pereira. 2001. <i>Condi­tional Random Fields: Probabilistic models for segmenting and labeling sequence data. </i>18th Inter­national Conference on Machine Learning</p><p>Wei Li and A. McCallum. 2003. <i>Rapid Development of Hindi Named Entity Recognition using Condi­tional Random Fields and Feature Induction. </i>Transactions on Asian Language Information Processing.</p><page local="6" global="88"/></references></body></article>