<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="41"/><title>Named Entity Recognition for Telugu</title><pubinfo>Proceedings of the IJCNLP-08 Workshop on NER for South and South East Asian Languages,pages 41-50, Hyderabad, India, January 2008. ©2008 Asian Federation of Natural Language Processing</pubinfo><author surname="Srikanth" givenname="P"><org  name="University of Hyderabad" country="India" city="Hyderabad"/></author><author surname="Murthy" givenname="Kavi Narayana"><org  name="University of Hyderabad" country="India" city="Hyderabad"/></author></firstpageheader><frontmatter><p><b>Named Entity Recognition for Telugu</b></p><p><b>P Srikanth and Kavi Narayana Murthy</b></p><p>Department of Computer and Information Sciences, University of Hyderabad,</p><p>Hyderabad, 500 046,</p><p>email: patilsrik@yahoo.co.in, knmuh@yahoo.com</p></frontmatter><abstract>This paper is about Named Entity Recogni­tion (NER) for Telugu. Not much work has been done in NER for Indian languages in general and Telugu in particular. Adequate annotated corpora are not yet available in Telugu. We recognize that named entities are usually nouns. In this paper we there­fore start with our experiments in building a CRF (Conditional Random Fields) based Noun Tagger. Trained on a manually tagged data of 13,425 words and tested on a test data set of 6,223 words, this Noun Tagger has given an F-Measure of about 92%. We then develop a rule based NER system for Telugu. Our focus is mainly on identify­ing person, place and organization names. A manually checked Named Entity tagged corpus of 72,157 words has been developed using this rule based tagger through boot­strapping. We have then developed a CRF based NER system for Telugu and tested it on several data sets from the Eenaadu and Andhra Prabha newspaper corpora de­veloped by us here. Good performance has been obtained using the majority tag con­cept. We have obtained overall F-measures between 80% and 97% in various experi­ments. <b>Keywords: </b>Noun Tagger, NER for Telugu, CRF, Majority Tag. </abstract></header><body><section number="1" title="Introduction"><p>organizations, monetary expressions, dates, numer­ical expressions etc. In the taxonomy of Compu­tational Linguistics, NER falls within the category of Information Extraction which deals with the ex­traction of specific information from given docu­ments. NER emerged as one of the sub-tasks of the DARPA-sponsored Message Understanding Confer­ence (MUCs). The task has important significance in the Internet search engines and is an important task in many of the Language Engineering applications such as Machine Translation, Question-Answering systems, Indexing for Information Retrieval and Au­tomatic Summarization.</p></section><section number="2" title="Approaches to NER"><p>There has been a considerable amount of work on NER in English (Isozaki and Kazawa, 2002; Zhang and Johnson, 2003; Petasis et al., 2001; Mikheev et al., 1999). Much of the previous work on name finding is based on one of the following approaches: (1) hand-crafted or automatically acquired rules or finite state patterns (2) look up from large name lists or other specialized resources (3) data driven ap­proaches exploiting the statistical properties of the language (statistical models).</p><p>The earliest work in named-entity recognition in­volved hand-crafted rules based on pattern matching (Appelt et al., 1993). For instance, a sequence of capitalized words ending in "Inc." is typically the name of an organization in the US, so one could im­plement a rule to that effect. Another example of such a rule is: Title Capitalized_word —&gt; Title Per-</p><p>son_name. Developing and maintaining rules and NER involves the identification of named entities dictionaries is a costly affair and adaptation to dif-such as person names, location names, names of   ferent domains is difficult.</p><page local="2" global="42"/><p>In the second approach, the NER system recog­nizes only the named entities stored in its lists, also called gazetteers. This approach is simple, fast, lan­guage independent and easy to re-target - just re­create the lists. However, named entities are too numerous and are constantly evolving. Even when named entities are listed in the dictionaries, it is not always easy to decide their senses. There can be semantic ambiguities. For example, "Washington" refers to both person name as well as place name.</p><p>Statistical models have proved to be quite ef­fective. Such models typically treat named-entity recognition as a sequence tagging problem, where each word is tagged with its entity type if it is part of an entity. Machine learning techniques are rela­tively independent of language and domain and no expert knowledge is needed. There has been a lot of work on NER for English employing the machine learning techniques, using both supervised learning and unsupervised learning. Unsupervised learning approaches do not require labelled training data -training requires only very few seed lists and large unannotated corpora (Collins and Singer, 1999). Su­pervised approaches can achieve good performance when large amounts of high quality training data is available. Statistical methods such as HMM (Bikel et al., 1997; Zhou and Su, 2001), Decision tree model (Baluja et al., 2000; Isozaki, 2001), and con­ditional random fields (McCallum, 2003) have been used. Generative models such as Hidden Markov Models (Bikel et al., 1997; Zhou and Su, 2001) have shown excellent performance on the Message Un­derstanding Conference (MUC) data-set (Chinchor, 1997). However, developing large scale, high qual­ity training data is itself a costly affair.</p></section><section number="3" title="NER for Indian languages"><p>NLP research around the world has taken giant leaps in the last decade with the advent of effective ma­chine learning algorithms and the creation of large annotated corpora for various languages. However, annotated corpora and other lexical resources have started appearing only very recently in India. Not much work has been done in NER in Indian lan­guages in general and Telugu in particular. Here we include a brief survey.</p><p>In (Eqbal, 2006), a supervised learning system based on pattern directed shallow parsing has been used to identify the named entities in a Bengali cor­pus. Here the training corpus is initially tagged against different seed data sets and a lexical con­textual pattern is generated for each tag. The entire training corpus is shallow parsed to identify the oc­currence of these initial seed patterns. In a position where the seed pattern matches wholly or in part, the system predicts the boundary of a named entity and further patterns are generated through bootstrap­ping. Patterns that occur in the entire training corpus above a certain threshold frequency are considered as the final set of patterns learned from the training corpus.</p><p>In (Li and McCallum, 2003), the authors have used conditional random fields with feature induc­tion to the Hindi NER task. The authors have iden­tified those feature conjunctions that will signifi­cantly improve the performance. Features consid­ered here include word features, character n-grams (n = 2,3,4), word prefix and suffix (length - 2,3,4) and 24 gazetteers.</p></section><section number="4" title="NER for Telugu"><p>Telugu, a language of the Dravidian family, is spo­ken mainly in southern part of India and ranks sec­ond among Indian languages in terms of number of speakers. Telugu is a highly inflectional and agglu­tinating language providing one of the richest and most challenging sets of linguistic and statistical fea­tures resulting in long and complex word forms (Ku­mar et al., June 2007). Each word in Telugu is in­flected for a very large number of word forms. Tel­ugu is primarily a suffixing Language - an inflected word starts with a root and may have several suffixes added to the right. Suffixation is not a simple con­catenation and morphology of the language is very complex. Telugu is also a free word order Language.</p><p>Telugu, like other Indian languages, is a resource poor language - annotated corpora, name dictionar­ies, good morphological analyzers, POS taggers etc. are not yet available in the required measure. Al­though Indian languages have a very old and rich literary history, technological developments are of recent origin. Web sources for name lists are avail­able in English, but such lists are not available in Telugu forcing the use of transliteration.</p><page local="3" global="43"/><p>In English and many other languages, named en­tities are signalled by capitalization. Indian scripts do not show upper-case - lower-case distinction. The concept of capitalization does not exist. Many names are also common nouns. Indian names are also more diverse i.e there are lot of variations for a given named entity. For example "telugude:s'aM" is written as Ti.Di.pi, TiDipi, te.de.pa:, de:s'aM etc. Developing NER systems is thus both challenging and rewarding. In the next section we describe our work on NER for Telugu.</p></section><section number="5" title="Experiments and Results"><subsection number="5.1" title="Corpus"><p>In this work we have used part of the LERC-UoH Telugu corpus, developed at the Language Engineer­ing Research Centre at the Department ofComputer and Information Sciences, University ofHyderabad. LERC-UoH corpus includes a wide variety ofbooks and articles, and adds up to nearly 40 Million words. Here we have used only a part ofthis corpus includ­ing news articles from two of the popular newspa­pers in the region. The Andhra Prabha (AP) cor­pus consists of 1.3 Million words, out of which there are approximately 200,000 unique word forms. The Eenaadu (EE) corpus consists of 26 Million words in all.</p></subsection><subsection number="5.2" title="Evaluation Metrics"><p>We use two standard measures, <i>Precision, Recall. </i>Here precision (P) measures the number of correct NEs in the answer file (Machine tagged data ) over the total number of NEs in the answer file and re­call (R) measures the number of correct NEs in the answer file over the total number of NEs in the key file (gold standard). F-measure (F) is the harmonic mean of precision and recall: <i>F = </i><b>^^p^ </b>when <i>/3<footnote anchor="2"/> </i>= 1. The current NER system does not handle multi-word expressions - only individual words are recognized. Partial matches are also considered as correct in our analyses here. Nested entities are not yet handled.</p></subsection><subsection number="5.3" title="Noun Identification"><p>Named entities are generally nouns and it is there­fore useful to build a noun identifier. Nouns can be recognized by eliminating verbs, adjectives and closed class words. We have built a CRF based bi­nary classifier for noun identification. Training data of 13,425 words has been developed manually by annotating each word as noun or not-noun. Next we have extracted the following features for each word of annotated corpus:</p><p><b>• Morphological features: </b>Morphological an­alyzer developed at University of Hyderabad over the last many years has been used to ob­tain the root word and the POS category for the given word. A morphological analyzer is use­ful in two ways. Firstly, it helps us to recog­nize inflected forms (which will not be listed in the dictionary) as not named entities. Secondly, word forms not recognized by morphology are likely to be named entities.</p><p><b>• Length: </b>This is a binary feature whose value is 1 if length of the given word is less than or equal 3 characters, otherwise 0. This is based on the observation that very short words are rarely nouns.</p><p><b>• Stop words: </b>A stop word list including func­tion words has been collected from exist­ing bi-lingual dictionaries. Bi-lingual dic­tionaries used for our experiments include C P Brown's English-Telugu dictionary (Brown, 1997), Telugu-Hindi dictionary developed at University of Hyderabad and the Telugu-English dictionary developed by V Rao Ve-muri. We have also extracted high frequency words from our corpora. Initially words which have occurred 1000 times or more were se­lected, hand filtered and added to the stop word list. Then, words which have occurred 500 to 1000 times were looked at, hand filtered and added to the stop word list. The list now has 1731 words. If the given word belongs to this list, the feature value is 1 otherwise 0.</p><p><b>• Affixes: </b>Here, we use the terms prefix/suffix to mean any sequence of first/last few charac­ters of a word, not necessarily a linguistically meaningful morpheme. The use of prefix and suffix information is very useful for highly in­flected languages. Here we calculate suffixes of length from 4 characters down to 1 char­acter and prefixes of length from 7 characters</p><page local="4" global="44"/><p>down to 1 character. Thus the total number of prefix/suffix features are 11. For example, for the word "virigiMdi" (broke), the suffixes are "iMdi, Mdi, di, i" and the prefixes are "virigiM, virigi, virig, viri, vir, vi, v". The feature values are not defined (ND) in the following cases:</p><p><b>- </b>If length of a word is less than or equal to 3 characters, all the affix values are ND.</p><p><b>- </b>If length of a word is from 4 to 6 charac­ters, initial prefixes will be ND.</p><p><b>- </b>If the word contains special symbols or digits, both the suffix and prefix values are</p><p>ND.</p><p><b>• Position: </b>This is a binary feature, whose value is 1 if the given word occurs at the end of the sentence, otherwise 0. Telugu is a verb final language and this feature is therefore signifi­cant.</p><p><b>• POS: </b>A single dictionary file is compiled from the existing bi-lingual dictionaries. This file in­cludes the head word and its Part of Speech. If a given word is available in this file, then its POS tag is taken as feature otherwise feature value is 0.</p><p><b>• Orthographic information </b>This is a binary feature whose value is 1 if a given word con­tains digits or special symbols otherwise the feature value is 0.</p><p><b>• Suffixes </b>A list of linguistic suffixes of verbs, adjectives and adverbs were compiled from (Murthy and J.P.L.Gywnn, 1985) to recognize not-nouns in a given sentence. This feature value is 1 if the suffix of the given word be­longs to this list, otherwise it is 0.</p><p>A feature vector consisting of the above features is extracted for each word in the annotated corpus. Now we have training data in the form of <b>(Wj; </b>Tj), where <b>W</b>j is the <i>ith </i>word and its feature vector, and <i>Tj</i><i> </i>is its tag - NOUN or NOT-NOUN. The feature template used for training CRF is shown in Table-1, where wj is the current word, is previous word, <i>wj_2</i><i> </i>is previous to previous word, <i>wj+1</i><i> </i>is next word and wj+2 is next to next word.</p><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">_Wj-2_</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">_Wj-l_</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">_Wi_</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">_Wj+l_</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">_Wj+2_</doubt><p>combination of <i>Wi-i, Wi </i>combination of <i>Wi, Wi+\ </i>feature vector of <i>Wi </i>morph tags of <i>Wj-2, w%-\, Wj, wi+i </i>and <i>wi+2 </i>output tag of current and previous word</p><table caption="Table 1: Feature Template used for Training CRF based Noun Tagger"></table><p>The inputs for training CRF consists of the train­ing data and the feature template. The model learned during training is used for testing. Apart from the basic features described above, we have also experi­mented by including varying amounts of contextual information in the form of neighbouring words and their morph features. Let us define:</p><doubt alpha="61.1" length="36" tooSmall="False" monospace="0.0">• F1:[(wj), feature vector ofwj, tj,</doubt><p>• F2 : [wj<b>_i, </b>wj<b>+i, </b>(wj<b>_i, </b>wj), (wj, wj<b>+i) </b>and the morph tags of wj_i and wj+i].</p><doubt alpha="53.5" length="43" tooSmall="False" monospace="0.0">• F3 :[wj_2, wj+2,morph tags ofwj_2andwj+2]</doubt><p>The CRF trained with the basic template F1, which consists of the current word, the feature vec­tor of the current word and the output tag of the pre­vious word as the features, was tested on a test data of 6,223 words and an F-measure of 91.95% was obtained. Next, we trained the CRF by taking the combination of F1 and F2. We also trained using combination of F1, F2 and F3. The performances of all 3 combinations are shown in Table-2. It may be seen that performance of the system is reducing as we increase the number ofneighbouring words as features. Adding contextual features does not help.</p></subsection><subsection number="5.4" title="Heuristic based NER system"><p>Nouns which have already been identified in the noun identification phase are now checked for named entities. In this work, our main focus is on identifying person, place and organization names. Indian place names and person names often have some suffix or prefix clues.<page local="5" global="45"/> For example "na:yuDu" is a person suffix clue for identifying "ra:ma:na:yuDu" as a person entity and "ba:d" is a location suffix clue for identifying "haidara:ba:d", "adila:ba:d" etc as place entities. We have manually prepared a list of such suffixes for both persons and locations as also a list of prefixes for person names. List of organization names is also prepared manu­ally. We have also prepared a gazetteer consisting of location names and a gazetteer of person name contexts since context lists are also very useful in identifying person names. For example, it has been observed that whenever a context word such as "maMtri" appears, a person name is likely to follow. Regular expressions are used to identify person entities like "en.rame:S" and organization entities which are in acronym form such as "Ti.Di.pi", "bi.je.pi" etc. Initially one file of the corpus is tagged using these seed lists and patterns. Then we manually check and tag the unidentified named entities. These new named entities are also added to the corresponding gazetteers and the relevant contexts are added to their corresponding lists. Some new rules are also observed during manual tagging of unidentified names. Here is an example of a rule:</p><p><b>"if </b>word[i] is NOUN and word[i-1] belongs to the person context list <b>then </b>word[i] is person name".</p><p>Currently the gazetteers include 1346 location names, 221 organization names, and small lists of prefixes, suffixes and other contextual cues that sig­nal the presence of named entities, their types, or their beginning or ending. Using these lists and rules, we then tag another file from the remain­ing corpus. This process of semi-automatic tagging is continued for several iterations. This way we have developed a named entity annotated database of 72,157 words, including 6,268 named entities (1,852 place names, 3,201 person names and 1,215 organization names).</p><subsubsection number="5.4.1" title="Issues in Heuristic NER"><p>There are ambiguities. For example, "ko:Tla" is a person first name in "ko:Tla vijaybha:skar" and it is also a common word that exists in a phrase such as "padi ko:Tla rupa:yalu" (10 crore rupees). There also exists ambiguity between a person entity and place entity. For example, "siMha:calaM" and "raMga:reDDi" are both person names as well as place names. There are also some problems while matching prefixes and suffixes of named entities. For example "na:Du" is a useful suffix for match­ing place names and the same suffix occurs with time entities such as "so:mava:raMna:Du". Prefixes like "ra:j" can be used for identifying person enti­ties such as "ra:jkiran", "ra:jgo:pa:l","ra:js'e:khar" etc. but the same prefix also occurs with common words like "ra:jaki:ya:lu". Thus these heuristics are not fool proof. We give below the results of our ex­periments using our heuristic based NER system for Telugu.</p></subsubsection><subsubsection number="5.4.2" title="Experiment 1"><p>Here, we have presented the performance of the heuristic-based NER system over two test data sets (AP-1 and AP-2). These test data sets are from the AP corpus. Total number ofwords (NoW) and num­ber of named entities in the test data sets AP-1 and AP-2 are given in Table-3. Performance of the sys­tem is measured in terms of F-measure. The rec­ognized named entity must be of the correct type (person, place or organization) for it to be counted as correct. A confusion matrix is also given. The notation used is as follows: PER - person; LOC -location; ORG - organization; NN - not-name. The results are depicted in Tables 4, 5 and 6.</p><table caption="Table 2: Performance of the CRF based Noun tagger with different feature combinations" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Feature combinations</p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>F-measure</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Fl</p></td><td class="cell"><p>91.64</p></td><td class="cell"><p>92.28</p></td><td class="cell"><p>91.95</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F1+F2</p></td><td class="cell"><p>91.46</p></td><td class="cell"><p>92.28</p></td><td class="cell"><p>91.86</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F1+F2+F3</p></td><td class="cell"><p>91.17</p></td><td class="cell"><p>91.99</p></td><td class="cell"><p>91.57</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="6" global="46"/><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">on AP-2</doubt></subsubsection></subsection><subsection number="5.5" title="CRF based NER system"><p>Now that we have developed a substantial amount of training data, we have also attempted supervised ma­chine learning techniques for NER. In particular, we have used CRFs. For the CRF based NER system, the following features are extracted for each word of the labelled training data built using the heuristic based NER system.</p><p><b>• Class Suffixes/Prefixes </b>This includes the fol­lowing three features:</p><p><b>- </b>Location suffix: If the given word contains a location suffix, feature value is 1 other­wise 0.</p><p><b>- </b>Person suffix: If the given word contains a person suffix, feature value is 1 otherwise it is 0.</p><p><b>- </b>Person prefix: If the given word contains a person prefix, feature value is 1 otherwise</p><p><b>• Gazetteers </b>Five different gazetteers have been used. If the word belongs to the person first name list, feature value is 1 else if the word be­longs to person middle name list, feature value is 2 else if the word belongs to person last name list, feature value is 3 else if the word belongs to location list, feature value is 4 else if the word belongs to organization list, feature value is 5 else feature value is 0.</p><p><b>• Context </b>If the word belongs to person context list, feature value is 1 else if the word belongs to location context list, feature value is 2 else if the word belongs to organization context list, feature value is 3 else the feature value is 0.</p><p><b>• Regular Expression </b>This includes two fea­tures as follows:</p><p><b>- </b>REP: This is regular expression used to identify person names. The feature value is 1 if the given word matches.</p><doubt alpha="19.8" length="81" tooSmall="False" monospace="0.0">/([a-zA-Z:~](1,3})\.( [a-zA-Z:~](1,3})?\.?( [a-zA-Z:~](1,3})?\.? [a-zA-Z:~']{4,}/</doubt><p><b>- </b>REO: This is regular expression used to identify organization names men­tioned in acronym format like "bi.je.pi", "e.ai.Di.eM.ke". etc. This feature value is 1, if the given word matches<page local="7" global="47"/></p><doubt alpha="0.0" length="62" tooSmall="False" monospace="0.0">/(.{1,3})\. (.{1,3})\. (.{1,3})\. (.{1,3})?\.? (,{1,3})?\.?/)/</doubt><table caption="Table 4: Performance of Heuristic based NER System" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>AP-1</p></td><td class="cell"><p>AP-2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>P(%)</p></td><td class="cell"><p>83.44</p></td><td class="cell"><p>97.5</p></td><td class="cell"><p>97.40</p></td><td class="cell"><p>60.57</p></td><td class="cell"><p>87.93</p></td><td class="cell"><p>87.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>r(%)</p></td><td class="cell"><p>84.84</p></td><td class="cell"><p>96.29</p></td><td class="cell"><p>87.20</p></td><td class="cell"><p>72.83</p></td><td class="cell"><p>86.56</p></td><td class="cell"><p>77.77</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F(%)</p></td><td class="cell"><p>84.13</p></td><td class="cell"><p>96.89</p></td><td class="cell"><p>92.01</p></td><td class="cell"><p>66.13</p></td><td class="cell"><p>87.23</p></td><td class="cell"><p>82.34</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table S: Number of Entities in Test Data Sets" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AP Corpus</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>NoW</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AP-1</p></td><td class="cell"><p>296</p></td><td class="cell"><p>81</p></td><td class="cell"><p>86</p></td><td class="cell"><p>3,537</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AP-2</p></td><td class="cell"><p>173</p></td><td class="cell"><p>321</p></td><td class="cell"><p>63</p></td><td class="cell"><p>7,032</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 5: Confusion Matrix for the Heuristic based System on AP-1" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Actual/Obtained</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>NN</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>285</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>12</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOC</p></td><td class="cell"><p>0</p></td><td class="cell"><p>81</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ORG</p></td><td class="cell"><p>6</p></td><td class="cell"><p>0</p></td><td class="cell"><p>75</p></td><td class="cell"><p>5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NN</p></td><td class="cell"><p>63</p></td><td class="cell"><p>3</p></td><td class="cell"><p>3</p></td><td class="cell"><p>3004</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 6: Confusion matrix of heuristic based system" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Actual/Obtained</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>NN</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>126</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>47</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOC</p></td><td class="cell"><p>2</p></td><td class="cell"><p>277</p></td><td class="cell"><p>0</p></td><td class="cell"><p>41</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ORG</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>49</p></td><td class="cell"><p>14</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NN</p></td><td class="cell"><p>80</p></td><td class="cell"><p>38</p></td><td class="cell"><p>7</p></td><td class="cell"><p>6351</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">_Wj-2_</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">_Wj-l_</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">_Wi_</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">_Wj+l_</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">47</doubt><p>• <b>Noun tagger </b>Noun tagger output is also used as a feature value.</p><p>• <b>Orthographic Information, Affixes, Mor­phological feature, Position feature, Length</b></p><p>are directly extracted from "Noun Identifica­tion" process.</p><p>The training data used for training CRFs consists of words, the corresponding feature vectors and the corresponding name tags. We have used "CRF++: Yet another CRF toolkit" (Taku, ) for our experi­ments. Models are built based on training data and the feature template. Results are given in the next subsection. These models are used to tag the test data. The feature template used in these experiments is as follows:</p><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">_Wj-3_</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">_Wi+2_</doubt><p>combination of <i>Wi-i, Wi </i>combination of <i>Wi, Wi+\ </i>feature vector of <i>Wi </i>morph tags of <i>Wj-2, w%-\, Wj, wi+i </i>and <i>wi+2</i></p><p>output tag of the previous word context information of the neighbour words</p><table caption="Table 7: Feature Template used for Training CRF"></table><subsubsection number="5.5.1" title="Experiment 2"><p>In this experiment, we took 19,912 words of training data (TR-1) and trained the CRF engine with different feature combinations of the feature template. Details of the training data (TR-1 c TR-2 c TR-3) and test data sets used in these experiments are given in Tables 8 and 9. Here the experiments are performed by varying the number of neighbouring words in the feature template. In the first case, feature template consists of current word (wj), feature vector of the current word, two neighbours of the current word (wj_i, wj+i), morph tags of the neighbour words, context information of the neighbour words, combination of current word and its neighbours and the output tag of the previous word. A model is built by training the CRF engine using this template. The model built is used in testing data sets (AP-1 and AP-2). Similarly, we repeated the same experiment by considering 4 and 6 neighbouring words of the current word in the feature template. The results are shown in Table-9 with varying number of neighbour words represented as window-size. It is observed that there is not much improvement in the performance of the system by including more of the neighbouring words as features.</p><p>Performance of the system without taking gazetteer features is shown in Table-11. We see that the performance of the system reduces when we have not considered morph features and Noun tagger output in the feature template as can be seen from</p><p>Table-12.</p><p>Finally, we have tested the performance of the system on two new test data sets (EE-1 and EE-2) from the EE corpus with varying amounts oftraining data. Total number of words (NoW) and the number of named entities in the test data sets EE-1 and EE-2 are depicted in Table-8. Performance of the system in terms of F-measure is shown in table 13.</p><p>Gazetteers have a major role in performance while morph is adding a bit. F-Measures of 74% to 93% have been obtained.<page local="8" global="48"/> Effect of training corpus size has been checked by using 19,912 words, 34,116 words and 60,525 words training corpora built from the AP newspaper corpus. Test data was from EE newspaper. It is clearly seen that larger the training data, better is the performance. See table 13.</p><table caption="Table 12: Performance of the CRF based NER Sys­tem without Morph and Noun Tagger Features"></table><table caption="Table 9: Number of Entities in Training Data Sets" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE Corpus</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>NoW</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE-1</p></td><td class="cell"><p>321</p></td><td class="cell"><p>177</p></td><td class="cell"><p>235</p></td><td class="cell"><p>6,411</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE-2</p></td><td class="cell"><p>325</p></td><td class="cell"><p>144</p></td><td class="cell"><p>187</p></td><td class="cell"><p>5221</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table S: Number of Entities in Test Data Sets</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AP corpus</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>NoW</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TR-1</p></td><td class="cell"><p>804</p></td><td class="cell"><p>433</p></td><td class="cell"><p>175</p></td><td class="cell"><p>19,912</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TR-2</p></td><td class="cell"><p>1372</p></td><td class="cell"><p>832</p></td><td class="cell"><p>388</p></td><td class="cell"><p>34,116</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TR-3</p></td><td class="cell"><p>2555</p></td><td class="cell"><p>1511</p></td><td class="cell"><p>793</p></td><td class="cell"><p>60,525</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 12: Performance of the CRF based NER System without Morph and Noun Tagger Features" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>AP-1</p></td><td class="cell"><p>AP-2</p></td><td class="cell"><p>EE-1</p></td><td class="cell"><p>EE-2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>93.76</p></td><td class="cell"><p>79.36</p></td><td class="cell"><p>70.91</p></td><td class="cell"><p>69.84</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOC</p></td><td class="cell"><p>96.81</p></td><td class="cell"><p>89.78</p></td><td class="cell"><p>81.84</p></td><td class="cell"><p>70.91</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ORG</p></td><td class="cell"><p>80.27</p></td><td class="cell"><p>91.66</p></td><td class="cell"><p>71.73</p></td><td class="cell"><p>80.75</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">Data</doubt></subsubsection><subsubsection number="5.5.2" title="Experiment 3: Majority Tag as an Additional Feature"><p>There are some names like "kRSNa:", which can refer to either person name, place name or a river name depending up on the context in which they are used. Hence, if the majority tag is incorporated as a feature, a classifier can be trained to take into account the context in which the named entity is used, as well as frequency information. In this experiment, we have used an unlabelled data set as an additional resource from the EE news corpus. The unlabelled data set consists of11,789 words.</p><p>Initially, a supervised classifier <i>hi</i><i> </i>is trained on the labelled data (TR-3) of 60,525 words. Then this classifier labels the unlabelled data set (U) (11,789 words) and produces a machine tagged data set U'. Although our NER system is not so robust, useful information can still be gathered as we shall see be­low.</p><p>Next, a majority tag list (L) is produced by ex­tracting the list of named entities with their associ­ated majority tags from the machine tagged data set U'. The process of extracting majority tag list (L) is simple: We first identify possible name classes as­signed for the named entities in U' and we assign the class that has occurred most frequently. Next, in order to recover unidentified named entities (inflec­tions of named entities already identified), we com­pare the root words of those words whose class is as­signed neither to person, place or organization with the named entities already identified. If there is any match with any of the named entities, the tag of the identified named entity is assigned to the unidentified named entity.<page local="9" global="49"/> L thus consists of (NE, Maj-tag) pairs, where Maj-tag is the name class that occurs most frequently for the named entity (NE) in the ma­chine tagged data set U'.</p><table caption="Table 10: Performance of CRF based NER system with different window sizes" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Win-Size</p></td><td class="cell"><p>AP-1</p></td><td class="cell"><p>AP-2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p>2</p></td><td class="cell"><p>99.62</p></td><td class="cell"><p>100</p></td><td class="cell"><p>98.41</p></td><td class="cell"><p>90.07</p></td><td class="cell"><p>93.55</p></td><td class="cell"><p>98.21</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p>99.62</p></td><td class="cell"><p>100</p></td><td class="cell"><p>96.96</p></td><td class="cell"><p>89.36</p></td><td class="cell"><p>93.53</p></td><td class="cell"><p>98.21</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>6</p></td><td class="cell"><p>99.62</p></td><td class="cell"><p>100</p></td><td class="cell"><p>96.96</p></td><td class="cell"><p>90.71</p></td><td class="cell"><p>93.55</p></td><td class="cell"><p>98.21</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>R</b></p></td><td class="cell"><p>2</p></td><td class="cell"><p>89.86</p></td><td class="cell"><p>93.82</p></td><td class="cell"><p>72.09</p></td><td class="cell"><p>72.15</p></td><td class="cell"><p>85.98</p></td><td class="cell"><p>87.30</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p>89.86</p></td><td class="cell"><p>93.82</p></td><td class="cell"><p>74.41</p></td><td class="cell"><p>71.59</p></td><td class="cell"><p>85.66</p></td><td class="cell"><p>87.30</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>6</p></td><td class="cell"><p>89.52</p></td><td class="cell"><p>93.82</p></td><td class="cell"><p>74.41</p></td><td class="cell"><p>72.15</p></td><td class="cell"><p>85.98</p></td><td class="cell"><p>87.30</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F</p></td><td class="cell"><p>2</p></td><td class="cell"><p>94.49</p></td><td class="cell"><p>96.81</p></td><td class="cell"><p>83.22</p></td><td class="cell"><p>80.12</p></td><td class="cell"><p>89.61</p></td><td class="cell"><p>92.43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p><b>94.49</b></p></td><td class="cell"><p><b>96.81</b></p></td><td class="cell"><p><b>84.21</b></p></td><td class="cell"><p>79.49</p></td><td class="cell"><p>89.43</p></td><td class="cell"><p>92.43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>6</p></td><td class="cell"><p>94.30</p></td><td class="cell"><p>96.81</p></td><td class="cell"><p>84.21</p></td><td class="cell"><p><b>80.37</b></p></td><td class="cell"><p><b>89.61</b></p></td><td class="cell"><p><b>92.43</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 11: Performance of the CRF based NER system without Gazetteers" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>AP-1</p></td><td class="cell"><p>AP-2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p>90.86</p></td><td class="cell"><p>97.95</p></td><td class="cell"><p>97.91</p></td><td class="cell"><p>89.05</p></td><td class="cell"><p>96.88</p></td><td class="cell"><p>96.15</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>R</p></td><td class="cell"><p>57.09</p></td><td class="cell"><p>59.25</p></td><td class="cell"><p>54.65</p></td><td class="cell"><p>69.31</p></td><td class="cell"><p>67.91</p></td><td class="cell"><p>79.36</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F</p></td><td class="cell"><p>70.12</p></td><td class="cell"><p>73.84</p></td><td class="cell"><p>70.14</p></td><td class="cell"><p>77.95</p></td><td class="cell"><p>79.85</p></td><td class="cell"><p>86.95</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 13: Performance of CRF based NER system with varying amounts of Training Data on EE Test" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test Data</p></td><td class="cell"><p>CLASS</p></td><td class="cell"><p>TR-1</p></td><td class="cell"><p>TR-2</p></td><td class="cell"><p>TR-3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE-1</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>75.14</p></td><td class="cell"><p>79.70</p></td><td class="cell"><p>81.58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOC</p></td><td class="cell"><p>81.84</p></td><td class="cell"><p>80.66</p></td><td class="cell"><p>81.45</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ORG</p></td><td class="cell"><p>76.76</p></td><td class="cell"><p>78.46</p></td><td class="cell"><p>79.89</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE-2</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>69.98</p></td><td class="cell"><p>74.47</p></td><td class="cell"><p>79.70</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOC</p></td><td class="cell"><p>70.91</p></td><td class="cell"><p>70.96</p></td><td class="cell"><p>71.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ORG</p></td><td class="cell"><p>82.13</p></td><td class="cell"><p>82.82</p></td><td class="cell"><p>83.69</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Now, we add this Maj-tag as an additional feature to labelled data (TR-3): if a word in labelled data matches with a named entity in the majority tag list (L), then the corresponding Maj-tag (name class) is assigned as a feature value to that word in the la­belled data. Finally, a classifier h2 is trained on the labelled data (TR-3). We use this classifier (h2) to tag the test data sets (EE-1 and EE-2). It can be observed from tables 14 and 15 that including the majority tag feature improves the performance a bit.</p></subsubsection></subsection></section><section number="6" title="Conclusions"><p>Not much work has been done in NER in Telugu and other Indian languages so far. In this paper, we have reported our work on Named Entity Recogni­tion for Telugu. We have developed a CRF based noun tagger, whose output is used as one of the feature for the CRF based NER system. We have also described how we have developed a substantial training data using a heuristic based system through boot-strapping. The CRF based system performs better when compared with the initial heuristic based system. We have also shown that performance of the system can be improved by adding gazetteers as features. Morphological analyser has shown a small contribution to the performance of the system. It is also observed that there is some increase in performance of the system by using majority tag con­cept. We have obtained F-measures between 80% and 97% in various experiments. It may be observed that we have not used any POS tagger or parser or annotated corpora tagged with POS or syntactic in­formation. Once adequate POS taggers and chun-kers are developed, we may be able to do better. The current work is limited to recognizing single word NEs. We plan to consider multi-token named enti­ties and nested structures in our future work.</p><table caption="Table 14: Performance of CRF based NER using Maj-tag on EE-1" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE Corpus</p></td><td class="cell"><p>Without Majority Tag</p></td><td class="cell"><p>With Majority Tag</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>P</p></td><td class="cell"><p>96.99</p></td><td class="cell"><p>98.4</p></td><td class="cell"><p>99.36</p></td><td class="cell"><p>97.02</p></td><td class="cell"><p>98.38</p></td><td class="cell"><p>98.78</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>r</p></td><td class="cell"><p>70.40</p></td><td class="cell"><p>69.49</p></td><td class="cell"><p>66.80</p></td><td class="cell"><p>71.02</p></td><td class="cell"><p>68.92</p></td><td class="cell"><p>68.93</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F</p></td><td class="cell"><p>81.58</p></td><td class="cell"><p>81.45</p></td><td class="cell"><p>79.89</p></td><td class="cell"><p>82.01</p></td><td class="cell"><p>81.06</p></td><td class="cell"><p>81.20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 15: Performance of CRF based NER using Maj-tag on EE-2" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EE Corpus</p></td><td class="cell"><p>Without Majority Tag</p></td><td class="cell"><p>With Majority Tag</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>P</p></td><td class="cell"><p>98.18</p></td><td class="cell"><p>83.96</p></td><td class="cell"><p>98.55</p></td><td class="cell"><p>98.22</p></td><td class="cell"><p>84.11</p></td><td class="cell"><p>97.88</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>R</p></td><td class="cell"><p>67.07</p></td><td class="cell"><p>61.80</p></td><td class="cell"><p>72.72</p></td><td class="cell"><p>68</p></td><td class="cell"><p>62.5</p></td><td class="cell"><p>74.31</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F</p></td><td class="cell"><p>79.70</p></td><td class="cell"><p>71.2</p></td><td class="cell"><p>83.69</p></td><td class="cell"><p>80.36</p></td><td class="cell"><p>71.71</p></td><td class="cell"><p>84.49</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>D. Appelt, J. Hobbs, J. Bear, D. Israel, M. Kameyama, A.Kehler, D. Martin, K.Meyers, and M. Tyson. 1993. SRI international FASTUS system: MUC-6 test results and analysis.</p><p>Shumeet Baluja, Vibhu O. Mittal, and Rahul Suk-thankar. 2000. Applying Machine Learning for High-Performance Named-Entity Extraction. <i>Compu­tational Intelligence, </i>16(4):586-596.</p><p>Daniel M. Bikel, Scott Miller, Richard Schwartz, and Ralph Weischedel. 1997. Nymble: a high­performance learning name-finder. In <i>Proceedings of the fifth conference on Applied natural language pro­cessing, </i>pages 194-201, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.</p><p>Charles Philip Brown. 1997. <i>Telugu-English dictionary. </i>New Delhi Asian Educational Services.</p><p>Nancy Chinchor. 1997. MUC-7 Named Entity Task Def­inition (version 3.0). In <i>Proceedings of the 7th Mes­sage Understanding Conference (MUC-7).</i></p><page local="10" global="50"/><p>M. Collins and Y. Singer. 1999. Unsupervised mod­els for named entity classification. Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora.</p><p>Asif Eqbal. 2OO6. Named Entity Recognition for Ben­gali. Satellite Workshop on Language, Artificial Intel­ligence and Computer Science for Natural Language Applications (LAICS-NLP), Department of Computer Engineering Faculty of Engineering Kasetsart Univer­sity, Bangkok, Thailand.</p><p>Hideki Isozaki and Hideto Kazawa. 2OO2. Efficient sup­port vector classifiers for named entity recognition.</p><p>In <i>Proceedings of the 19th international conference on Computational linguistics, </i>pages 1-7, Morristown, NJ, USA. Association for Computational Linguistics.</p><p>Hideki Isozaki. 2OO1. Japanese named entity recogni­tion based on a simple rule generator and decision tree learning. In <i>ACL '01: Proceedings of the 39th An­nual Meeting on Association for Computational Lin­guistics, </i>pages 314-321, Morristown, Nj, USA. Asso­ciation for Computational Linguistics.</p><p>G. Bharadwaja Kumar, Kavi Narayana Murthy, and B.B.Chaudhari. June 2OO7. Statistical Analysis of Telugu Text Corpora. <i>IJDL,Vol 36, No 2, </i>pages 71­99.</p><p>Wei Li and Andrew McCallum. 2OO3. Rapid develop­ment of Hindi named entity recognition using con­ditional random fields and feature induction. <i>ACM</i> <i>Transactions on Asian Language Information Process­ing (TALIP), </i>2(3):29O-294.</p><p>McCallum. 2OO3. Early results for Named Entity Recog­nition with Conditional Random Fields, feature induc­tion and web-enhanced lexicons. In <i>Proceedings of the seventh conference on Natural language learning atHLT-NAACL 2003, </i>pages 188-191, Morristown, NJ, USA. Association for Computational Linguistics.</p><p>Andrei Mikheev, Marc Moens, and Claire Grover. 1999. Named Entity Recognition without gazetteers. In <i>Pro­ceedings ofthe ninth conference on European chap­ter ofthe Association for Computational Linguistics, </i>pages 1-S, Morristown, NJ, USA. Association for Computational Linguistics.</p><p>Bh.Krishna Murthy and J.P.L.Gywnn. 1985. <i>A Grammar ofModern Telugu. </i>Oxford University Press, Delhi.</p><p>Georgios Petasis, Frantz Vichot, Francis Wolinski, Geor-gios Paliouras, Vangelis Karkaletsis, and Constan-tine D. Spyropoulos. 2OO1. Using machine learning to maintain rule-based named-entity recognition and classification systems. In <i>ACL '01: Proceedings of the 39th Annual Meeting on Association for Compu­tational Linguistics, </i>pages 426-433, Morristown, NJ, USA. Associationfor Computational Linguistics.</p><p>Taku. http://crfpp.sourceforge.net/.</p><p>Tong Zhang and David Johnson. 2OO3. A Robust Risk Minimization based Named Entity Recognition sys­tem. In <i>Proceedings ofthe seventh conference on Nat­ural language learning at HLT-NAACL 2003, </i>pages 2O4-2O7, Morristown, NJ, USA. AssociationforCom-putational Linguistics.</p><p>GuoDong Zhou and Jian Su. 2OO1. Named Entity Recognition using an HMM-based chunk tagger. In <i>ACL '02:</i><i> Proceedings </i><i>ofthe</i><i> 40th Annual Meeting on Association for Computational Linguistics, </i>pages 473-4SO, Morristown, NJ, USA. Association for Com­putational Linguistics.</p></references></body></article>