<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="25"/><title>Sensiting Inflectionality: Estonian Task for SENSEVAL-2</title><author surname="Kahusk" givenname="Neeme"><org  name="University of Tartu" country="Estonia" city="Tartu"/></author><author surname="Orav" givenname="Heili"><org  name="University of Tartu" country="Estonia" city="Tartu"/></author><author surname="Õim" givenname="Haldur"><org  name="University of Tartu" country="Estonia" city="Tartu"/></author></firstpageheader><frontmatter><p>Sensiting inflectionality: Estonian task for SENSEVAL-2</p><p>Neeme Kahusk and Heili Orav and Haldur Oim</p><p>University of Tartu Research Group of Computational Linguistics Tiigi 78, 50410 Tartu, Estonia {nkahusk,horav,hoim}@psych.ut.ee</p></frontmatter><abstract>This paper describes the all-word sense disam­biguation task provided by Estonian team at Senseval-2. About 10,000 words are manually disambiguated according to Estonian WordNet word senses. Language-specific problems and lexicon features are discussed. </abstract></header><body><section number="1" title="Introduction"><p>We got interested in word sense disambiguation (WSD) for two reasons. First, already a couple of years ago it was evident that WSD is beco­ming one of the new "hot" topics in computatio­nal linguistics and language engineering as our knowledge of how to handle semantic parame­ters of texts and semantic features of words in texts increased. The second reason was purely practical. Since 1996 we have been involved in a large project of building a semantic database of Estonian; participating in the Euro WordNet project has been a part of it (but a very impor­tant part, of course). The main source of buil­ding this database have been different corpora of Estonian, and in working with corpora the question of whether we are dealing with diffe­rent meanings of a word in case of its concrete occurrences or not arises constantly. So we got interested in the possibility to use some objec­tive methods here.</p><p>Our task was all-words task. This choice is explained with our "practical" interests explai­ned above.</p><p>A large amount of work was done to provide training data where disambiguation was done manually. The same kind of work had to be done with test data, of course. The description of this work is given below. Let us note already here that this work appeared to be very useful and informative for us as builders of Estonian WordNet (EstWN).</p><p>And let us stress that this was our first at­tempt of WSD at all.</p></section><section number="2" title="Corpora and lexicon"><p>The test and training texts come from Corpus of the Estonian Literary Language (CELL), the 1980-s. We used this part of the corpus, that was morphologically disambiguated, initially for the syntactic analysis.</p><p>The morphological analysis was made with ESTMORF (Kaalep, 1997). Lemma and word class in the output of the program are relevant to our task, but it is impossible to get them without morphological disambiguation, because of frequent homonymy among word forms.</p><p>All training texts and most of test texts (5 of 6 total) are fiction. One of the test texts is from newspaper. Six training and six test files provided for the task contain about 2000 tokens each. More information about the texts used in the task is in Table 1.</p><table caption="Table 1: Statistics on training and test corpora"></table><subsection number="2.1" title="Lexicon"><p>The Estonian part of Euro WordNet<footnote anchor="1"/> served as the lexicon. Like other wordnets, EstWN is a lexical-semantic database, the basic unit of which is concept. Concepts are represented as synonym sets (synsets) that are linked to each other by semantic relations. The description of<page local="2" global="26"/></p><footnote label="1"> http://www . hum. uva.nl/~ewn/</footnote><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">2R</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Corpus</p></td><td class="cell"><p>Training</p></td><td class="cell"><p>Test</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Total words</p></td><td class="cell"><p>12162</p></td><td class="cell"><p>11440</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Words to disambiguate</p></td><td class="cell"><p>5854</p></td><td class="cell"><p>5650</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>of them being</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verbs</p></td><td class="cell"><p>2431</p></td><td class="cell"><p>2191</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>nouns</p></td><td class="cell"><p>3423</p></td><td class="cell"><p>3459</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>EstWN is given in the final document of Eu­roWordNet (Vider et al., 1999).</p><p>EstWN is supposed to cover the Estonian base vocabulary in its initial version. The base vocabulary will be determined by statistical analysis of the reference corpus. Even so it is not always easy (nor appropriate) to stop enco­ding words with frequencies below a certain th­reshold. For this reason we expect EstWN to cover more than just the base vocabulary.</p><p>Still the EstWN is rather small, there were 9436 synsets, 13277 words and 16961 senses (li­terals) in it when the disambiguation was done. That makes about 1.28 senses per word as ave­rage.</p><p>Most of synsets are connected with hypero-nym-hyponym relations building corresponding hierarchies.</p></subsection><subsection number="2.2" title="Procedure"><p>Four linguists disambiguated the texts, each text was disambiguated by two persons. Only nouns and verbs were disambiguated, as ente­ring adjectives into EstWN is in the very begin­ning. The sense number was marked according to sense number in EstWN. If the word was mis­sing from the EstWN, "0" was marked as sense number, and if the word was in EstWN, but missed the appropriate sense, "+1" was mar­ked.</p><p>If inconsistencies were met, they were discus­sed until agreement was achieved. On about 28% of the cases the disambiguators had diffe­rent opinions.</p><p>One of the problems that the disambiguators ran into concerned dividing words into diffe­rent senses in EstWN. It turned out as over-differentiation—word meaning marked as too specific, or over-generalisation—word meaning marked as too general.</p></subsection><subsection number="2.3" title="How much the lexicon covers"><p>Not all senses found in EstWN are represented in texts. Maximum number of senses per word found in texts is 13. This is more than app­ropriate senses in lexicon (see Table 3), but we must remeber about the "+1" that disambigua­tors had, if they found that there are not enough meanings in EstWN. Table 2 describes distri­bution of senses in usage and Table 3 shows the top of lemmas according to number of senses.</p><table caption="Table 2: Distribution of lemmas according1number of senses in texts_j"></table><p>It would be the best, if all words to disambi­guate were in the lexicon with all their possibk meanings. Apparently this presumption is not met.</p><p>The number of compounds in Estonian is in­definite. It is quite easy for a writer to in­vent new compounds that are not in any dic­tionary, but nevertheless are easily understood by readers. That is one reason, why there are so many sense numbers "0" in the texts. About 46 % of words that are not in EstWN, are com­pounds.</p><p>Another remarkable class of words not in lexicon are proper names, as there are no pro­per names in EstWN. There are 17.5 <i>% </i>of words proper names.</p><p>If we will postpone phrasal verbs and some strange words that contain hyphens (about 7 %), it leaves us with about half thousand words to check why they are not in EstWN.<page local="3" global="27"/></p><table caption="Table 3: Comparison of richest words in sense" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Corpus</p></td><td class="cell"><p>Training</p></td><td class="cell"><p>Tesl</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Total number of lemmas</p></td><td class="cell"><p>2340</p></td><td class="cell"><p>226É</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Number of lemmas not in</p></td><td class="cell"><p>819</p></td><td class="cell"><p>94É</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>lexicon</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Number of lemmas with 1</p></td><td class="cell"><p>2040</p></td><td class="cell"><p>2003</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>sense in texts</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lemmas with 2 senses in</p></td><td class="cell"><p>215</p></td><td class="cell"><p>183</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>texts</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lemmas with 3 senses in</p></td><td class="cell"><p>51</p></td><td class="cell"><p>50</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>texts</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lemmas with 4 senses in</p></td><td class="cell"><p>17</p></td><td class="cell"><p>17</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>texts</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lemmas with more than 4</p></td><td class="cell"><p>17</p></td><td class="cell"><p>15</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>senses in texts</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>POS</p></td><td class="cell"><p>No of senses</p></td><td class="cell"><p>Lemma</p></td><td class="cell"><p>No of senses</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>in text</p></td><td class="cell"><p></p></td><td class="cell"><p>in lexicon</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>13</p></td><td class="cell"><p>saama</p></td><td class="cell"><p>12</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>10</p></td><td class="cell"><p>pidama</p></td><td class="cell"><p>12</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>noun</p></td><td class="cell"><p>10</p></td><td class="cell"><p>asi</p></td><td class="cell"><p>11</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>9</p></td><td class="cell"><p>olema</p></td><td class="cell"><p>9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>9</p></td><td class="cell"><p>käima</p></td><td class="cell"><p>23</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>7</p></td><td class="cell"><p>vötma</p></td><td class="cell"><p>7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>7</p></td><td class="cell"><p>panema</p></td><td class="cell"><p>11</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>7</p></td><td class="cell"><p>nägema</p></td><td class="cell"><p>7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>7</p></td><td class="cell"><p>minema</p></td><td class="cell"><p>17</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>verb</p></td><td class="cell"><p>7</p></td><td class="cell"><p>leidma</p></td><td class="cell"><p>8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>noun</p></td><td class="cell"><p>7</p></td><td class="cell"><p>elu</p></td><td class="cell"><p><footnote anchor="7"/>I</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>But why are there missing senses (tagged with "+1")? The reason is simply historical: such words were included into EstWN as sy­nonyms of some base vocabulary word and the other senses of them are not considered yet.</p></subsection><subsection number="2.4" title="Phrases and multi-word units"><p>The initial format of text was as it came from ESTMORF and semantic disambiguation: every word on separate line, followed by an adi-tional line of morphological analysis and sense number, with multi-word phrase marked if word was part of it. The task to convert into Senseval XML format seemed trivial at first, but phrases turned out to be problematic. Unfortunately enough, all the story about phrases is concer­ning the training corpus only, because in test corpus the multi-word phrases were unmarked.</p><p>Estonian is a flective language with a free word order and that makes it complicated to figure out all phrases. The elements of a ph­rase can be scattered around the sentence in an unpredictable order.</p><p>In the initial texts, the disambiguators mar­ked down the whole phrase on the line where the phrase occured. They were not told to mark it on each line, where the non-disambiguatable parts of the phrase were, and it happened that the phrase was not marked on the line, where the head of the phrase was. The algorithm of calculating head or satellite took into account the part of speech and the form. For verb phra­ses, if both components were verbs, declinable form of verb infinitive was marked as satellite. For noun phrases, substantive makes head and adjective satellite. If both words are substanti­ves, head is the second one... well, mostly.</p><p>However, it is known that expressions tend to contain frozen forms, including inflectional en­dings. For example, one may not say "*Human Right" or "*Humans Right". "Human Rights" is the only correct expression and should be ad­ded into thesauri in such form. Phrasal verbs like "ara maksma" (to pay off) and idiomatic verbal expressions like "end tiikkideks naerma" (to laugh oneself into pieces) represent a situa­tion that is different from the occasion desc­ribed above: the verb part may inflect freely, but the other word(s) are frozen forms. He­reby, even if we have determined what is phrase or collocational multi-word unit, we still have a question— are they commonly used and should we add them into the lexicon.</p><p>Multiword expressions are included into EstWN if they build up a conceptual unit and are commonly used as lexical units.</p></subsection></section><section number="3" title="Results"><p>There were two systems to solve the task on Estonian. The results are in Table 4. Table 5 shows the recall and precision of the COMMO­NEST baseline</p><table caption="Table 4: Estonian all-words fine-grained scoringresults_,_,"></table><p>As this is the first attempt to disambiguate Estonian nouns and verbs in text, there is no comparison data. These results will set the level that future systems will try to outgo.</p></section><section number="4" title="Conclusions"><p>Results of WSD of corpus texts turned to be a good way to add missing synsets and sen­ses into our wordnet. There were significant inconsistencies in opinions of these people, who disambiguated the texts. This shows us the most problematic entries in EstWN, the need to reconsider the borders of meaning of some concepts. By now, the last version of EstWN contains 9524 synsets, 13344 words and 17076 senses.</p><p>For an inflectional language like Estonian, morphological analysis is extremely important and morphological and semantic disabiguation can help each other.</p><table caption="Table 5: COMMONEST baseline for Estonianall-words task__" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>System</p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>Attempted</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>JHU</p><p>est-semyh</p></td><td class="cell"><p>0.67 0.66</p></td><td class="cell"><p>0.67 0.66</p></td><td class="cell"><p>100 100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Data</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>Precision</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall Polysemous</p></td><td class="cell"><p>0.85 0.69</p></td><td class="cell"><p>0.73</p><p>0.51</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>H.-J. Kaalep. 1997. An estonian morphological analyser and the impact of a corpus on its de­velopment. <i>Computers and the Humanities, </i>31:115-133.</p><page local="4" global="28"/><p>K. Vider, L. Paldre, H. Orav, and H. Öim. 1999. The Estonian Wordnet. In C. Kunze, edi­tor, <i>Final Wordnets for German, French, Es­tonian and Czech. </i>Euro WordNet (LE-8328), Deliverable 2D014.</p></references></body></article>