Word of sense disambiguation appropriate (WSD) meaning is the pro- cess for selecting given the word in document. Obvi- or sense ously, portant WSD a processes is one needed of for fundamental a the many natural and lan- im- guage decade, processing many (NLP) applications. been Over made the past WSD of Japanese. studies Most current have on machine learning (Li and research Takeuchi, uses 1997; 2001), Murata techniques achieved et al., good 2001; performance. Takamura et How- al., ever, has word as sense-tagged supervised corpora, learning methods often require from sparseness, i.e., words they which suffer occur data in training do not not disambiguated. frequently Therefore, a we corpus cannot can be pervised learning algorithms in use su- NLP to applications, alone disambiguate both when frequency it practical especially is neces- sary low words. high and frequency method To tackle combine this problem, two WSD this paper classifiers. proposes One a is to The a learning classifier algorithm obtained by supervised learning. used for this classifier is the Support will Vector work well Machine (SVM); disambiguation this clas- sifier frequency words. for The the second of high Naive Bayes model, which will classifier work is the In the disambiguation model, of hypernyms low frequency well for this words. nition in sentences in overcome dictionary extracted from to a considered defi- order sparseness. are scribed The in details Section of data 2, the and SVM Naive classifier Bayes are model de- in Section 3. The combination the is in Section of 4. these The two experi- clas- sifiers mental described ported evaluation in Section of 5. the We proposed mention method some is re- works Section in Section 6, and conclude related 7. the paper in

The SVM first is classifier supervised is learning the SVM algorithm, classifier. Since sense-tagged a corpus is required training a word classifier not as to disam- data, and biguate the words which can do not be occur used in SVM However, frequently the data. widely reported as the for effectiveness NLP been including a variety of of has Takamura tasks 2001), WSD we (Murata know et al., for disambiguation al., that it 2001; work well et of high will words. frequency ing When instance training the SVM We be following represented classifier, each train- vector. should features, by a feature typical for used WSD. the which are •S (0) ,S ( − 1) ,S ( − 2) ,S (+1) ,S (+2) Surface forms or of a target word word. and in a target position A words just before after indicates num- ber word parentheses from word. the of a a target •P ( − 1) ,P ( − 2) ,P (+1) ,P (+2) Parts-of-speech or (POSs) word. of words just before after a target •S ( − 2) ·S ( − 1), S (+1) ·S (+2), S ( − 1) ·S (+1) Pairs of surface word. forms of words surrounding a target •P ( − 2) ·P ( − 1), P (+1) ·P (+2), P ( − 1) ·P (+1) Pairs of POSs word. of words surrounding a tar- get •B sent Base forms of content words in a sentence 1 . •C sent Semantic sentence. classes of content Semantic words in a derived from classes Japanese used here are “Nihongo-Goi-Taikei” the the- saurus 1997). (Ikehara et al., •B head , B mod Base ( forms ) of of the head ( word. B head ) or modifiers B mod a target • ( B case ; B noun ) A ( pair ) of and the base forms noun of a ( case marker B case word a case is filler verb. B noun ) when the target a • ( B case ; C noun ) A ( pair ) of and the base form of a case noun semantic ) class of marker B case ( the when a is noun the target word case filler verb. C a • ( B case ; B verb ) A ( pair ) of and the base forms ( of a case case head verb B verb ) when marker B verb. word is a case filler noun of the target a a certain ing We used the LIBSVM package 2 SVM The SVM model for train- SVM the (Schölkopf, classifier. 2000) with kernel, is where ν− method parameter to ν =0 SVM . 0001. a linear the is to multi The pairwise cation. used apply classifi-

In this WSD section, we will using describe hypernyms the details of words of the classifier Figure 1: Sense Set of “ 漫談 ” extracted nary. from definition sentences in a dictio-

The in details of two WSD two classifiers sections: are one is described the previous for frequency words, the SVM classifier other is high Naive Bayes for and low the the words. These classifier bined to two classifiers fre- quency construct robust WSD are com- developed two the kinds of combined system. in 4.1 models, We below subsections and 4.2. de- scribed

In to this section, EDR our we will describe proposed (EDR, method. the experiment evaluate corpus 1995) in We experiment. used the is made of about 200,000 Japanese the It up from newspaper sentences extracted zines. In with EDR corpus, articles sense ID (CID). each word and maga- notated the We was an- sentences in a EDR used 20,000 20,000 sentence the corpus as the test data, remaining 161,332 as the sentences validation data, training and the data. The training was as the SVM and data Naive used Bayes to train the while classifier the bined validation data in was used for classifier, the model Subsection 4.2. the com- instances described for were am- The target biguous content words used in evaluation all of instances was the test data; the num- ber We target combined three single 91,986. two evaluated models: WSD classifiers and • BL

The baseline which always model. This is the WSD most clas- sifier sense. selects When the is more fre- quently than one used sense with there chooses equally high senses. frequency, the classifier all those • NB

The Naive Bayes classifier (Section 3).

• SVM

The SVM classifier (Section 2).

• SVM+NB(simple)

The (Subsection combined 4.1). model by simple ensemble • SVM+NB(valid)

The combined (Subsection model 4.2). using the validation data measure(F) Table 1 reveals , applicability(A) the precision(P), recall(R), word of classifiers and number F- 4 types(T) on of these five the test Table 1: Results of WSD Classifiers data. A(applicability) number of instances indicates disambiguated the ratio of classi- the to indicates total number number of of word target instances; by a fier the which T disambiguated the types could be by a classifier. The SVM two combined classifier, for models criteria outperformed precision. the The in all remarkable. applicability except gains recall 1: Notice and were classifiers the figures in column espe- cially “T” in only Table to 4,575 the words, SVM while could Naive be Bayes ap- plied classifiers cluding were to 10,501 the low frequency applicable words. Thus, words, ensem- in- of two would significantly the ble improve these of precision. applicability classifiers and recall with little loss bined Comparing models, the “SVM+NB(validation)” performance of the two com- outperformed was no “SVM+NB(simple)”, slightly significant difference between them. but there correctness, , of SVM on The C w was the classifier than the validation Naive Bayes data usually greater SVM that of the was when classifier, both were so the applicable. classifier was preferred almost same This the and we think strategy this was for the simple why en- semble, performance of two combined the models reason the almost will the same. In the rest of this section, were show for combined model we using the results only. the the validation data WSD Our system. goal was The to improve naive way the to robustness construct of the WSD is ensemble a of ro- bust supervised learned system to create and an a So, we compared classifier our proposed a baseline method clas- sifier. (SVM+NB) SVM and baseline with the combined shown Table classifier 2 and (SVM+BL). model Figure 4. The of the in Table re- sults shows are same criteria in Table 1, indicating 2 “SVM+NB” the outperformed as that criteria. Figure 4 shows “SVM+BL” between for all F-measure of classifiers the relation the in training the The horizontal and word fre- quency the data. axis Table 2: Results of the Combined Models (1) F(o) N(o) Figure 4: Results of the Combined Models (2) indicates ing the ) in occurrence log scale. of Squares words in and the triangles train- with data “SVM+BL”, lines ( indicates o respectively, the F ( o ) of where the “SVM+NB” and macro of F-measures for words F ( o which )isthe occur average indicates o times in ( the ), training number data. of The word broken line which occur N o the . For convenience, types = 0, we o times 5 plot (0) and N (0) when o F at x = − instead of (= 0). As shown in Fig- 0 . 5 4, “SVM+NB” −∞ log significantly 10 ure “SVM+BL” low frequency words, outperformed number of word for ( ( )) obviously and the when types Naive classifier o was small. N o proposed In became greater Bayes other could words, prob- the handle many more low frequency here ably classifier. Therefore, it words than more the baseline to combine Naive Bayes clas- was effective with SVM the sifier baseline the in order classifier bustness of overall WSD to improve rather than the classifier system. the ro- the Finally, we constructed SVM, a combined Naive Bayes model and of all baseline three classifiers, the model classifiers. outperformed As shown in Table two-classifier 3, this combined slightly models shown in Table the 2. Table 3: Results of SVM+NB+BL

As described was to in Section improve 1, robustness the goal of of this project system. One of promising ways to the WSD robust WSD the system is construct a ing with EM unsupervised algorithm (Manning learn- Schütze, as 1999), the i.e. training WSD and from our an to data set. On a classifier approach unlabeled is machine the other hand, in addition use a corpus readable dic- tionary WSD. Notice to a we as knowledge resources nyms of definition for sentences that in used hyper- train Naive Bayes a and dictionary pro- to worked the well for words classifier, which did this cess in corpus. However, not we did oc- cur not frequently compare our method the learning method empirically. and the of projects. This unsupervised our will be one future Using to hypernyms semantic of definition derived sentences from is similar thesaurus. using One of advantages classes of our method a is the senses that a thesaurus according is not obligatory to machine when read- word dictionary. are defined Furthermore, a able probabilistic model our method predicts is to proaches hypernym train the of a word, while most that a semantic previous (i.e., ap- condition use of posterior classes as of the Bayes model). probability features the Naive In facts, in the we case the also derived use features from associated ), described thesaurus, with semantic ( classes the in Section C 2. sent and B case ; C noun as Several previous machine studies have dictionary used both for a corpus WSD (Litkowski, and a 2002; readable Stevenson and Wilks, 2001). Rigau The et al., those and ours is difference 1997; between we information methods for Training derived probabilistic from the model dictionary the way use predicts WSD. hypernym the in dictionary is our that a not competition However, a with these our various method. methods own approach. in In are robustness of WSD would fact, the more improved the with by combining system in this paper. these meth- be even ods that described

This paper robust has WSD proposed system. a We method to combined develop a obtained supervised a WSD classifier frequency by words and learning pernyms classifier in using for high in definition sentences a hy- for low frequency words. a dictionary showed remarkably both Experimental results that recall improved and with applicability our method. were ture, we to investigate optimum In two or to train way fu- combine plan the to probabilistic these model classifiers using hypernyms in a defini- single tion and low sentences, which frequency is suitable words. for both high References EDR. 1995. guide EDR (second electronic edition). dictionary Technical Report technical Japan Ikehara Electronic Dictionary 1997. Nihongo Research Goi Institute. TR–045, Satoshi Japanese) . Iwanami et al. Shoten, Publishers. Taikei (in Hand Li and is Jun-ichi both strong Takeuchi. and 1997. in Using Japanese evi- dence homograph that disambiguation. In reliable Processing Litkowski. Society of Japan SIG-NL, Sense , pages 53–59. Informa- Kenneth tion disambiguation: C. Confluence 2002. of information for and methods. In Proceedings supervised SIGLEX/SENSEVAL unsupervised Sense of the Disambiguation , Workshop 47–53. on Word Christopher Foundations D. Manning pages Statistical and Natural Hinrich Language Sch¨ utze, Pro- 1999. Masaki cessing, of Murata, chapter Masao 7. MIT Utiyama, Press. moto, Qing Ma, Hitoshi Kiyotaka Japanese and disambiguation Isahara. Uchi- word 2001. and sense support machine using In Proceedings vector SENSEVAL-2 meth- the simple ods. bayes 135–138. of the , pages Ted ing Pedersen. ensembles 2000. of naive A simple baysian approach to build- word disambiguation. In Proceedings classifiers for NAACL, sense 63–69. of the German Rigau, pages 1997. Combining Jordi unsupervised Atserias, and Eneko knowledge Agirre. methods for word disambiguation. lexical ACL sense Sch the ölkopf. 2000. , pages New 48–55. In Pro- Bernhard ceedings of gorithms. Neural Computation support , 12:1083–1121. vector al- Mark action Stevenson of knowledge and Yorick sources Wilks. in word 2001. The disam- inter- biguation. Computational Linguistics sense 349. , 27(3):321– Hiroya Kaoru Takamura, Yamamoto, Hiroyasu Yuji Yamada, Matsumoto. Taku Kudoh, Ensembling with based to on and WSD. feature In space Proceedings restructuring 2001. NLPRS, application 41–48. of the pages