<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="134"/><title>UHD: Cross-Lingual Word Sense Disambiguation Using Multilingual Co-Occurrence Graphs</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 134-137, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Silberer" givenname="Carina"><org  name="Heidelberg University" country="Germany" city="Heidelberg"/></author><author surname="Ponzetto" givenname="Simone Paolo"><org  name="Heidelberg University" country="Germany" city="Heidelberg"/></author></firstpageheader><frontmatter><p><b>UHD: Cross-Lingual Word Sense Disambiguation Using Multilingual Co-occurrence Graphs</b></p><p><b>Carina Silberer </b>and <b>Simone Paolo Ponzetto</b></p><p>Department of Computational Linguistics Heidelberg University</p><p>{silberer,ponzetto}@cl.uni-heideIberg.de</p></frontmatter><abstract>We describe the University of Heidelberg (UHD) system for the Cross-Lingual Word Sense Disambiguation SemEval-2010 task (CL-WSD). Lhe system performs CL-WSD by applying graph algorithms pre­viously developed for monolingual Word Sense Disambiguation to multilingual co­occurrence graphs. UHD has participated in the Best and out-of-frve (OOL) eval­uations and ranked among the most com­petitive systems for this task, thus indicat­ing that graph-based approaches represent a powerful alternative for this task. </abstract></header><body><section number="1" title="Introduction"><p>Lhis paper describes a graph-based system for Cross-Lingual Word Sense Disambiguation, i.e. the task of disambiguating a word in context by providing its most appropriate translations in dif­ferent languages (Lefever and Hoste, 2010, CL-WSD henceforth). Our goal at SemEval-2010 was to assess whether graph-based approaches, which have been successfully developed for monolingual Word Sense Disambiguation, represent a valid framework for CL-WSD. Lhese typically trans­form a knowledge resource such as WordNet (Fell-baum, 1998) into a graph and apply graph algo­rithms to perform WSD. In our work, we follow this line of research and apply graph-based meth­ods to <i>multilingual co-occurrence graphs </i>which are automatically created from parallel corpora.</p></section><section number="2" title="Related Work"><p>Our method is heavily inspired by previous pro­posals from Véronis (2004, Hyperlex) and Agirre et al. (2006).   Hyperlex performs graph-based</p><p>WSD based on <i>co-occurrence graphs: </i>given a monolingual corpus, for each target word a graph is built where nodes represent content words co-occurring with the target word in context, and edges connect the words which co-occur in these contexts. Lhe second step iteratively selects the node with highest degree in the graph (root hub) and removes it along with its adjacent nodes. Each such selection corresponds to isolating a high-density component of the graph, in order to select a sense of the target word. In the last step the root hubs are linked to the target word and the Mini­mum Spanning Lree (MSL) of the graph is com­puted to disambiguate the target word in context. Agirre et al. (2006) compare Hyperlex with an al­ternative method to detect the root hubs based on PageRank (Brin and Page, 1998). PageRank has the advantage of requiring less parameters than Hyperlex, whereas the authors ascertain equal per­formance of the two methods.</p></section><section number="3" title="Graph-based Cross-Lingual WSD"><p>We start by building for each target word a mul­tilingual co-occurrence graph based on the target word's aligned contexts found in parallel corpora (Sections 3.1 and 3.2). Multilingual nodes are linked by translation edges, labeled with the target word's translations observed in the corresponding contexts. We then use an adapted PageRank al­gorithm to select the nodes which represent the target word's different senses (Section 3.3) and, given these nodes, we compute the MSL, which is used to select the most relevant words in con­text to disambiguate a given test instance (Section 3.4). Lranslations are finally given by the incom­ing translation edges of the selected context words.</p><page local="2" global="135"/><subsection number="3.1" title="Monolingual Graph"><p>Let <i>Cs </i>be all contexts of a target word <i>w </i>in a source language <i>s,</i><i> </i>i.e. English in our case, within a (PoS-tagged and lemmatized) monolin­gual corpus. We first construct a monolingual co­occurrence graph <i>G </i><b><i>g </i></b><i>= (VS,ES). </i>We collect all pairs <i>(cwi,cwj) </i>of co-occurring nouns or adjec­tives in <i>Cs </i>(excluding the target word itself) and add each word as a node into the initially empty graph. Each co-occurring word pair is connected with an edge <i>(vi,Vj) </i>g <i>Es, </i>which is assigned a weight <i>w(vi, </i><i>Vj)</i><i> </i>based on the strength of associa­tion between the respective words <i>cwi </i>and <i>cwj :</i></p><p><i>w(vi,Vj) </i>= 1 — max <i>[p</i><i>(cwi\cwj</i><i>),p</i><i>(cwj\cwi)].</i></p><p>Lhe conditional probability of word <i>cwi</i><i> </i>given word <i>cwj </i>is estimated by the number of contexts in which <i>cwi</i><i> </i>and <i>cwj </i>co-occur divided by the number of contexts containing <i>cwj.</i></p></subsection><subsection number="3.2" title="Multilingual Graph"><p>Given a set of target languages <i>L, </i>we then ex­tend <i>G </i><b><i>g </i></b>to a labeled multilingual graph <i>G ml = (Vml,Eml) </i>where:</p><p>1 • <i>y </i><i>m l </i><i>= Vs </i>u UieL ^ is a set °f n°des represent­ing content words from either the source <i>(Vs) </i>or the target <i>(Vi) </i>languages;</p><p>2. <i>EML = Es </i>u <i>\JleL{Ei u E*,i} is a set of</i>edges. Lhese include (a) <i>co-occurrence edges Ei ÇVixVi </i>between nodes representing words in a target language <i>(Vi), </i>weighted in the same way as the edges in the monolingual graph; (b) labeled <i>translation edges Esj </i>which repre­sent translations of words from the source lan­guage into a target language. Lhese edges are assigned a complex label <i>t </i>g <i>Twj </i>compris­ing a translation of the word <i>w </i>in the target language <i>I </i>and its frequency of translation, i.e. <i>Es,i </i>ç^x <i>Tw&gt;i </i>x <i>Vh</i></p><p>Lhe multilingual graph is built based on a word-aligned multilingual parallel corpus and a multi­lingual dictionary. Lhe pseudocode is presented in Algorithm 1. We start with the monolingual graph from the source language (line 1) and then for each target language <i>l </i>g <i>L </i>in turn, we add the transla­tion edges <i>(vs,t,vi) </i>g <i>Esj </i>of each word in the source language (lines 5-15). In order to include the information about the translations of <i>w </i>in the different target languages, each translation edge <b>Algorithm 1 </b>Multilingual co-occurrence graph. <b>Input: </b>target word <i>w </i>and its contexts <i>Cs</i> monolingual graph <i>Gs = (Vs, Es)</i></p><p>set of target languages <i>L </i><b>Output: </b>a multilingual graph <i>G ml</i> 4: <i>Ci := </i>aligned sentences of <i>Cs </i>in lang. <i>I</i> 5: <b>for each </b><i>vs </i>g <i>Vs</i> 6: <i>TVsti := </i>translations of <i>vs </i>found in <i>Ci</i> 7: <i>CVs </i>ç <i>Cs </i>:= contexts containing <i>w </i>and <i>vs</i> 8: <b>for each </b>translation <i>vi </i>g <i>TVayi</i> 9: <i>CVl := </i>aligned sentences of <i>CVs </i>in lang. <i>I</i> 10: <i>Tw</i><i> c„ </i>&lt;— translation labels of <i>w </i>from <i>Cv,</i></p><doubt alpha="57.8" length="45" tooSmall="False" monospace="0.0">l:Gml = (Vml, Eml)&lt;—Gs= (Vs, Es)2:for eachlgL</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">3:Vj^0</doubt><p>il: <b>if </b><i>vi £ </i><i>Vml </i><b>then</b></p><doubt alpha="64.3" length="14" tooSmall="False" monospace="0.0">12:Vml&lt;-Vmluvt</doubt><doubt alpha="58.3" length="12" tooSmall="False" monospace="0.0">13:Vi&lt;-ViUVi</doubt><p>14: <b>for each </b><i>t </i>g <i>TWyCvi</i> 16: <b>for each </b><i>Vi </i>g <i>Vi</i> 17: <b>for each </b><i>Vj</i><i> </i>g <i>V, </i><i>i </i>/ <i>j</i> 18: <b>if </b><i>Vi </i>and <i>Vj</i><i> </i>co-occur in <i>Ci </i><b>then</b> 20: <b>return </b>Gml <i>(vs, t, vi) </i>receives a translation label <i>t.</i><i> </i>Formally, let <i>CVs </i>ç <i>Cs </i>be the contexts where <i>vs </i>and w co-occur, and <i>CVl </i>the word-aligned contexts in lan­guage <i>I </i>of <i>CVs, </i>where vs is translated as <i>vi. </i>Lhen each edge between nodes <i>vs </i>and <i>vi </i>is labeled with a translation label i (lines 14-15): this includes a translation of <i>w </i>in C^, its frequency of transla­tion and the information of whether the transla­tion is monosemous, as found in a multilingual dictionary, i.e. EuroWordNet (Vossen, 1998) and PanDictionary (Mausam et al., 2009). Finally, the multilingual graph is further extended by inserting all possible co-occurrence edges <i>(vi, </i><i>Vj)</i><i> </i>g <i>Ei </i>be­tween the nodes for the target language <i>I </i>(lines 16-19, i.e. we apply the step from Section 3.1 to <i>I </i>and <i>Ci). </i>As a result of the algorithm, the multilingual graph is returned (line 20).</p><doubt alpha="52.2" length="23" tooSmall="False" monospace="0.0">15:Eml&lt;-Emlu(vs, t, vi)</doubt><doubt alpha="50.0" length="22" tooSmall="False" monospace="0.0">19:Eml *   Emlu(vi,Vj)</doubt></subsection><subsection number="3.3" title="Computing Root Hubs"><p>We compute the root hubs in the multilingual graph to discriminate the senses of the target word in the source language. Hubs are found using the adapted PageRank from Agirre et al. (2006):</p><page local="3" global="136"/><doubt alpha="34.6" length="26" tooSmall="False" monospace="0.0">PR(vi) =(1 -d)+d^-—-Pfi(^)</doubt><p>where <i>d </i>is the so-called damping factor (typically set to 0.85), <i>deg</i><i>(vi) </i>is the number of adjacent nodes of node <i>Vi </i>and <i>Wij </i>is the weight of the co­occurrence edge between nodes <i>Vi </i>and <i>Vj.</i></p><p>Since this step aims to induce the senses for the target word, only nodes referring to words in English can become root hubs. However, in order to use additional evidence from other lan­guages, we furthermore include in the computa­tion of PageRank co-occurrence edges from the target languages, as long as these occur in con­texts with 'safe', i.e. <i>monosemous, </i>translations of the target word. Given an English co-occurrence edge <i>(vs,i,vsj) </i>and translation edges <i>(vs,i,viti) </i>and <i>(vsj,vij) </i>to nodes in the target language <i>I,</i><i> </i>labeled with monosemous translations, we in­clude the co-occurrence edge <i>(viti,vij) </i>in the PageRank computation. For instance, animal and biotechnology are translated in German as Tier and Biotechnologie, both with edges labeled with the monosemous Pflanze: accordingly, we in­clude the edge (Tier, Biotechnologie) in the com­putation of <i>PR</i><i>(vi), </i>where <i>Vi </i>is either animal or biotechnology.</p><p>Finally, following Véronis (2004), a MST is built with the target word as its root and the root hubs of <i>G ml </i>forming its first level. By using a multilingual graph, we are able to obtain MSTs which contain translation nodes and edges.</p></subsection><subsection number="3.4" title="Multilingual Disambiguation"><p>Given a context <i>W </i>for the target word <i>w </i>in the source language, we use the MST to find the most relevant words in <i>W </i>for disambiguating <i>w. </i>We first map each content word <i>cw </i>g <i>W </i>to nodes in the MST. Since each word is dominated by ex­actly one hub, we can find the relevant nodes by computing the correct hub <i>disHub </i>(i.e. sense) and then only retain those nodes linked to <i>disHub. </i>Let <i>Wh </i>be the set of mapped content words dominated by hub <i>h. </i>Then, <i>disHub </i>can be found as:</p><doubt alpha="65.2" length="23" tooSmall="False" monospace="0.0">disHub =argmax -—d(cw)-</doubt><doubt alpha="57.9" length="19" tooSmall="False" monospace="0.0">hTm/dist{cw, h) + l</doubt><p>where <i>d(cw) </i>is a function which assigns a weight to <i>cw </i>according to its distance to <i>w, </i>i.e. the more words occur between <i>w </i>and <i>cw </i>within <i>W,</i><i> </i>the smaller the weight, and <i>dist(cw,h) </i>is given by the number of edges between <i>cw </i>and <i>h </i>in the MST. Finally, we collect the translation edges of the retained context nodes <i>WdisHub </i>and we sum the translation counts to rank each translation.</p></subsection></section><section number="4" title="Results and Analysis"><p><b>Experimental Setting. </b>We submitted two runs for the task (UHD-1 and UHD-2 henceforth). Since we were interested in assessing the impact of using different resources with our methodology, we automatically built multilingual graphs from different sentence-aligned corpora, i.e. Europarl (Koehn, 2005) for UHD-1, augmented with the JRC-Acquis corpus (Steinberger et al., 2006) for UHD-2<footnote anchor="1"/>. Both corpora were tagged and lemma-tized with LreeLagger (Schmid, 1994) and word aligned using GIZA++ (Och and Ney, 2003). For German, in order to avoid the sparseness deriving from the high productivity of compounds, we per­formed a morphological analysis using Morphisto (Zielinski et al., 2009).</p><p>To build the multilingual graph (Section 3.2), we used a minimum frequency threshold of 2 oc­currences for a word to be inserted as a node, and retained only those edges with a weight less or equal to 0.7. After constructing the multilin­gual graph, we additionally removed those trans­lations with a frequency count lower than 10 (7 in the case of German, due to the large amount of compounds). Finally, the translations gener­ated for the Best evaluation setting were ob­tained by applying the following rule onto the ranked answer translations: add translation <i>tri </i>while <i>count(tri) &gt; count(tri-\)/?&gt;, </i>where <i>i </i>is the <i>i-th </i>ranked translation.</p><p><b>Results and discussion. </b>The results for the Best and out-of-five (OOF) evaluations are pre­sented in Tables 1 and 2 respectively. Results are computed using the official scorer (Lefever and Hoste, 2010) and no post-processing is applied to the system's output, i.e. we do not back-off to the baseline most frequent translation in case the sys­tem fails to provide an answer for a test instance. For the sake of brevity, we present the results for UHD-1, since we found no statistically significant difference in the performance of the two systems (e.g. UHD-2 outperforms UHD-1 only by +0.7% on the best evaluation for French).</p><footnote label="1">As in the case of Europarl, only 1-to-l-aligned sentences were extracted.</footnote><page local="4" global="137"/><p>Lable 2: OOF results (UHD-1).</p><p>Overall, in the best evaluation our system ranked in the middle for those languages where the majority of systems participated - i.e. sec­ond and fourth out of 7 submissions for french and Spanish. When compared against the base­line, i.e. the most frequent translation found in Europarl, our method was able to achieve in the best evaluation a higher precision for italian and Spanish (+1.9% and +2.1%, respectively), whereas French and German lie near below the baseline scores (—0.5% and —1.0%, respectively). Lhe trade-off is a recall always below the base­line. In contrast, we beat the Mode precision base­line for all languages, i.e. up to +5.1% for Span­ish. Lhe fact that our system is strongly precision-oriented is additionally proved by a low perfor­mance in the OOF evaluation, where we always perform below the baseline (i.e. the five most fre­quent translations in Europarl).</p></section><section number="5" title="Conclusions"><p>We presented in this paper a graph-based system to perform CL-WSD. Key to our approach is the use of a co-occurrence graph built from multilin­gual parallel corpora, and the application of well-studied graph algorithms for monolingual WSD (Véronis, 2004; Agirre et al., 2006). Future work will concentrate on extensions of the algorithms, e.g. computing hubs in each language indepen­dently and combining them as a joint problem, as well as developing robust techniques for unsuper­vised tuning of the graph weights, given the obser­vation that the most frequent translations tend to receive too much weight and accordingly crowd out more appropriate translations. Finally, we plan to investigate the application of our approach directly to multilingual lexical resources such as PanDictionary (Mausam et al., 2009) and Babel-Net (Navigli and Ponzetto, 2010).</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Language</p></td><td class="cell"><p>P        R     ModeP ModeR</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>French German Italian Spanish</p></td><td class="cell"><p>20.22 16.21 12.20 9.32 15.94 12.78 20.48 16.33</p></td><td class="cell"><p>17.59 14.56 11.05 7.78 12.34 8.48 28.48 22.19</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lable 1: Best results (UHD-1).</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Language</p></td><td class="cell"><p>P        R     ModeP ModeR</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>French German Italian Spanish</p></td><td class="cell"><p>39.06 32.00 27.62 22.82 33.72 27.49 38.78 31.81</p></td><td class="cell"><p>37.00 26.79 25.68 21.16 27.54 21.81 40.68 32.38</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Eneko Agirre, David Martinez, Oier Lopez de Lacalle, and Aitor Soroa. 2006. Two graph-based algorithms for state-of-the-art WSD. In <i>Proc. of EMNLP-06, </i>pages 585-593.</p><p>Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine. <i>Computer Networks and ISDN Systems, </i>30(1-7): 107-117.</p><p>Christiane Fellbaum, editor. 1998. <i>WordNet: An Elec­tronic Database. </i>MIT Press, Cambridge, MA.</p><p>Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In <i>Proceedings of Machine Translation Summit </i><i>X.</i></p><p>Els Lefever and Véronique Hoste. 2010. SemEval-2010 Task 3: Cross-lingual Word Sense Disam­biguation. In <i>Proc. of SemEval-2010.</i></p><p>Mausam, Stephen Soderland, Oren Etzioni, Daniel Weld, Michael Skinner, and Jeff Bilmes. 2009. Compiling a massive, multilingual dictionary via probabilistic inference. In <i>Proc. of ACL-IJCNLP-09, </i>pages 262-270.</p><p>Roberto Navigli and Simone Paolo Ponzetto. 2010. BabelNet: Building a very large multilingual seman­tic network. In <i>Proc. of ACL-10.</i></p><p>Franz Josef Och and Hermann Ney. 2003. A sys­tematic comparison of various statistical alignment models. <i>Computational Linguistics, </i>29(1): 19-51.</p><p>Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In <i>Proceedings of the International Conference on New Methods in Lan­guage Processing (NeMLaP '94), </i>pages <i>44^9.</i></p><p>Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufi§, and Daniel Varga. 2006. The JRC-Acquis: A multilin­gual aligned parallel corpus with 20+ languages. In <i>Proc. ofLREC '06.</i></p><p>Jean Véronis. 2004. Hyperlex: lexical cartography for information retrieval. <i>Computer Speech &amp; Lan­guage, </i>18(3):223-252.</p><p>Piek Vossen, editor. 1998. <i>EuroWordNet: A Multi­lingual Database with Lexical Semantic Networks. </i>Kluwer, Dordrecht, The Netherlands.</p><p>Andrea Zielinski, Christian Simon, and Tilman Wittl. 2009. Morphisto: Service-oriented open source morphology for German. In <i>State of the Art in Com­putational Morphology, </i>volume 41 of <i>Communica­tions in Computer and Information Science, </i>pages 64-75. Springer.</p></references></body></article>