<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="656"/><title>Example-Based Paraphrasing for Improved Phrase-Based Statistical Machine Translation</title><pubinfo>Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,pages 656-666, MIT, Massachusetts, USA, 9-11 October 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Max" givenname="Aurélien"><org  name="CNRS" country="France"/></author></firstpageheader><frontmatter><p><b>Example-based Paraphrasing for Improved Phrase-Based Statistical Machine Translation</b></p><p><b>Aureiien Max</b></p><p>LIMSI-CNRS &amp; Univ. Paris Sud</p><p>Orsay, France aurelien.max@limsi.fr</p></frontmatter><abstract>In this article, an original view on how to improve phrase translation estimates is pro­posed. This proposal is grounded on two main ideas: first, that appropriate examples of a given phrase should participate more in build­ing its translation distribution; second, that paraphrases can be used to better estimate this distribution. Initial experiments provide ev­idence of the potential of our approach and its implementation for effectively improving translation performance. </abstract></header><body><section number="1" title="Introduction"><p>Phrase translation estimation in Statistical Phrase-based Translation (Koehn et al., 2003) is hampered by the availability of both too many and too few training instances. Recent results on tera-scale SMT (Lopez, 2008) show that access to many training examples<footnote anchor="1"/> can lead to significant improvements in translation quality. Also, providing indirect train­ing instances via synonyms or paraphrases for pre­viously unseen phrases can result in gains in trans­lation quality, which are more apparent when little training data is originally available (Callison-Burch et al., 2006; Marton et al., 2009; Mirkin et al., 2009; Aziz et al., 2010). Although there is a consensus on the importance of using more parallel data in SMT, it has never been formally shown that all additional training instances are actually useful in predicting contextually appropriate translation hypotheses.</p><p>!To be more accurate, works such as that of (Lopez, 2008) have recourse to random sampling to build models of a manage­able size in a reasonable amount of time.</p><p>Attempts at limiting training parallel sentences to those resembling test data through thematic adapta­tion (Hildebrand et al., 2005) indeed confirm that large quantities of training data cannot compen­sate for the requirement for contextually appropriate training instances. In fact, it is important that phrase translation models adequatly reflect contextual pref­erences for each phrase occurrence in a text. A vari­ety of recent works have used dynamically adapted translation models, where each phrase occurrence has its own translation distribution (Carpuat and Wu, 2007; Stroppa et al., 2007; Max et al., 2008; Gim­pel and Smith, 2008; Haque et al., 2009) derived from local contextual information in the training ex-amples.<footnote anchor="2"/> These approaches are supported by the study of (Wisniewski et al., 2010) which shows that phrase-based SMT systems are expressive enough to achieve very high translation performance and there­fore suggests a better scoring of phrases.</p><p>The apparent tradeoff between the number of training examples and their appropriateness in each indivual context naturally asks for means of increas­ing the number of appropriate examples. Exploiting comparable corpora for acquiring translation equiva­lents (Munteanu and Marcu, 2005; Abdul-Rauf and Schwenk, 2009) offers interesting prospects to this issue, but so far focus has not been so much on con­text appropriateness as on globally increasing the number of biphrase examples.</p><footnote label="2">The study of (Carpuat, 2009) shows that the one transla­tion per discourse hypothesis holds in some cases, but to our knowledge no SMT systems have attempted to exploit it yet. However, in our view, this finding does not contradict the need for estimating translation distributions at the individual phrase level, but they should be integrated as additional information.</footnote><page local="2" global="657"/><p>The approach we take in this article is motivated by the fact that natural language allows for multiple text views on a given content, and that if two phrases are good paraphrases in context, then considering appropriate training examples of one of the phrases could provide larger quantities of training data for translating the other. In other words, we hypothe­size that there may be more training data to learn a phrase's translations in a bilingual corpus than what SMT approaches typically use.</p><p>In contrast to previous attempts at using para­phrases to improve Statistical Machine Translation, which require external data in the form of additional parallel bilingual corpora (Callison-Burch et al., 2006), monolingual corpora (Marton et al., 2009), lexico-semantic resources (Mirkin et al., 2009; Aziz et al., 2010), or sub-sentential (Resnik et al., 2010) or sentential paraphrases of the input (Schroeder et al., 2009), the approach we take here can be endoge­nous with respect to the original training data. It also significantly departs from previous work in that paraphrasing is not simply considered as a way of finding alternative wordings that can be translated given the original training data for out-of-vocabulary phrases only (Callison-Burch et al., 2006; Marton but as a means to better estimate translations for <i>any </i>possible phrase. Also, as opposed to the work by (Schroeder et al., 2009; Onishi et al., 2010; Du et al., 2010), we do not encode paraphrases into input lattices to have them compete against each other to belong to the source sentential paraphrase that will lead to the highest scoring output sentence<footnote anchor="3"/>. Instead, we make use of all contextually appropriate para­phrases of a source phrase, which <i>collectively eval­uate the quality of each translation for that phrase.</i></p><doubt alpha="40.7" length="54" tooSmall="False" monospace="0.0">et al., 2009; Mirkin et al., 2009; Aziz et al., 2010),</doubt><p>This work can thus be seen as a contribution to­wards shifting from global phrase translation dis­tributions to contextual translation distributions for contextually equivalent source units. The remainder of this paper is organized as followed. In section 2 we review relevant previous works and discuss how they differ from our approach. Section 3 provides a description of the details of our approach. We de­scribe an experimental setup in section 4 and comment on our results. Finally, we discuss our future work in section 5.</p><footnote label="3">This highly depends on how well estimated translations for each independent paraphrase are.</footnote></section><section number="2" title="Relation to previous work"><subsection number="2.1" title="Contextual estimation of phrase translations"><p>In standard approaches to phrase-based SMT, evi­dence of a translation is accumulated uniformely ev­ery time it is found associated with a source phrase in the training corpus. In addition to the fact that errors in automatic word alignment and non literal translations often produce useless biphrases, this re­sults in rare but appropriate translations being very unlikely to be considered during decoding. Some approaches on source context modelling (Carpuat and Wu, 2007; Stroppa et al., 2007; Max et al., 2008; Haque et al., 2009) build classifiers offline for the phrases in a test set, so that context similarity can for example reinforce scores associated with rare but appropriate translations. However, heavy offline computation makes scaling to larger corpora an is­sue. Other approaches (Callison-Burch et al., 2005; Lopez, 2008) instead focus on accessing very large corpora. Indexing by suffix arrays is used to allow fast access to phrase instances in the corpus, and ran­dom sampling to avoid collecting the full set of ex­amples has been shown to perform well. However, these approaches consider all instances of a phrase as equivalent for the estimation of its translations.</p><p>These works converge on the need for accessing a sufficient number of examples that are relevant for any source phrase in context, fast enough to permit on-the-fly phrase table building. This paper pro­poses an intermediate step: the full set of phrase examples is found efficiently, and a measure of the adequacy of each example with a phrase in context provides evidence for its translation that depends on this value of adequacy. In this way, the translation associated with an example for a different sense of a polysemous word would in the best scenario only be considered marginally when computing the trans­lation distribution. As in most previous works, ad­equacy can be approximated by context similarity between phrase occurrences and training examples.</p><p>Ideally, one would stop extracting examples when enough appropriate examples have been found to es­timate a reliable translation distribution. (Callison-<page local="3" global="658"/></p><p>Figure 1: Effect of number of samples on translation quality (measured on German to English translation on Europarl data) reported by (Callison-Burch et al., 2005)</p><p>Burch et al., 2005) measured the impact on transla­tion quality of the sample size in random sampling of source phrase examples in the training corpus to estimate a phrase's translation probabilities. As Ta­ble 1 shows, quality (in terms of BLEU scores) al­most remains constant for samples of size 100 or more. This apparent confirmation of the efficiency of random sampling is backed up by the authors with the following possible explanations: 1) the most probable translations remain the same for dif­ferent sample sizes; 2) misestimated probabilities are ruled out by the target language model; and 3) longer or less frequent phrases, which are not affected by sampling, are preferred. However, as said previously, random sampling cannot guarantee that contextually-appropriate examples are selected. In fact, (Lopez, 2008) points out to using discrimi-natively trained models with contextual features of source phrases in conjunction with phrase sampling as an open problem. This work does not attempt to directly address it, but instead resorts to complete analysis of the training data to guarantee that all contextually-appropriate examples are considered.</p></subsection><subsection number="2.2" title="Using paraphrases for translating"><p>For some phrases, not enough examples can be found in the training corpus to estimate reliable translation probabilities in context. In such cases, one might be interested in finding more appropri­ate examples, which seems at first impossible us­ing the sole original bilingual corpus. We can in fact consider the set of source phrases that have similar translations in context. This set is roughly made up of a subset of what can be referred to as <i>paraphrases. </i>One possible approach to extract lo­cal (i.e. phrasal) paraphrases precisely exploits sim­ilarity <i>on the target side </i>in another language by ex­tracting source phrases that share common transla­tions (Bannard and Callison-Burch, 2005), but re­cent approaches have combined this approach with</p><p><b>Source phrase Paraphrases</b></p><p>Figure 2: Examples of paraphrases obtained by pivoting via French; values indicate <i>paraphrase probability </i>as de­fined in (Bannard and Callison-Burch, 2005).</p><p>similarity computation in the "source" (i.e. original) and Brockett, 2010). Figure 2 provides examples of English paraphrases obtained by automatically piv­oting via French. As can be seen, some examples would be clearly useful to better estimate transla­tions of the original source phrase: <i>(Balkan War &lt;-&gt; war in the Balkans) </i>are syntactic variants that can generally substitute with each other, <i>(Balkan War &lt;-&gt; Balkans war) </i>are character-level variants<footnote anchor="4"/>. Other examples, however, clearly illustrate the need for validation in context: <i>(Dalai Lama's &lt;-&gt; of the Dalai </i><i>Lama)</i><i> </i>require different syntactic contexts, and (I <i>don't see &lt;-&gt; I do not believe) </i>are only inter­changeable in specific semantic contexts.</p><doubt alpha="58.7" length="46" tooSmall="False" monospace="0.0">language (Callison-Burch, 2008; Max, 2008; Kok</doubt><p>Previous attempts at exploiting paraphrases in SMT have first concentrated on obtaining transla­tions for phrases absent from the training corpus</p><doubt alpha="54.0" length="50" tooSmall="False" monospace="0.0">(Callison-Burch et al., 2006; Marton et al., 2009;</doubt><p>Mirkin et al., 2009)<footnote anchor="5"/>, with modest gains in trans­lation performance as measured by automatic met­rics. (Callison-Burch et al., 2006) obtain para­phrases by pivoting via additional bilingual corpora and use the translations of known paraphrases to translate unseen phrases, which requires that the ad­ditional bilingual corpora contain the unseen source phrases and that some of the extracted paraphrases be present in the original corpus.  (Marton et al., 2009) proceed similarly but obtain their paraphrases from comparatively much larger monolingual cor­pora by following the <i>distributionality hypothesis.</i><page local="4" global="659"/><i> </i>In both cases, gains are only obtained in very spe­cific conditions where very few training data are available and where useful additional knowledge can be brought in from external resources. Furthermore, the described implementations do not consider ac­ceptability of the paraphrases in context, as their un­derlying hypothesis is that it might be more desir­able to translate some paraphrase than not to trans­late a given phrase.<footnote anchor="6"/> In contrast, the work by (Mirkin et al., 2009) attempts to model context when using replacements for words (synonyms or hypernyms).</p><footnote label="4">To our knowledge, most implementations of SMT decoders do not integrate flexible matching of phrases.</footnote><footnote label="5">The work by (Mirkin et al., 2009) in fact considers both paraphrases and entailed texts to increase the number of prop­erly translated texts.</footnote><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>sample size</p></td><td class="cell"><p>100</p></td><td class="cell"><p>500</p></td><td class="cell"><p>1K</p></td><td class="cell"><p>5K</p></td><td class="cell"><p>10k</p></td><td class="cell"><p>50k</p></td><td class="cell"><p>unlim.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BLEU score</p></td><td class="cell"><p>28.8</p></td><td class="cell"><p>28.8</p></td><td class="cell"><p>28.8</p></td><td class="cell"><p>28.9</p></td><td class="cell"><p>29.1</p></td><td class="cell"><p>28.9</p></td><td class="cell"><p>29.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Balkan War</i></p></td><td class="cell"><p><i>Balkan war </i>(0.25) <i>Balkans War </i>(0.125) <i>Balkans </i>(0.125) <i>Balkans war </i>(0.125) <i>war in the Balkans </i>(0.125) <i>Balkan conflict </i>(0.125)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>British forces</i></p></td><td class="cell"><p><i>British troops </i>(0.29) <i>British armedforces </i>(0.19)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Czech president</i></p></td><td class="cell"><p><i>President of the Czech Republic </i>(0.5)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Dalai Lama's</i></p></td><td class="cell"><p><i>of the Dalai Lama </i>(0.27)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>I don't see</i></p></td><td class="cell"><p><i>I do not believe </i>(0.18) <i>I do not think </i>(0.18) <i>I do not see </i>(0.15)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>The natural next step that we take here is to exploit the complementarity of the original bilin­gual training corpus for finding paraphrases and the monolingual (source) side of the same corpus for validating them in context. Furthermore, our fo­cus here is not on paraphrasing unseen phrases<footnote anchor="7"/>, but possibly any phrase, or any phrase seen less than a given number of times, or any types of difficult-to-translate phrases (Mohit and Hwa, 2007).</p><p>The recent work of (Resnik et al., 2010) uses crowdsourcing to obtain paraphrases for source phrases corresponding to mistranslated target phrases. The spotting of the incorrect target phrases and the paraphrasing of the source phrases can be automated. Promising oracle figures are obtained, validating the claim that some variations of the input sentence might be more easily translated than oth­ers by a given system. Paraphrases have also been used to represent alternative inputs encoded in lat­tices using existing (Schroeder et al., 2009) or au­tomatically built paraphrases (Onishi et al., 2010; Du et al., 2010). In this scenario, paraphrases are in fact competing with each other, whereas in our pro­posal paraphrases <i>collectively participate in evalu­ating the quality </i><i>ofeach</i><i> translation for a source phrase. </i>We believe that if two phrases are indeed paraphrases in context, then their respective set of translations are both relevant to translate the two phrases. The target language model nevertheless still has an important role to play to select appropriate translations among semantically-compatible translations (i.e., target side paraphrases) in the spe­cific context of a generated target hypothesis.</p><footnote label="6">The default strategy for most decoders is to copy out-of-vocabulary tokens into the final text.</footnote><footnote label="7">Doing it in conjunction with our approach for improving the translation of known phrases is part of our future work.</footnote><p>Lastly, automatic sentential paraphrasing has also been used in SMT to build alternative reference translations for parameter optimization (Madnani et al., 2008) and to build alternative training cor­pora (Bond et al., 2008).</p><p><b>3   Towards better exploitation of training corpora in phrase-based SMT</b></p><p>In typical phrase-based SMT settings (Koehn et al., 2003), words from the source side of the corpus are first aligned to words on the target side and biphrases are extracted from each training sentence using some heuristics on the word alignments. A source phrase f in a sentence being translated may therefore be aligned to a variety of target phrases. In the example on Figure 3, f is aligned some num­ber of times in the training corpus to target phrases <i>e1, </i>e2, <i>e3 </i>and e5. Using the number of times f is paired with some target phrase e^, <i>count(f, </i>e^), rela­tive frequency estimation can be used to compute the probability of translation   given source phrase f :</p><p><i>count(f,e</i><i>i)</i></p><doubt alpha="31.8" length="22" tooSmall="False" monospace="0.0">Vrel(ei\f) = =-——- (1)</doubt><doubt alpha="60.0" length="15" tooSmall="False" monospace="0.0">}2jcountf , ej)</doubt><p>This value, together with other estimates of how appropriate a translation pair (f, e^) is, are recorded in a phrase table, which typically discards all con­textual information.<footnote anchor="8"/> Therefore, the translation dis­tribution of some phrase is globally estimated from a training corpus independently of the actual context of that phrase.<footnote anchor="9"/> On Figure 3, phrase <i>f </i>has at least two distinct senses: one represented by set <i>E</i>, which in our example corresponds to the appropriate sense for a particular occurrence of <i>f </i>in a test sentence; and one which corresponds to translation e5. A typ­ical problem, due to the lack of context modeling,<page local="5" global="660"/></p><footnote label="8">See (Carpuat and Wu, 2007; Stroppa et al., 2007; Max et al., 2008; Gimpel and Smith, 2008; Haque et al., 2009) for no­table exceptions.</footnote><footnote label="9">Context is in fact taken into account to some extent by the target language model, which should score higher translations that are more appropriate given a target translation hypothesis being built. In fact, in this work we consider the target language model as the main source of information for selecting among acceptable target phrases (target language paraphrases).</footnote><p><b>un bon avocat un avocat performant un avocat brillant</b> <b>un avocat succulent</b></p><p>Figure 3: Example of possible source equivalents and translations for phrase occurrence <i>f "un bon avocat" </i>in the sentence <i>"L'embauche d'un bon avocat est cruciale quelle que soit l'activité" ("Hiring a good lawyer is cru­cial to any business"). </i>Set E represents target phrase types that are acceptable translations given the particu­lar context of f, and set F represents source phrase types that can be in a paraphrasing relation to f depending on the context they appear in.</p><p>is that in situations such as Vei G E, <i>count(f, e5) </i>» <i>count(f, e</i><i>i), </i>it is very unlikely that a correct trans­lation will be selected during decoding against the incorrect but much more frequent one. Taking an extreme view on this issue, it is in fact desirable that when estimating phrase translation probabilities for a phrase <i>f </i>, translations of incompatible senses be not considered.<footnote anchor="10"/> Of course, this raises the diffi­cult issue of sense clustering of phrases. We propose here an intermediary solution, which consists in con­sidering each occurrence in the training corpus as counting a number of times that depends on its con­textual similarity with the occurrence of <i>f </i>from the test file, through the following additional translation model :</p><doubt alpha="66.7" length="12" tooSmall="False" monospace="0.0">Pcant(ei\f)=</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">E</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">(fk,et)</doubt><doubt alpha="60.0" length="20" tooSmall="False" monospace="0.0">simcont(C(f),C(fk ))</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">(fk,ej&gt;</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(2)</doubt><p>where <i>f </i>is some source phrase to translate and <i>f</i><i>k </i>an example of f in the training corpus, (fk, ei) is a biphrase from the training corpus, C(f ) the context of some source phrase, <i>C(f</i><i>k)</i><i> </i>the context of a par­ticular example of f in the training corpus, simphra function indicating the contextual similarity be­tween two phrase contexts, and <i>e</i><i>j </i>is any possible translation of <i>f.</i><i></i></p><footnote label="10">Put differently, is it more acceptable to copy a source word in the target hypothesis or to incorrectly translate it when the confidence about its being incorrect is high?</footnote><p>The problem of modeling phrase translation is however not limited to inappropriate training exam­ples. For various reasons, legitimate occurrences of source phrases may not be considered when building a phrase's translation distribution. We describe those cases by considering the possible source phrases <i>pi</i>from Figure 3:</p><p>• pi's only translation, ei, is a common transla­tion with f ; each contextually-appropriate ex­ample of <i>p1 </i>should reinforce the probability of <i>e</i>i being a translation for <i>f.</i></p><p>• Contextually-appropriate examples of p2 can reinforce <i>e3. </i>Translation <i>e6 </i>should correspond to contextually-inappropriate examples of p2, so <i>e6 </i>should not be considered as a new po­tential translation for <i>f.</i></p><p>• Contrarily to the examples of <i>p</i>2 translating as <i>e6, </i>examples of <i>p3 </i>translating as e4 are much more likely of being contextually-appropriate with <i>f,</i><i> </i>meaning that <i>f</i><i> </i>could be substituted with most <i>p3 </i>examples. Therefore, e4, which was not initially considered as a possible trans­lation of <i>f,</i><i> </i>could now be considered as such.</p><p>• <i>p</i>4 shares a translation with <i>f,</i><i> </i><i>e</i>2, but this is due to the polysemous nature of this transla­tion. Again, all examples of p4 should be found contextually-inappropriate with <i>f,</i><i> </i>and their translations should not be considered when es­timating the translations of <i>f.</i></p><p>• Lastly, the case of the common translation e5between f and <i>p5 </i>illustrates a consequence of the polysemous nature of the source phrase cor­responding to word sequence f : translations corresponding to other senses of <i>f </i>should not get reinforced by paraphrase examples such as those of <i>p5 </i>as these examples should be found contextually-inappropriate with <i>f.</i></p><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b><i>S</i></b></p><p><b>1</b></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a good lawyer</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a performing lawyer</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a brillant attorney</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>-e4</b></p></td><td class="cell"><p><b>a good attorney</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a tasty avocado</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a shiny avocado</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>a lawyer who sings</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="6" global="661"/><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">E</doubt><p>We build a separate translation model for transla­tions estimated through paraphrases, defined as fol­lows:</p><doubt alpha="66.7" length="12" tooSmall="False" monospace="0.0">ppara(ei\f)—</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">(Pk,ei&gt;</doubt><doubt alpha="100.0" length="7" tooSmall="False" monospace="0.0">simpara</doubt><doubt alpha="44.4" length="9" tooSmall="False" monospace="0.0">(Pk ,ej &gt;</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(3)</doubt><p>where <i>pk </i>is a paraphrase of <i>f</i>, <i>(pk, </i>ei) is a biphrase from the training corpus such that <i>e</i><i>i </i>is also a transla­tion of <i>f</i>, <i>C</i><i> </i><i>(f</i><i> </i><i>)</i><i> </i>the context of a given source phrase for which we are estimating the translation distribu­tion, <i>C (pk) </i>the context of a particular example of <i>pk</i>in the training corpus, <i>simpara </i>a function indicat­ing the contextual similarity between a phrase con­text and a paraphrase context, and <i>e</i><i>j </i>is any possible translation of <i>f.</i></p><p>Several requirements can be drawn from the pre­vious description:</p><p>1. <b>List of potential paraphrases: </b>some mech­anism for finding potential paraphrases for source phrases is required, and several such mechanisms could be combined. Pivoting via bilingual corpora, a natural strategy given the issue at hand, is just one among many different proposed strategies (Madnani and Dorr, 2010).</p><p>2. <b>Contextual similarity measure: </b>a similarity measure between the contexts of two phrases or two potential local paraphrases is required. This automatic measure should ideally be able to model not only syntactic but also semantic and pragmatic information.</p><p>3. <b>Robust translation evaluation: </b>our ap­proach is designed to reinforce estimates for any contextually-appropriate translations of a phrase, as shown by set E on Figure 3. It is therefore important to have some means of ac­cepting them as subparts of valid translations. Robustness in Machine Translation evaluation is an active domain, and potential candidates include using BLEU-like metrics with multiple references, Human-targeted Translation Error Rate (Snover et al., 2006) and the use of para­phrases for reference translations (Kauchak and Barzilay, 2006).</p><p>In this paper, we want to evaluate whether an en­dogenous approach for finding paraphrases can lead to some improvement in translation performance. Note that we will not consider in this initial work the possibility of adding new translations to phrases (such as <i>e</i>4 for <i>f </i>on Figure 3) as it adds complexity and should be investigated when the other simpler cases can be handled successfully.</p><p>In the following section, we describe experiments in which the original bilingual corpus is the only re­source used to find potential paraphrases and to esti­mate phrase translations in context. We chose a very simple measure of similarity, and let to our future work the task of improving context modeling. As regards evaluation, we will resort to various ways to measure the impact of our implementation on trans­lation performance.</p></subsection></section><section number="4" title="Experiments and results"><subsection number="4.1" title="Data and baseline SMT systems"><p>We have conducted our experiments using the Moses<footnote anchor="11"/> package to build state-of-the-art phrase-based SMT systems for phrases of up to 5 tokens, using standard parameters and MERT for optimizing model weights. We used a subpart of the Europarl corpus<footnote anchor="12"/> in French and English as our training cor­pus and built baseline MOSES systems <b>(bsl) </b>in both directions. The target side of the training corpus was used to train 3-gram target language model with modified Kneser-Ney smoothing. Held-out datasets were used for development and testing. The charac­teristics of all corpora are described in Figure 4.</p></subsection><subsection number="4.2" title="Example-based Paraphrasing SMT systems"><p>We also built systems that exploit phrase and para­phrase context under the form of two additional models <i>p</i><i>cont</i><i> </i>and <i>ppara </i>described in section 3. These</p><p>http://www.statmt.org/moses <footnote anchor="2"/>http://www.statmt.org/europarl<page local="7" global="662"/></p><table caption="Figure 4: Statistics of the corpora used." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>train</p></td><td class="cell"><p>dev.</p></td><td class="cell"><p>test</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p># sent.</p></td><td class="cell"><p>#tok.</p></td><td class="cell"><p># sent.</p></td><td class="cell"><p>#tok.</p></td><td class="cell"><p># sent.</p></td><td class="cell"><p>#tok.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>en</p></td><td class="cell"><p>318K</p></td><td class="cell"><p>9.1M</p></td><td class="cell"><p>500</p></td><td class="cell"><p>14,0K</p></td><td class="cell"><p>500</p></td><td class="cell"><p>13,6K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>fr</p></td><td class="cell"><p>318K</p></td><td class="cell"><p>10.3M</p></td><td class="cell"><p>500</p></td><td class="cell"><p>16,1K</p></td><td class="cell"><p>500</p></td><td class="cell"><p>15,7k</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Figure 5: Statistics on the size and the number of entries of the phrase tables filtered on the development set.</p><p>models are added to the list of models used to eval­uate the various translations of a phrase in the ap­propriate phrase tables, and are optimized with the other models by standard MERT.</p><p>In order to model context, we modified the source texts so that each phrase becomes unique in the phrase table, i.e. it has its own translation distribu­tion. This is done (as in other works (Carpuat and Wu, 2007; Stroppa et al., 2007)) by transforming each token into a unique token, e.g. <i>token — to-ken@337. </i>This therefore leads to a significant in­crease in the size of the phrase table, as illustrated on Figure 5, as all occurrences for the same phrase are not factored anymore.<footnote anchor="13"/></p><p>We chose a very simple initial definition of con­text similarity based on the presence of common <i>n</i>-grams in the immediate vicinity of two phrases. Let <i>length</i><i>ieft </i>(resp. <i>length</i><i>right) </i>be the length of the longest common <i>n</i>-gram in the immediate vicin­ity on the left (resp. right) of two phrases in context (C<i>(f</i><i> </i><i>) </i>and <i>C</i>(fi)). For instance, given the two fol­lowing contexts (phrases under focus are in bold and common <i>n</i>-grams are underlined):</p><p>1. <i>the commission accepts the substance of the <b>amendments@11257 proposed@11258 by@11259 </b><u>the committee on</u> fisheries...</i></p><p>2. <i>this is why we shall support all of the <b>amend­ments put forward by </b><u>the committee on</u> agri­culture and rural development ...</i></p><p><i>length</i><i>left </i><i>— </i>2 and <i>lengthright </i><i>— </i>3. We further define <i>length </i>as:</p><footnote label="13">These volumes of data and our available hardware facilities for these experiments led us to initially limit the size of our data sets. We will discuss in section 5 how we intend to address this limitation in our future work.</footnote><p><i>length —</i></p><p><i>{ length</i><i>ieft </i><i>+ </i><i>length</i><i>right </i>if <i>length</i><i>ieft </i><i>&gt; </i><i>0 </i>and <i>lengthright &gt; </i><i>0 0 </i>otherwise</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(4)</doubt><p>We can now define the two similarity functions used in Equations 2 and 3 that we used for our ex­periments:</p><doubt alpha="63.2" length="19" tooSmall="False" monospace="0.0">simcont(C(f),C(fi))</doubt><doubt alpha="63.2" length="19" tooSmall="False" monospace="0.0">simpara(C(f),C(pi))</doubt><doubt alpha="50.0" length="26" tooSmall="False" monospace="0.0">(1+length)"(5)(length)ß(6)</doubt><p>The rationale for these functions is the follow­ing. Exact phrase examples add at least a transla­tion count of 1, i.e. their translation is always taken into account to estimate <i>pcont. </i>Paraphrase exam­ples add a translation count of 0 if <i>length </i><i>—</i><i> </i><i>0,</i><i> </i>i.e. their translation is not taken into account at all if surrounding <i>n</i>-gram similarity is too low. We used <i>a </i><i>— </i><i>ß </i>— 1.5. Algorithm 1 describes how the two models are estimated from the training data.</p><p><b>foreach </b><i>phrase </i><i>f </i><i>in training file </i><b>do </b>extract <i>C</i><i> </i><i>(f</i><i> </i>); /* phrase count */</p><p><b>foreach </b><i>unique phrase </i><i>f</i><i>i </i><i>in </i><i>test(f )</i><i>) </i><b>do </b>extract <i>C </i>(fi);</p><p>compute <i>sim</i><i>cont</i><i>(C</i><i>(f</i><i>),C</i><i>(f</i><i>i));</i> /* paraphrase count */</p><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">end</doubt><p><b>foreach </b><i>phrase </i><i>pi</i><i> </i><i>in </i><i>para(f ) </i><b>do</b></p><p><b>foreach </b><i>unique phrase </i><i>f</i><i>j </i><i>in </i><i>test(pi) </i><b>do </b>extract <i>C</i><i> </i><i>(f</i><i>j </i>);</p><p>compute <i>simpara(C</i><i>(f</i><i> ), C</i><i>(f</i><i>j</i><i>))</i><i>;</i> estimate <i>p</i><i>cont </i>and para;</p><p><b>Algorithm 1: </b>Model estimation for <i>pcont</i><i> </i>and <i>p</i><i>para. </i>Function <i>test(f)</i><i> </i>returns all unique phrases corresponding to phrase <i>f </i>from the test file. Function <i>para(f) </i>return all phrases for which <i>f </i>is a known paraphrase.</p><p>We implemented the following strategy to find paraphrases for phrases in the test file. We extract all<page local="8" global="663"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>phrase table size num. entries</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>en—fr</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>baseline</b></p></td><td class="cell"><p>240Mb</p></td><td class="cell"><p>2.4M</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>our systems</b></p></td><td class="cell"><p>5.0Gb</p></td><td class="cell"><p>37.5M</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>fr—en</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>baseline</b></p></td><td class="cell"><p>193Mb</p></td><td class="cell"><p>1.9M</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>our systems</b></p></td><td class="cell"><p>4.0Gb</p></td><td class="cell"><p>30.2M</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Figure 6: Examples of paraphrases in context from the development file. The input sentence (IS) contains a source phrase of interest (in bold), the paraphrase example (PE) contains a paraphrase of that source phrase (in bold) for which a paraphrase translation (PT) is known.</p><p>paraphrases <i>p </i>for a phrase <i>f </i>by pivot: all target lan­guage phrases <i>e </i>aligned to <i>f </i>are first extracted, and all source language phrases <i>p </i>aligned to <i>e </i>are ex­tracted. The following constraints are then applied to define which paraphrases are kept:</p><p>• string <i>p </i>is not included in string <i>f </i>and vice versa (in order to minimize the impact of align­ment errors in the training corpus);</p><p>• the paraphrasing probability is greater than a fixed threshold: <i>para(f,p) &gt; </i><i>10-2,</i><i> </i>where <i>para(f,p) </i><i>— </i>E<i>ep(e\f)p(p\e) </i>(Bannard and</p><doubt alpha="59.1" length="22" tooSmall="False" monospace="0.0">Callison-Burch, 2005);</doubt><p>• the number of occurrences of phrase <i>f </i>and paraphrase <i>p </i>are equal or less than indepen­dent thresholds: <i>numOccs(f</i><i>) </i><i>&lt; </i><i>100 </i>and <i>numOccs(p) &lt; </i>1000.<footnote anchor="14"/></p><p>Figure 6 shows examples of paraphrases in con­text with high similarity with some original phrase, and Figure 7 provides various statistics on the para­phrases extracted on the test file.</p></subsection><subsection number="4.3" title="Results and analysis"><p>Automatic evaluation results are reported in Table 8 for various configurations. We also wanted to focus our measures on content words, which are known</p><footnote label="14">The first threshold value was chosen as (Callison-Burch et al., 2005) report it to be an optimal sample size for estimating phrase translation probabilities. The relatively low value for the second threshold was selected to reduce computation time.</footnote><p>Figure 7: Statistics on numbers of phrases, numbers of paraphrased phrases and numbers of paraphrases per phrase length.</p><p>to be important as regards information content in translation. We applied the contrastive lexical eval­uation (CLE) methodology described in (Max et al., 2010), which indicates how many times source words grouped into user-defined classes were cor­rectly translated or not across systems. These addi­tional results are reported on Figure 9.</p><p>On English to French translation, both additional features lead to improvements over the baseline with all metrics, including CLE, and their combi­nation shows a strong improvement in TER (-1.55). CLE on content words reveals that the <b>para </b>feature seems particularly effective in reducing the number of words in all categories that only the baseline sys­tem translated correctly.</p><p>Results on French to English translation are less positive: neither <b>cont </b>nor <b>para </b>alone improve over the baseline with any metrics. However, their com­bination improves over the baseline with all met­rics except BLEU, including a reduction of -1.07 in TER. Detailed analysis of CLE results shows that the translation of adjectives and nouns benefited more from using our two additional models.<page local="9" global="664"/> Verbs, whose translation improved slightly, are strongly in­flected in French, so finding examples for a given form is more difficult than for less inflected word categories, as is finding paraphrases with the appro­priate inflection. Also, pivoting via English is one reason why paraphrases obtained via a low-inflected language can be of varying quality. Furthermore, the simplicity of our context modeling may have been ineffective in filtering out some bad examples. Over­all, <b>para </b>was more effective with the low-inflected English as the source language, improving over the baseline with all metrics.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Left context</p></td><td class="cell"><p>phrase/paraphrase</p></td><td class="cell"><p>Right context</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>IS#1</p></td><td class="cell"><p><i>at the moment it is</i></p></td><td class="cell"><p><b>up to each</b></p></td><td class="cell"><p><i>member state to decide, and practice dif­fers considerably from country to country</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PE#1</p></td><td class="cell"><p><i>... as regards the terminal portion in the cycle of nuclear fuel, it is</i></p></td><td class="cell"><p><b>the responsability of each</b></p></td><td class="cell"><p><i>member state to define its own policy .</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PT#1</p></td><td class="cell"><p></p></td><td class="cell"><p>la responsabilite de chaque</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>IS#2</p></td><td class="cell"><p><i>that is why ifind it</i></p></td><td class="cell"><p><b>extremely regrettable that</b></p></td><td class="cell"><p><i>the amendment on harmonising the re-registration of cars that have been involved in accidents ...</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PE#2</p></td><td class="cell"><p><i>for all these reasons and given your most excellent statement, ifind it</i></p></td><td class="cell"><p><b>a pity that</b></p></td><td class="cell"><p><i>the new legal base for the daphne pro­gramme is so restrictive ...</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PT#2</p></td><td class="cell"><p></p></td><td class="cell"><p>dommage que</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>phrase length</b></p></td><td class="cell"><p><b># phrases</b></p></td><td class="cell"><p><b># paraphrased</b></p></td><td class="cell"><p><b># paraphrases</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>en</p></td><td class="cell"><p>fr</p></td><td class="cell"><p>en</p></td><td class="cell"><p>fr</p></td><td class="cell"><p>en</p></td><td class="cell"><p>fr</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>13,620</p></td><td class="cell"><p>15,707</p></td><td class="cell"><p>458</p></td><td class="cell"><p>725</p></td><td class="cell"><p>1,824</p></td><td class="cell"><p>2,684</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>13,120</p></td><td class="cell"><p>15,207</p></td><td class="cell"><p>4,127</p></td><td class="cell"><p>4,481</p></td><td class="cell"><p>18,871</p></td><td class="cell"><p>19,700</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p>12,620</p></td><td class="cell"><p>14,707</p></td><td class="cell"><p>4,782</p></td><td class="cell"><p>5,715</p></td><td class="cell"><p>24,111</p></td><td class="cell"><p>27,377</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p>12,120</p></td><td class="cell"><p>14,208</p></td><td class="cell"><p>2,859</p></td><td class="cell"><p>4,078</p></td><td class="cell"><p>15,071</p></td><td class="cell"><p>20,345</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p>11,623</p></td><td class="cell"><p>13,711</p></td><td class="cell"><p>1,171</p></td><td class="cell"><p>2,275</p></td><td class="cell"><p>6,077</p></td><td class="cell"><p>12,132</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>These results confirm that translation perfor­mance can be improved by exploiting context and paraphrases in the original training corpus only. We next attempted to measure whether some improve­ment in the quality of the paraphrases used would have some measurable impact on translation perfor­mance. To this end, we devised a semi-oracle ex­periment in the following way: the source and target test files were automatically aligned, and for each source phrase possible target phrases (i.e., reference translations) were extracted, and used as pivots to extract potential paraphrases, which were then fil­tered with the same constraints as previously. In this way, we exploit the information that paraphrases can at least produce the desired translation, but they may also propose other incorrect translations and/or be present in very few examples. Results appear in the <b>inf </b>rows of Tables 8 and 9. We obtain the most important improvement over the baseline in BLEU for the two language pairs (resp. +0.99 and +0.44), though the results for the other metrics for French to English translation are more difficult to interpret. For this language pair, possible reasons include again that the pivot language may not be appropriate, and also that the limitation to a sin­gle pivot<footnote anchor="15"/> may not have produced more monolin­gual variation that might have proved useful. CLE on English to French, however, reveals significant gains with a relative improvement over the baseline of +116 content words. Under this condition, this result shows that the higher the quality of the para­phrases used, the more translation quality can be im-</p><footnote label="15">Several pivot phrases may in fact have been automatically extracted for a given phrase, some of which being possible bad candidates.</footnote><p>Figure 8: Automatic scores for the Moses baseline sys­tems <b>(bsl), </b>systems additionnally using the contextual feature <b>(+cont), </b>systems additionnally using the para­phrasing feature <b>(+para), </b>systems using both features <b>(all), </b>and pivot-informed systems <b>(inf).</b></p><p>Figure 9: Contrastive lexical evaluation results per part-of-speech measured on the test file. '-'(resp. '+') rows indicate the number of source words that only <b>bsl </b>(resp. the compared system) correctly translated.</p><p>proved, which is in line with works that make use of human-made paraphrases to improve translation quality (Schroeder et al., 2009; Resnik et al., 2010).</p><p>Table 10 presents a typology ofparaphrases found in our development set and classifies the impact of using them for phrase translation estimation. As can be seen, more work is needed to better understand the characteristics of the phrases that should be para­phrased and of their paraphrases.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>BLEU            NIST             TER METEOR</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>en^fr</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>bsl</b></p></td><td class="cell"><p>30.28 -</p></td><td class="cell"><p>6.66 -</p></td><td class="cell"><p>57.86 -</p></td><td class="cell"><p>54.79 -</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+cont</b></p></td><td class="cell"><p><b>31.11 </b>+0.83</p></td><td class="cell"><p>6.77 +0.11</p></td><td class="cell"><p>57.24 -0.62</p></td><td class="cell"><p>55.22 +0.43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+para</b></p></td><td class="cell"><p>30.97 +0.69</p></td><td class="cell"><p>6.74 +0.08</p></td><td class="cell"><p>57.38 -0.48</p></td><td class="cell"><p><b>55.39 </b>+0.60</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>all</b></p></td><td class="cell"><p>30.93 +0.65</p></td><td class="cell"><p><b>6.84 </b>+0.18</p></td><td class="cell"><p><b>56.31 </b>-1.55</p></td><td class="cell"><p>55.28 +0.49</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>inf</b></p></td><td class="cell"><p>31.27 +0.99</p></td><td class="cell"><p>6.78 +0.12</p></td><td class="cell"><p>57.22 -0.64</p></td><td class="cell"><p>55.80 +1.01</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>fr^en</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>bsl</b></p></td><td class="cell"><p><b>29.90 </b>-</p></td><td class="cell"><p>6.90 -</p></td><td class="cell"><p>54.64 -</p></td><td class="cell"><p>61.36 -</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+cont</b></p></td><td class="cell"><p>29.56 -0.34</p></td><td class="cell"><p>6.89 -0.01</p></td><td class="cell"><p>54.95 +0.31</p></td><td class="cell"><p>60.98 -0.38</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+para</b></p></td><td class="cell"><p>29.70 -0.20</p></td><td class="cell"><p>6.92 +0.02</p></td><td class="cell"><p>54.64 +0.00</p></td><td class="cell"><p>61.10 -0.26</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>all</b></p></td><td class="cell"><p>29.75 -0.15</p></td><td class="cell"><p><b>7.03 </b>+0.13</p></td><td class="cell"><p><b>53.57 </b>-1.07</p></td><td class="cell"><p><b>61.63 </b>+0.27</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>inf</b></p></td><td class="cell"><p>30.34 +0.44</p></td><td class="cell"><p>6.93 +0.03</p></td><td class="cell"><p>54.90 +0.26</p></td><td class="cell"><p>60.99 -0.37</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Adj Adv Noun Verb </b>J]</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>en^fr</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+cont</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>74</p></td><td class="cell"><p>28</p></td><td class="cell"><p>113</p></td><td class="cell"><p>60</p></td><td class="cell"><p>275</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>55</p></td><td class="cell"><p>35</p></td><td class="cell"><p>114</p></td><td class="cell"><p>85</p></td><td class="cell"><p>289 +14</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+para</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>62</p></td><td class="cell"><p>12</p></td><td class="cell"><p>82</p></td><td class="cell"><p>46</p></td><td class="cell"><p>202</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>58</p></td><td class="cell"><p>32</p></td><td class="cell"><p>111</p></td><td class="cell"><p>78</p></td><td class="cell"><p>279 +77</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>all</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>72</p></td><td class="cell"><p>25</p></td><td class="cell"><p>91</p></td><td class="cell"><p>72</p></td><td class="cell"><p>260</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>50</p></td><td class="cell"><p>37</p></td><td class="cell"><p>118</p></td><td class="cell"><p>97</p></td><td class="cell"><p>302 +42</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>inf</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>58</p></td><td class="cell"><p>20</p></td><td class="cell"><p>108</p></td><td class="cell"><p>56</p></td><td class="cell"><p>242</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>65</p></td><td class="cell"><p>43</p></td><td class="cell"><p>147</p></td><td class="cell"><p>103</p></td><td class="cell"><p>358 +116</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>fr^en</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+cont</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>30</p></td><td class="cell"><p>16</p></td><td class="cell"><p>80</p></td><td class="cell"><p>69</p></td><td class="cell"><p>195</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>15</p></td><td class="cell"><p>21</p></td><td class="cell"><p>69</p></td><td class="cell"><p>46</p></td><td class="cell"><p>151 -44</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>+para</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>32</p></td><td class="cell"><p>19</p></td><td class="cell"><p>72</p></td><td class="cell"><p>60</p></td><td class="cell"><p>183</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>12</p></td><td class="cell"><p>18</p></td><td class="cell"><p>65</p></td><td class="cell"><p>43</p></td><td class="cell"><p>138 -45</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>all</b></p></td><td class="cell"><p></p></td><td class="cell"><p>21</p></td><td class="cell"><p>18</p></td><td class="cell"><p>67</p></td><td class="cell"><p>61</p></td><td class="cell"><p>167</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>30</p></td><td class="cell"><p>18</p></td><td class="cell"><p>94</p></td><td class="cell"><p>48</p></td><td class="cell"><p>190 +23</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>inf</b></p></td><td class="cell"><p>-</p></td><td class="cell"><p>38</p></td><td class="cell"><p>21</p></td><td class="cell"><p>83</p></td><td class="cell"><p>66</p></td><td class="cell"><p>208</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>+</p></td><td class="cell"><p>31</p></td><td class="cell"><p>23</p></td><td class="cell"><p>106</p></td><td class="cell"><p>57</p></td><td class="cell"><p>217 +9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="10" global="665"/><p>Figure 10: Main types of paraphrase pairs found in our dev. and training corpora. Pairs shown have <i>l</i><i>ength &gt; </i>0.</p></subsection></section><section number="5" title="Conclusion and future work"><p>We have introduced an original way of exploiting both context and paraphrasing for the estimation of phrase translations in phrase-based SMT. To our knowledge, this is the first time that paraphrases ac­quired in an endogenous manner have been shown to improve translation performance, which shows that bilingual corpora can be better exploited than they typically are. Our experiments further showed the promises of our approach when paraphrases of higher quality are available.</p><p>In the light of our results and our initial typology of paraphrases presented on Figure 10, as well as previous work on paraphrasing for SMT, the diffi­cult question of what units should be paraphrased for what success should be addressed, taking into ac­count parameters such as language pairs, quantity of training data and availability of external resources.</p><p>Our future work includes three main areas: first, we want to improve the modeling of context, by no­tably working on techniques inspired from Informa­tion Retrieval to quickly access contextually-similar examples of source phrases in bilingual corpora. Such <i>contextual sampling </i>on large bilingual corpora for phrases and their paraphrases, which could inte­grate more complex linguistic information, will al­low us to assess our approach on more challenging conditions. This would also allow us to build con­textual models on-the-fly, and experiment with us­ing lattices to encode contextually estimated para­phrases. Second, we will combine paraphrases ob­tained via different techniques and resources, which will allow us to also learn translation distributions for phrases absent from the original corpus. Lastly, we want to also exploit paraphrases for the <i>addi­tional </i>translations that they propose (such as e4 on Figure 3) and that would be contextually similar in the target language to other existing translations of a given phrase or that could even represent a new sense of the original phrase.</p></section><section title="Acknowledgements"><p>This work was partly supported by ANR project Trace (ANR-09-CORD-023). The author would like to thank the anonymous reviewers for their helpful questions and comments.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Type</b></p></td><td class="cell"><p><b>Impact</b></p></td><td class="cell"><p><b>Examples</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Morphological variants</p></td><td class="cell"><p></p></td><td class="cell"><p>(yugoslav republic &lt;-&gt; yugoslavian republic), (go far &lt;-&gt; goes far)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Synonymy</p></td><td class="cell"><p>+</p></td><td class="cell"><p>(duties &lt;-&gt; obligations), (to look into &lt;-&gt; to study)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Grammatical word substitution</p></td><td class="cell"><p></p></td><td class="cell"><p>(states in the &lt;-&gt; the states of the), (amendments by &lt;-&gt; amendments to)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Word deletion or insertion</p></td><td class="cell"><p></p></td><td class="cell"><p>(first reading, the —&gt; first reading the), (amendments by &lt;-&gt; amendments proposed by)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Syntactic rewritings</p></td><td class="cell"><p>+</p></td><td class="cell"><p>(approval of the majority &lt;-&gt; majority support), (capacity of the euro­pean union &lt;-&gt; european union's ability)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Phrasal idiomatic substitutions</p></td><td class="cell"><p>+</p></td><td class="cell"><p>(must be said that the &lt;-&gt; goes without saying that the), (is fully in line &lt;-&gt; is totally coherent), (is amazing &lt;-&gt; strikes me)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Context-dependent substitu-</p></td><td class="cell"><p></p></td><td class="cell"><p>(is not right &lt;-&gt; is unacceptable), (offer my &lt;-&gt; express my)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tions</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Alignment and translation prob-</p></td><td class="cell"><p>-</p></td><td class="cell"><p>(unnecessary if &lt;-&gt; vital if), (the crime &lt;-&gt; organized), (ill-advised &lt;-&gt;</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>lems</p></td><td class="cell"><p></p></td><td class="cell"><p>wise), (to begin by thanking &lt;-&gt; to begin by congratulating)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Sadaf Abdul-Rauf and Holger Schwenk. 2009. On the Use of Comparable Corpora to Improve SMT perfor­mance. In <i>Proceedings ofEACL, </i>Athens, Greece.</p><p>Wilker Aziz, Marc Dymetman, Shachar Mirkin, Lucia Specia, Nicola Cancedda, and Ido Dagan. 2010. Learning an Expert from Human Annotations in Sta­tistical Machine Translation: the Case of Out-of-Vocabulary Words. In <i>Proceedings of EAMT, </i>Saint-Raphael, France.</p><p>Colin Bannard and Chris Callison-Burch. 2005. Para­phrasing with Bilingual Parallel Corpora. In <i>Proceed­ings ofACL, </i>Ann Arbor, USA.</p><p>Francis Bond, Eric Nichols, Darren Scott Appling, and Michael Paul. 2008. Improving statistical machine translation by paraphrasing the training data. In <i>Pro­ceedings ofIWSLT, </i>Hawai, USA.</p><p>Chris Callison-Burch, Colin Bannard, and Josh Schroeder.    2005.    Scaling Phrase-Based Statistical Machine Translation to Larger Corpora and Longer Phrases.<page local="11" global="666"/> In <i>Proceedings ofACL, </i>Ann Arbor,</p><p>USA.</p><p>Chris Callison-Burch, Philipp Koehn, and Miles Os-borne. 2006. Improved Statistical Machine Transla­tion Using Paraphrases. In <i>Proceedings ofNAACL, </i>New York, USA.</p><p>Chris Callison-Burch. 2008. Syntactic Constraints on Paraphrases Extracted from Parallel Corpora. In <i>Pro­ceedings ofEMNLP, </i>Hawai, USA.</p><p>Marine Carpuat and Dekai Wu. 2007. Context-Dependent Phrasal Translation Lexicons for Statisti­cal Machine Translation. In <i>Proceedings ofMachine Translation Summit XI, </i>Copenhagen, Denmark.</p><p>Marine Carpuat. 2009. One Translation Per Discourse. In <i>Proceedings </i><i>ofthe</i><i> NAACL-HLT Workshop on Se­mantic Evaluations, </i>Boulder, USA.</p><p>Jinhua Du, Jie Jiang, and Andy Way. 2010. Facilitating Translation Using Source Language Paraphrase Lat­tices. In <i>Proceedings ofEMNLP, </i>Cambridge, USA.</p><p>Kevin Gimpel and Noah A. Smith. 2008. Rich Source-Side Context for Statistical Machine Translation. In <i>Proceedings </i><i>ofthe</i><i> ACL Workshop on Statistical Ma­chine Translation, </i>Columbus, USA.</p><p>Rejwanul Haque, Sudip Kumar Naskar, Yanjun Ma, and Andy Way. 2009. Using Supertags as Source Lan­guage Context in SMT. In <i>Proceedings ofEAMT, </i>Barcelona, Spain.</p><p>Almut Silja Hildebrand, Matthias Eck, Stephan Vogel, and Alex Waibel. 2005. Adaptation of the Transla­tion Model for Statistical Machine Translation Based on Information Retrieval. In <i>Proceedings ofEAMT, </i>Budapest, Hungary.</p><p>David Kauchak and Regina Barzilay. 2006. Paraphras­ing for Automatic Evaluation.   In <i>Proceedings of</i> <i>NAACL HLT, </i>New York, USA.</p><p>Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In <i>Pro­ceedings ofNAACL HLT, </i>Edmonton, Canada.</p><p>Stanley Kok and Chris Brockett. 2010. Hitting the Right Paraphrases in Good Time. In <i>Proceedings ofNAACL, </i>Los Angeles, USA.</p><p>Adam Lopez. 2008. Tera-Scale Translation Models via Pattern Matching. In <i>Proceedings </i><i>ofCOLING,</i><i> </i>Manchester, UK.</p><p>Nitin Madnani and Bonnie J. Dorr. 2010. Generating Phrasal &amp; Sentential Paraphrases: A Survey of Data-Driven Methods. <i>Computational Linguistics, </i>36(3).</p><p>Nitin Madnani, Philip Resnik, Bonnie J. Dorr, and Richard Schwartz. 2008. Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Op­timization. In <i>Proceedings ofAMTA, </i>Waikiki, USA.</p><p>Yuval Marton, Chris Callison-Burch, and Philip Resnik.</p><p>2009. Improved Statistical Machine Translation Using Monolingually-derived Paraphrases. In <i>Proceedings ofEMNLP, </i>Singapore.</p><p>Aurelien Max, Rank Makhloufi, and Philippe Langlais. 2008. Explorations in using grammatical dependen­cies for contextual phrase translation disambiguation. In <i>Proceedings ofEAMT, </i>Hamburg, Germany.</p><p>Aurelien Max, Josep M. Crego, and Francois Yvon.</p><p>2010. Contrastive Lexical Evaluation of Machine Translation. In <i>Proceedings </i><i>ofLREC,</i><i> </i>Valletta, Malta.</p><p>Aurelien Max. 2008. Local rephrasing suggestions for supporting the work of writers. In <i>Proceedings ofGo-TAL, </i>Gothenburg, Sweden.</p><p>Shachar Mirkin, Lucia Specia, Nicola Cancedda, Ido Dagan, Marc Dymetman, and Idan Szpektor. 2009. Source-Language Entailment Modeling for Translat­ing Unknown Terms. In <i>Proceedings ofACL, </i>Singa­pore.</p><p>Behrang Mohit and Rebecca Hwa. 2007. Localization of Difficult-to-Translate Phrases. In <i>Proceedings of the ACL Workshop on Statistical Machine Translation, </i>Prague, Czech Republic.</p><p>Dragos Stefan Munteanu and Daniel Marcu. 2005. Im­proving Machine Translation Performance by Exploit­ing Non-parallel Corpora. <i>Computational Linguistics, </i>31(4).</p><p>Takashi Onishi, Masao Utiyama, and Eiichiro Sumita. 2010. Paraphrase Lattice for Statistical Machine Translation. In <i>Proceedings ofACL, short paper ses­sion, </i>Uppsala, Sweden.</p><p>Philip Resnik, Olivia Buzek, Chang Hu, Yakov Kron­rod, Alex Quinn, and Benjamin B. Bederson. 2010. Improving Translation via Targeted Paraphrasing. In <i>Proceedings ofEMNLP, </i>Cambridge, USA.</p><p>Josh Schroeder, Trevor Cohn, and Philipp Koehn. 2009. Word Lattices for Multi-Source Translation. In <i>Pro­ceedings ofEACL, </i>Athens, Greece.</p><p>Matthew Snover, Bonnie J. Dorr, Richard Schwartz, Lin-nea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annota­tion. In <i>Proceedings ofAMTA, </i>Boston, USA.</p><p>Nicolas Stroppa, Antal van den Bosch, and Andy Way. 2007. Exploiting Source Similarity for SMT using Context-Informed Features. In <i>Proceedings </i><i>ofTMI,</i><i> </i>Skovde, Sweden.</p><p>Guillaume Wisniewski, Alexandre Allauzen, and Francois Yvon. 2010. Assessing Phrase-based Trans­lation Models with Oracle Decoding. In <i>Proceedings</i> <i>ofEMNLP, </i>Cambridge, USA.</p></references></body></article>