<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="1288"/><title>Word-reordering for Statistical Machine Translation Using Trigram Language Model</title><pubinfo>Proceedings of the 5th International Joint Conference on Natural Language Processing,pages 1288-1293, Chiang Mai, Thailand, November 8 - 13, 2011. ©2011 AFNLP</pubinfo><author surname="He" givenname="Jing"><org  name="Tsinghua University" country="China" city="Beijing"/></author><author surname="Liang" givenname="Hongyu"><org  name="Tsinghua University" country="China" city="Beijing"/></author></firstpageheader><frontmatter><p><b>Word-reordering for Statistical Machine Translation Using Trigram</b></p><p><b>Language Model</b></p><p><b>Jing He</b></p><p>IIIS, Tsinghua University Beijing, China</p><p>hejing2 92 9@gmail.com</p><p><b>Hongyu Liang</b></p><p>sjqxzlhy@gmail.com</p></frontmatter><abstract>In this paper we study the word-reordering problem in the decoding part of statisti­cal machine translation, but independently from the target language generating pro­cess. In this model, a permuted sentence is given and the goal is to recover the correct order. We introduce a greedy algorithm called <i>Local-</i><i>(k,</i><i> l)-Step, </i>and show that it performs better than the DP-based algo­rithm. Our word-reordering algorithm can be used in the statistical machine transla­tion process for improving the quality of the translation. Furthermore, motivated by the rank evaluation method, we introduce a novel way for evaluating the results of word-reordering by calculating the inver­sion pair cardinality. </abstract></header><body><section number="1" title="Introduction"><p>Statistical machine translation is a machine trans­lation method based on statistical models, which is in contrast with rule-based machine translation as well as with example-based machine transla­tion. The most commonly used model in statistical machine translation is the source-channel model built by Brown et. al. (1993). In their pro­posal of translation between English and French, English strings are generated according to some stochastic process and then transformed stochas­tically into French strings. Therefore, to trans­late French to English, it is needed to search for an English source string that is most likely ac­cording to the English language model (Ponte and Croft, 1998) and the channel model. This process of translation is called decoding. Usually, a de­coding process in statistical machine translation is combined with two sub-processes (Chang and Toutanova, 2007; Koehn, Och and Marcu, 2003): generating the words or phrases of the target lan­guage, and deciding the correct order of the words or phrases to get a desired target language sen­tence. For some language pairs, such as English and Chinese, the word-reordering problem is re­ally hard to solve, as the target word order differs a lot from the source word order and little informa­tion about the target word order is obtainable from the source sentence. This is because the grammars of English and Chinese differ from one another significantly.</p><p>Language model has been successfully applied in the word-reordering process of statistical ma­chine translation. Generally speaking, a language model assigns a probability to a sequence ofwords according to some probability distribution. A sentence then gets a score under the language model by means of standard conditional proba­bility. Knight (1999) studied this abstract prob­lem and proved that the word-reordering (also called word-replacement) problem under bigram language model is <b>NP</b>-hard.</p><p>In this article, we study the word-reordering problem under trigram language model in the de­coding part of statistical machine translation, but independently from the target language generat­ing process. More precisely, suppose we want to translate a sentence from one language to an­other, and some methods have been applied to generate all the target words, whereas the words need reordering because two languages may have totally different grammars (like English and Chi­nese). Thus, our goal is to recover the correct, or reasonable, word order of the target sentence.</p><p>We reduce this problem to the traveling sales­man problem (TSP) in 3-uniform hypergraphs. We show that by some modification, the dynamic-programming (DP) based algorithm for TSP (Held and Karp, 1962) can be used in solving this gen­eralized problem. Nevertheless, the time com­plexity of this DP-based algorithm is exponential in the number of words in the sentence, and is thus unreasonable in practice. We design a class of greedy algorithms called <b>Local</b>-(k, l<b>)-Step, </b>pa­rameterized by <i>k </i>and <i>l.</i><page local="2" global="1289"/><i> </i>Roughly speaking, the <b>Local</b>-(k, l<b>)-Step </b>algorithm finds, in each step, k words which maximize the language model score, and then add the first l words to the partial sen­tence. For small k, say k &lt; <b>5, </b>this algorithm runs much faster then the DP-based algorithm.</p><p>We also propose a novel way to evaluate the results of word-reordering, motivated by the rank (ordering) evaluation measures in information re­trieval. Standard retrieval evaluation measures, such as Mean Average Precision (MAP), are used for evaluating the results of retrieval systems. The reordering or ranking of items is an important task in many real-world applications, and it is needed to compare two different orderings of the same item list. One commonly-used measure is the Kendall's tau coefficient (Kendall, 1938) which uses the notion of <i>inversion pairs.</i></p><p>Motivated by these existing methods, we de­sign the following evaluation process for word-reordering: Given a sentence outputted by the al­gorithm, we regard it as a permutation of the cor­rect sentence, and count the the number of in­version pairs in it, which can be seen as the dis­tance between the result sentence and the cor­rect one. The notion of inversion pairs is often used to measure the distance between permuta­tions. Due to the special structure of our model, it is also a proper measurement of the quality of word-reordering results. Note that to calculate the number of inversion pairs, the correct sen­tence, or a "standard answer", should be given as well. This can be done by the following design of experiments: We choose 1500 English sentences from http://www.nlp.org.cn, and randomly per­mute each sentence. Then, the permuted sentences are given as the input to the word-reordering prob­lem, while the original sentences are just "standard answers" which will be used in the evaluation.</p><p>We implement both the DP-based algorithm and the <b>Local-</b>(k, l<b>)-Step </b>algorithm, and evaluate their results according to their performance on the cho­sen sentences. From the comparison, it is shown that for well chosen parameters k and l, the <b>Local-( </b>k, l <b>) -Step </b>algorithm performs even better than the DP-based algorithm. This seems to contradict with the fact that the latter solves the problem ex­actly while the former only obtains an "approxi­mate" solution. This, however, is not a real prob­lem since the score under the language model is not always compatible with that under the evalua­tion using inversion pairs. In fact, neither of the two evaluation methods can accurately measure the "quality" or "correctness" of sentences, be­cause the human language itself comprises many other perspectives that cannot be qualified or char­acterized exactly by current techniques. It is pos­sible that a sentence that makes no sense obtains a higher language model score than that of a normal sentence in real world.</p><p>Finally let us mention that, although through­out the paper we talk about the word-based re­ordering model, our algorithms can be easily mod­ified to be applicable in the re-ordering process of statistical machine translation whose working unit is phrase (Koehn, Och and Marcu, 2003).</p><p>This paper is organized as follows: Section 2 defines the word-reordering model rigorously, and introduces the dynamic programming algorithm and the greedy <b>Local-</b>(k, l<b>)-step </b>algorithm. In Section 3 we show the experimental results as well as the evaluation based on the inversion pair cardi­nality. The last section concludes the whole article with some remarks and future work.</p></section><section number="2" title="The Word-reordering Model"><p>In the word-reordering model, a disordered En­glish sentence is given and the goal is to find the most reasonable order. For example, given the sentence "overrate to is importance their it easy", the best answer should be "It is easy to overrate their importance."</p><p>It is hard to establish a standard criterion for evaluating the "quality" of a sentence. In practice, language model is used as a statistical tool to help people find approximate solutions. For our pur­pose, we adopt the commonly-used trigram source model, given by <i>lm(w0\w1,w2) </i>for all possible English words wo, wi and w2. Given a disordered sentence, we want to rearrange its words in order to get a maximum score according to the trigram source model. This can be theoretically formal­ized as the following search problem.</p><p><b>Word-reordering Problem</b> such that fii=3 <i>lm(sij \ sij_1 ,sij_2 </i>) is maxi­mized.<page local="3" global="1290"/></p><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">Input:</doubt><doubt alpha="51.6" length="31" tooSmall="False" monospace="0.0">1. A dictionaryD = [di\1&lt;i &lt;m};</doubt><doubt alpha="57.4" length="54" tooSmall="False" monospace="0.0">2. A trigram source model[lm(wi\wj,wk) : 1&lt;i,j,k &lt; m};</doubt><doubt alpha="59.2" length="76" tooSmall="False" monospace="0.0">3. A set of wordsS = [s1, s2,..., sn}ÇD.Output:A sentencesi1si2...sin, where</doubt><doubt alpha="45.5" length="44" tooSmall="False" monospace="0.0">i2,..., in)is a permutation of[1, 2,..., n},</doubt><p>A similar problem with bigram source model <i>[lm(wi\wj)} </i>is proved to be NP-hard by Knight (1999) using a reduction from the famous Hamil­ton ian Path problem. The NP-hardness of this problem with trigram source model can be proved analogously.</p><p>Despite the hardness result, we can still cope with many instances in practice, since a sentence is usually not so long. The trivial way is to enu­merate all n! permutations and find the one achiev­ing maximum score, which takes O(n <i>• </i>n!) time. We will describe better ways for solving it.</p><subsection number="2.1" title="Dynamic Programming"><p>An instance of the TSP problem consists of a di­rected graph <i>G = </i>(V, E<i>) </i>and a cost function c : <i>E — </i><b>R, </b>and the goal is to find a path of minimum cost which visits each vertex in V exactly once (that is, a Hamiltonian path). Note that the word-reordering problem can be reduced to Traveling S alesman Problem (TSP) with a slight modifi­cation that each edge in the graph is a triple instead of a pair (i.e., TSP in 3-uniform hypergraphs). The reduction is as follows: Construct a vertex <i>vi </i>for each word si G S, and then add an hyperedge (vi; Vj, vk) with cost — ln(lm(wk<i>\wi, </i><i>Wj</i><i>)) </i>for ev­ery triple <i>(vi,Vj,vk</i>). Then, finding a sentence s with maximum score is equivalent to finding a minimum cost Hamiltonian path in this hyper­graph, where the cost of a path is defined to be the sum of costs of all triples containing three consecutive nodes in the path. We can assume that the graph is complete (i.e., all possible edges <i>(vi,vj,vk) </i>exist) by adding dummy edges with sufficiently large costs.</p><p>Held and Karp (1962) designed a dynamic-programming based (DP-based) algorithm for the original TSP which runs in time O(2n • n<footnote anchor="2"/>), n be­ing the number of vertices in the graph. Their al­gorithm runs iteratively on all subsets of vertices and finds a minimum cost tour in that subset, with the start and end points specified. We will describe a similar algorithm for solving TSP on 3-uniform hypergraphs, which can be directly applied to the word-reordering problem.</p><p><b>DP-based algorithm for TSP in 3-uniform hy­pergraphs (3-uni-DP)</b> <b>Input:</b><b> </b>A 3-uniform hypergraph G = (V, E); a cost function <i>c </i>: <i>E </i><b>R.</b></p><p><b>Output: </b>A minimum cost Hamiltonian path. <b>Algorithm:</b></p><p>• Return <i>fail </i>if \V\ &lt; 3.</p><p>• For each triple (vi,vj- ,vk ) where all three vertices are pairwise distinct, let</p><p><i>C ({vi, vj ,vk }, vi,vj ,vk ) = </i>c(v<b>i, </b>v<b>j</b>, vfc ).</p><doubt alpha="50.0" length="18" tooSmall="False" monospace="0.0">•Form = 4to\ V \do</doubt><doubt alpha="46.1" length="128" tooSmall="False" monospace="0.0">-For each tuple(V', vi,v2,v3)where{vi,v2,v3} ç V' C Vand\V'\ = m,compute:C (V ',v1, v2,v3) = minv4€y/{C(V' \ {v3}, vi, v4, v2) +</doubt><doubt alpha="33.3" length="12" tooSmall="False" monospace="0.0">C(v4,v2,v3)}</doubt><p>• Find the vertices vi , v2, v3 for which C(V, vi, v2, v3) is minimized. Trace back to find the corresponding path.</p><p>The algorithm <b>3-uni-DP </b>runs in time O(2n • n<footnote anchor="4"/>), where n = V . In each step it computes C(V', vi, v2, v3), which stands for the minimum cost Hamiltonian path in V' that starts with vi and ends at (v2, v3), by enumerating the third vertex on the path from end and concatenating the shorter path and the last edge. It is not hard to see that this algorithm correctly computes the minimum cost Hamiltonian path in 3-uniform hypergraphs.</p></subsection><subsection number="2.2" title="Local -(k, l )-Step Algorithm"><p>The DP-based algorithm <b>3-uni-DP </b>solves the word-reordering problem exactly but runs in ex­ponential time, which is unaffordable for long sen­tences. An idea for reducing the running time is to consider the problem "locally". In each step, we look for a fixed number of unvisited points and try to minimize the cost of the "local path" in which these points are involved. More specifically, we seek for k unvisited points v2, v3,..., vk+1 min­imizing <i>J2</i><b>j</b>to<footnote anchor="1"/> c(v<b>i</b>,v<b>i</b>+i,v<b>i</b>+2), where v0 and vi are the last two nodes in the current partial path. Then, we add the first l points to our partial path, for some l &lt; k. We give a more formal descrip­tion as follows.</p><p><b>Algorithm Local</b>-(k, l<b>)-Step</b> <b>Input:</b><b> </b>A 3-uniform hypergraph G = (V, E); a cost function c : E — <b>R. Output: </b>A minimum cost Hamiltonian path.</p><p><b>Algorithm:</b></p><p>Do the following for all vertex pairs (v0, vi ) to find the best solution:</p><page local="4" global="1291"/><p><b>• </b>V' <b>&lt;</b>- V \{vo,vi}.</p><p><b>• </b>PartialPath <b>&lt;</b>— (v0,vi).</p><doubt alpha="60.0" length="15" tooSmall="False" monospace="0.0">•While\V'\ &gt;kdo</doubt><p><b>- </b>Find k distinct vertices v2, v3, . . . , v +i in        V' which minimizes £i=0 c(vi,vi+i,vi+2).</p><p><b>- </b>Add  (v2,..., v1+i)  to  the  end of PartialPath.</p><p><b>- </b>V' <b>&lt;</b>- V' \{v2,v3,...,v|+i<b>}</b>.</p><p><b>• </b>In case V' = <b>0, </b>perform an exhaustive search to find the best order to visit the remaining vertices in V', and add this to PartialPath.</p><p>Return the best PartialPath (with the minimum cost) over all start pairs (v0, vi).</p><p>The algorithm <b>Local</b>-(k, l<b>)-Step </b>runs in time O(n<footnote anchor="2"/>-nfc<b>• </b>j) = O(nk+<footnote anchor="3"/>/l), where n is the number of vertices in the graph. This is efficient in practice if we choose a small k, say, k &lt; 5. It should be noticed that this algorithm cannot be a constant-factor approximation algorithm for TSP, since for any constant k it runs in polynomial time, and thus cannot approximate TSP within any constant ratio unless <b>P </b>= <b>NP. </b>However, it may perform well on real-world instances of the word-reordering prob­lem.</p><p>In fact we will show that, for some well cho­sen parameters k and l, the algorithm <b>Local-</b>( k, l ) <b>-Step </b>performs even better than the algo­rithm <b>3-uni-DP. </b>This seems to contradict with the fact that the latter solves the problem optimally while the former only looks for a reasonable solu­tion. However, this is not a problem because our word-reordering model itself cannot catch accu­rately the quality of a sentence. Thus, when eval­uating the outcomes of our algorithms, it is more proper to use some other measurements, like the <i>inversion pair </i>which will be introduced later. The reason why this measurement cannot be adopted in the design of our algorithms is that it can only be calculated if a standard answer to the problem is given.</p></subsection></section><section number="3" title="Experiments"><subsection number="3.1" title="The trigram language model training"><p>In order to build a reasonable trigram language model for the experiment, we download the third version of the Europarl corpus (Koehn, 2005) which is extracted from the proceedings of the Eu­ropean Parliament. This data set is usually used as a base material in statistical machine translation contest or other research projects involving Euro­pean languages, and the English part can be used for training a trigram English language model. There are about 0.307 million English sentences in the material. Thus, the trigram language model built by SRILM (Stolcke, 2002) can reflect the properties of the English language.</p></subsection><subsection number="3.2" title="The reordering experiment"><p>How to evaluate the results of our algorithms is a key problem in our research, as there is no stan­dard for testing the accuracy of word-reordering. Usually, the reordering of words in sentences serves as a subprocess in statistical machine trans­lation, especially in the decoding step. All the ex­isting methods and standards are designed to test the accuracy of the translation results, but not the single process of word-reordering. Therefore, we designed an experiment model and a testing prin­ciple for our own purpose, shown as follows.</p><p>First, we choose 1500 English-Chinese sen­tence pairs (from http://www.nlp.org.cn) as the collection of standard answers. We then choose from them all sentences of length less than 10 as our data set. For every sentence in this data set, we generate a disordered sentence by performing a random permutation on the set of words in the original sentence. For example, if the original sen­tences is "sometimes you are overly frank", one possible disordered sentence could be "overly are sometimes frank you". The original sentence is used as a standard answer for later evaluation. We run the <b>Local</b>-(k, l<b>)-Step </b>algorithm on all the per­muted sentences for all pair of parameters (k, l) such that 1 <b>&lt; </b>l <b>&lt; </b>k <b>&lt; </b>4. We also run the <b>3-uni-DP </b>algorithm on these sentences for the sake of contrast.</p><p>There is another issue considering the start of a sentence. Since the language model includes the possibility of a word being the first word of a sen­tence, we may add a special "start symbol" to ev­ery disordered sentence and force it to be the first word in the output sentence. This is easy to im­plement in practice, and will make the result more reasonable. In all the experiments we adopt this setting.</p><p>Now comes the evaluation part. How to mea­sure the quality of our results, or, the distances between the output sentences and the standard an­swers?<page local="5" global="1292"/> It turns out that any method for determin­ing the distance between permutations also works in our model. The concept of <i>inversion pairs </i>is usually used to measure the distance between two permutations, and is thus brought into our exper­iments. Let <i>a </i>be a permutation of {1,2,..., n}. A pair of indices (i, <i>j</i>), where 1 &lt; <i>i &lt; j &lt; n, </i>is called an <i>inversion pair </i>of <i>a </i>if <i>a(i) &gt; a(j</i>). The total number of inversion pairs of a, also called <i>in­version pair cardinality </i>of a, is a proper measure­ment of the distance between a and the identity permutation (1,2,..., n). Although distances be­tween two arbitrary permutations can be similarly defined, this measurement already suffices for our purpose since we only need to calculate the dis­tance between a permuted sentence and the stan­dard answer.</p><p>Take again the sentence "sometimes you are overly frank" as an illustration. We mark this sentence as the identity permutation (1, 2, 3, 4, 5). If the disordered sentence is "sometimes are you frank overly", it should be associated with the per­mutation (1, 3, 2, 5, 4), and thus has 2 inversion pairs in total (3 comes before 2, and 5 appears be­fore 4).</p><p>In this way we can calculate the number of inversion pairs to measure the degree of disor­dering. For every pair of parameters <i>(k, l) </i>used in the <b>Local</b>-(k, l<b>)-Step </b>algorithm, we count the number of output sentences which have smaller inversion pair cardinality than the corresponding randomly permuted sentences. Call this number <i>Better(k, </i>l). We then choose the pair (k, l) which maximizes <i>Better(k, </i>l), denoted by (ko, lo), and adopt it as the proper parameter for our algorithm. We also calculate <i>Better(DP</i>), which is the num­ber of sentences outputted by the <b>3-uni-DP </b>al­gorithm having smaller inversion pair cardinality than the corresponding disordered ones.</p><p>All the experiments are conducted repeatedly for 5 times, each time generating different ran­domly disordered sentences. The final result <i>Better(DP) </i>and <i>Better(k, </i><i>l)</i><i> </i>are the average of the results of 5 independent experiments. These results are shown in Tables 1 and 2.</p></subsection><subsection number="3.3" title="Analysis"><p>The experimental result shows that setting (k, l) = (4, 3) gives the best outcome among all chosen pa­rameters. Thus we let ko = 4 and lo = 3. It</p><p>Table 1: <i>Better (DP </i>) and <i>Better </i><i>(k,l)</i><i> </i>under all chosen parameters is a little surprising that <i>Better(DP) </i>is less than <i>Better(k0, l0</i>), which indicates that the exact al­gorithm performs worse than the "approximate" algorithm. As explained before, this is due to our lack of modeling the "quality" or "correctness" of sentences. It is possible that a sentence with high score under trigram language model makes little sense, and a normal sentence appearing in real word obtains a low score under this language model. The algorithm <b>Local</b>-(k, l<b>)-Step, </b>on the other hand, makes use of the locality of English sentences, and thus it should be expected to per­form well.</p></subsection></section><section number="4" title="Conclusions"><p>In this paper, we study the word-reordering prob­lem in the decoding process of statistical machine translation. We adopt the commonly-used trigram language model, and abstract the word-reordering problem as instances of Traveling Salesman Prob­lem in 3-uniform hypergraphs. We show that Held and Karp's dynamic-programming based al­gorithm for solving TSP (in normal graphs) can be adapted to solve this problem. We also design a greedy algorithm called <b>Local</b>-(k, l<b>)-Step, </b>param­eterized by <i>k </i>and <i>l, </i>which has a faster running time but gets a non-optimal solution, where by optimal we mean achieving maximum score under the tri-gram language model.<page local="6" global="1293"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>3</p></td><td class="cell"><p>4</p></td><td class="cell"><p>5</p></td><td class="cell"><p>average</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(1,1)</p></td><td class="cell"><p>347</p></td><td class="cell"><p>354</p></td><td class="cell"><p>329</p></td><td class="cell"><p>360</p></td><td class="cell"><p>349</p></td><td class="cell"><p>347.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(2,1)</p></td><td class="cell"><p>376</p></td><td class="cell"><p>389</p></td><td class="cell"><p>370</p></td><td class="cell"><p>386</p></td><td class="cell"><p>388</p></td><td class="cell"><p>381.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(2,2)</p></td><td class="cell"><p>383</p></td><td class="cell"><p>390</p></td><td class="cell"><p>369</p></td><td class="cell"><p>395</p></td><td class="cell"><p>388</p></td><td class="cell"><p>385</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(3,1)</p></td><td class="cell"><p>392</p></td><td class="cell"><p>396</p></td><td class="cell"><p>386</p></td><td class="cell"><p>399</p></td><td class="cell"><p>399</p></td><td class="cell"><p>394.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(3,2)</p></td><td class="cell"><p>392</p></td><td class="cell"><p>397</p></td><td class="cell"><p>383</p></td><td class="cell"><p>388</p></td><td class="cell"><p>388</p></td><td class="cell"><p>389.6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(3,3)</p></td><td class="cell"><p>396</p></td><td class="cell"><p>406</p></td><td class="cell"><p>380</p></td><td class="cell"><p>401</p></td><td class="cell"><p>387</p></td><td class="cell"><p>394</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(4,1)</p></td><td class="cell"><p>399</p></td><td class="cell"><p>413</p></td><td class="cell"><p>390</p></td><td class="cell"><p>405</p></td><td class="cell"><p>407</p></td><td class="cell"><p>402.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(4,2)</p></td><td class="cell"><p>400</p></td><td class="cell"><p>406</p></td><td class="cell"><p>398</p></td><td class="cell"><p>407</p></td><td class="cell"><p>403</p></td><td class="cell"><p>402.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(4,3)</p></td><td class="cell"><p>412</p></td><td class="cell"><p>414</p></td><td class="cell"><p>401</p></td><td class="cell"><p>421</p></td><td class="cell"><p>416</p></td><td class="cell"><p>412.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(4,4)</p></td><td class="cell"><p>406</p></td><td class="cell"><p>412</p></td><td class="cell"><p>404</p></td><td class="cell"><p>421</p></td><td class="cell"><p>420</p></td><td class="cell"><p>412.6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>DP</p></td><td class="cell"><p>362</p></td><td class="cell"><p>370</p></td><td class="cell"><p>364</p></td><td class="cell"><p>372</p></td><td class="cell"><p>353</p></td><td class="cell"><p>364.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 2: The comparison between DP-based and greedy algorithms" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>average number</p></td><td class="cell"><p>percentage</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Better(k0, l0)</i></p></td><td class="cell"><p>412.8</p></td><td class="cell"><p>62.55%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Better (DP</i>)</p></td><td class="cell"><p>364.2</p></td><td class="cell"><p>55.18%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>All sentences</p></td><td class="cell"><p>660</p></td><td class="cell"><p>100%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>We implement both algorithms and conduct some experiments. To evaluate the results, we adopt the concept of inversion pairs to measure the distances between the standard answers and the sentences outputted by the algorithms. From the experimental results, we find that setting <i>k </i>= 4 and <i>l </i>= 3 makes the <b>Local</b><i>-(k, l</i><b>)-Step </b>algorithm perform best. Moreover, the result obtained by the greedy algorithm is even better than that of the dynamic-programming based algorithm. This is because we are not able to model the quality and correctness of sentences accurately, and thus a sentence with maximum score under trigram lan­guage model is not necessarily a best answer to the reordering problem.</p><p>Since we study the word-reordering problem in­dependently from the target language generating process, one future direction for our research is to combine the two parts, namely, predicting the col­lection of words and deciding the correct order of the target sentences, together to design better de­coding algorithms.</p><p>J. M. Ponte and W. B. Croft. A Language Model Ap­proach to Information retrieval. <i>Research and Devel­opment in Information Retrieval, </i>275-281, 1998.</p><p>A. Stolcke. SRILM - An Extensible Language Mod­eling Toolkit. In <i>Proceedings </i><i>ofthe</i><i> International Conference on Spoken Language Processing, </i>vol. 2, pp. 901-904, Denver, 2002.</p><p>A. Aho and J. Ullman. <i>The Theory of Parsing, Trans­lation and Compiling, </i>vol. 1. Prentice-Hall, Engle-wood Cliffs, NJ, 1972.</p></section><references><p>M. Kendall. A New Measure of Rank Correlation. <i>Biometrika, </i>30(1-2): 81-89, 1938.</p><p>P. Brown, S. Della-Pietra, V. Della-Pietra, and R. Mer­cer. The mathematics of statistical machine transla­tion: Parameter estimation. <i>Computational Linguis­tics, </i>19(2): 263-311, 1993.</p><p>P. Chang and K. Toutanova. A Discriminative Syn­tactic Word Order Model for Machine Translation.</p><p>In <i>Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, </i>pp. 9-16, Prague, Czech Republic, June 2007.</p><p>M. Held and R. M. Karp. A dynamic programming ap­proach to sequencing problems. <i>Journal ofSociety for Industrial and Applied Mathematics, </i>10(1):196210.</p><p>K. Knight. Decoding Complexity in Word-Replacement Translation Models. <i>Computational</i> <i>Linguistics, </i>25(4):607-615, 1999.</p><p>P. Koehn. Europarl: A Parallel Corpus for Statistical Machine Translation. MT Summit, 2005.</p><p>P. Koehn, F. J. Och, and D. Marcu. Statistical Phrases-Based Translation. HLT/NAACL, 2003.</p></references></body></article>