<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="273"/><title>Efficient Incremental Decoding for Tree-to-String Translation</title><pubinfo>Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,pages 273-283, MIT, Massachusetts, USA, 9-11 October 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Huang" givenname="Liang"><org  name="Chinese Academy of Sciences" country="China" city="Beijing"/></author><author surname="Mi" givenname="Haitao"><org  name="Chinese Academy of Sciences" country="China" city="Beijing"/></author></firstpageheader><frontmatter><p><b>Efficient Incremental Decoding for Tree-to-String Translation</b></p><p><b>Liang Huang <footnote anchor="1"/> Haitao Mi </b><b><footnote anchor="21"/></b></p><p>information Sciences Institute    <footnote anchor="2"/>Key Lab. of Intelligent Information Processing</p><p>University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292, USA {lhuang,haitaomi}@isi.edu</p><p>Institute of Computing Technology Chinese Academy of Sciences P.O. Box 2704, Beijing 100190, China htmi@ict.ac.cn</p></frontmatter><abstract>Syntax-based translation models should <i>in principle </i>be efficient with polynomially-sized search space, but in practice they are often embarassingly slow, partly due to the cost of language model integration. In this paper we borrow from phrase-based decoding the idea to generate a translation <i>incrementally </i>left-to-right, and show that for tree-to-string models, with a clever encoding of deriva­tion history, this method runs in average-case polynomial-time in theory, and linear­time with beam search in practice (whereas phrase-based decoding is exponential-time in theory and quadratic-time in practice). Exper­iments show that, with comparable translation quality, our tree-to-string system (in Python) can run more than 30 times faster than the phrase-based system Moses (in C++). </abstract></header><body><section number="1" title="Introduction"><p>Most efforts in statistical machine translation so far are variants of either phrase-based or syntax-based models. From a theoretical point of view, phrase-based models are neither expressive nor efficient: they typically allow arbitrary permutations and re­sort to language models to decide the best order. In theory, this process can be reduced to the Traveling Salesman Problem and thus requires an exponential-time algorithm (Knight, 1999). In practice, the de­coder has to employ beam search to make it tractable (Koehn, 2004). However, even beam search runs in quadratic-time in general (see Sec. 2), unless a small <i>distortion limit </i>(say, d=5) further restricts the possi­ble set of reorderings to those local ones by ruling out any long-distance reorderings that have a "jump"</p><p>Table 1 : [main result] Time complexity of our incremen­tal tree-to-string decoding compared with phrase-based. In practice means "approximate search with beams." longer than d. This has been the standard prac­tice with phrase-based models (Koehn et al., 2007), which fails to capture important long-distance re-orderings like SVO-to-SOV.</p><p>Syntax-based models, on the other hand, use syntactic information to restrict reorderings to a computationally-tractable and linguistically-motivated subset, for example those generated by synchronous context-free grammars (Wu, 1997; Chiang, 2007). In theory the advantage seems quite obvious: we can now express global reorderings (like SVO-to-VSO) in polynomial-time (as opposed to exponential in phrase-based). But unfortunately, this polynomial complexity is super-linear (being generally cubic-time or worse), which is slow in practice. Furthermore, language model integration becomes more expensive here since the decoder now has to maintain target-language boundary words at <i>both </i>ends of a subtranslation (Huang and Chiang, 2007), whereas a phrase-based decoder only needs to do this at one end since the translation is always growing left-to-right. As a result, syntax-based models are often embarassingly slower than their phrase-based counterparts, preventing them from becoming widely useful.</p><p>Can we combine the merits of both approaches? While other authors have explored the possibilities of enhancing phrase-based decoding with syntax-aware reordering (Galley and Manning, 2008), we are more interested in the other direction, i.e., can syntax-based models learn from phrase-based de­coding, so that they still model global reordering, but in an efficient (preferably linear-time) fashion?<page local="2" global="274"/></p><table class="main" frame="box" rules="all" border="0" regular="True"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>in theory</i></p></td><td class="cell"><p><i>in practice</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>phrase-based tree-to-string</p></td><td class="cell"><p>exponential polynomial</p></td><td class="cell"><p>quadratic linear</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Watanabe et al. (2006) is an early attempt in this direction: they design a phrase-based-style de­coder for the hierarchical phrase-based model (Chi­ang, 2007). However, this algorithm even with the beam search still runs in quadratic-time in prac­tice. Furthermore, their approach requires grammar transformation that converts the original grammar into an equivalent binary-branching Greibach Nor­mal Form, which is not always feasible in practice.</p><p>We take a fresh look on this problem and turn our focus to one particular syntax-based paradigm, tree-to-string translation (Liu et al., 2006; Huang et al., 2006), since this is the simplest and fastest among syntax-based approaches. We develop an incremen­tal dynamic programming algorithm and make the following contributions:</p><p>• we show that, unlike previous work, our in­cremental decoding algorithm runs in average-case <b>polynomial-time </b>in theory for tree-to-string models, and the beam search version runs in <b>linear-time </b>in practice (see Table 1);</p><p>• large-scale experiments on a tree-to-string sys­tem confirm that, with comparable translation quality, our incremental decoder (in Python) can run more than 30 times faster than the phrase-based system Moses (in C++) (Koehn</p><doubt alpha="28.6" length="14" tooSmall="False" monospace="0.0">et al., 2007);</doubt><p>• furthermore, on the same tree-to-string system, incremental decoding is slightly faster than the standard cube pruning method at the same level of translation quality;</p><p>• this is also the first linear-time incremental de­coder that performs global reordering.</p><p>We will first briefly review phrase-based decod­ing in this section, which inspires our incremental algorithm in the next section.</p></section><section number="2" title="Background: Phrase-based Decoding"><p>We will use the following running example from Chinese to English to explain both phrase-based and syntax-based decoding throughout this paper:</p><p>o <i>Bùshî i yü </i>2 <i>Shalong 3 jüxing 4 le  5 huitan 6</i>Bush   with Sharon   hold    -ed meeting</p><p>'Bush held talks with Sharon'</p><subsection number="2.1" title="Basic Dynamic Programming Algorithm"><p>Phrase-based decoders generate partial target-language outputs in left-to-right order in the form of <i>hypotheses </i>(Koehn, 2004). Each hypothesis has a <i>coverage vector </i>capturing the source-language words translated so far, and can be extended into a longer hypothesis by a phrase-pair translating an un­covered segment. This process can be formalized as a deductive system. For example, the following de­duction step grows a hypothesis by the phrase-pair <i>(yü Shäälong, </i>with Sharon) covering Chinese span [1-3]:</p><doubt alpha="46.9" length="81" tooSmall="False" monospace="0.0">(•—•••6) :(w,"Bush held talks") (•••3»»») :(w',"Bush held talks with Sharon") (1)</doubt><p>where a <b>• </b>in the coverage vector indicates the source word at this position is "covered" and where <i>w </i>and w' = w+c+d are the weights of the two hypotheses, respectively, with c being the cost of the phrase-pair, and d being the <i>distortion cost. </i>To compute d we also need to maintain the ending position of the last phrase (the 3 and 6 in the coverage vector).</p><p>To add a bigram model, we split each <b>—</b>LM item above into a series of +LM items; each +LM item has the form <i>(v,a ) </i>where <i>a </i>is the last word of the hypothesis. Thus a +LM version of (1) might be:</p><doubt alpha="52.7" length="93" tooSmall="False" monospace="0.0">(•„•••6,telks) : (w, "Bush held talks") (•••.(•••,Sharon) :(w',"Bush held talks with Sharon")</doubt><p>where the score of the resulting +LM item now includes a <i>combination cost </i>due to the bigrams formed when applying the phrase-pair. The com­plexity of this dynamic programming algorithm for g-gram decoding is <i>O(2nn<footnote anchor="2"/>\V</i><b>|</b>a_<footnote anchor="1"/>) where n is the sentence length and |V<b>| </b>is the English vocabulary size (Huang and Chiang, 2007).</p><doubt alpha="51.5" length="33" tooSmall="False" monospace="0.0">w' = w + c + d—log P1m(with|talk)</doubt><page local="3" global="275"/><p>Figure 1: Beam search in phrase-based decoding expands the hypotheses in the current bin (#2) into longer ones.</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">VP</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">PP</doubt><p>held x2 with x1 <i>yü jüxing le</i></p><doubt alpha="63.6" length="22" tooSmall="False" monospace="0.0">PX1:NP VV ASX2:NPI I I</doubt><figure caption="Figure 2: Tree-to-string rule r3for reordering."></figure></subsection><subsection number="2.2" title="Beam Search in Practice"><p>To make the exponential algorithm practical, beam search is the standard approximate search method (Koehn, 2004). Here we group +LM items into <i>n </i>bins, with each bin Bi hosting at most b items that cover exactly i Chinese words (see Figure 1). The complexity becomes <i>O(n<footnote anchor="2"/>b) </i>because there are a to­tal of <i>O(nb) </i>items in all bins, and to expand each item we need to scan the whole coverage vector, which costs <i>O(n). </i>This quadratic complexity is still too slow in practice and we often set a small <i>distor­tion limit </i>of dmax (say, 5) so that no jumps longer than dmax are allowed. This method reduces the complexity to O(nbdmax) but fails to capture long­distance reorderings (Galley and Manning, 2008).</p></subsection></section><section number="3" title="Incremental Decoding for Tree-to-String"><p><b>Translation</b></p><p>We will first briefly review tree-to-string translation paradigm and then develop an incremental decoding algorithm for it inspired by phrase-based decoding.</p><subsection number="3.1" title="Tree-to-string Translation"><p>A typical tree-to-string system (Liu et al., 2006; Huang et al., 2006) performs translation in two steps: parsing and decoding. A parser first parses the source language input into a 1-best tree T, and the decoder then searches for the best <i>derivation </i>(a se(a) <i>Büshi </i>[yü <i>Shäälong </i>]<i>1  [jüxing le huitan </i>]<i>2</i></p><p><b>jj</b><b> </b><i>1-best parser</i></p><doubt alpha="50.0" length="8" tooSmall="False" monospace="0.0">(b) IP@e</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">NP@1I</doubt><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">Büi shi</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">VP@2</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">PP@2.1</doubt><doubt alpha="22.2" length="9" tooSmall="False" monospace="0.0">~Np@2.1.2</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">VP@2.2</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(c)</doubt><p><i>yü</i><i>   Shalong  jüxing   le hüitan</i> <i>Shalong  jüxing   le hüitan</i> <i>hüitai n</i> <i>Shaalong</i></p><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">njj</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">vp@2</doubt><doubt alpha="25.0" length="8" tooSmall="False" monospace="0.0">NP@2.1.2</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">T2jj</doubt><doubt alpha="62.5" length="8" tooSmall="False" monospace="0.0">(d) Bush</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">P</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">I</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">yü</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">held</doubt><doubt alpha="47.6" length="21" tooSmall="False" monospace="0.0">VV    AS NP@2.2.3IIII</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">T3j</doubt><doubt alpha="33.3" length="9" tooSmall="False" monospace="0.0">NP@2.2.3I</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">with</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">tajr5j</doubt><doubt alpha="53.3" length="45" tooSmall="False" monospace="0.0">(e) Bush    [held    talks]2[with    Sharon]1</doubt><p>Figure 3: An example derivation of tree-to-string trans­lation (much simplified from Mi et al. (2008)). Shaded regions denote parts of the tree that matches the rule.</p><p>quence of translation steps) d* that converts source tree T into a target-language string.</p><p>Figure 3 shows how this process works. The Chi­nese sentence (a) is first parsed into tree (b), which will be converted into an English string in 5 steps. First, at the root node, we apply rule <b>t</b>1 preserving the top-level word-order which results in two unfinished subtrees, NP@1 and VP@<footnote anchor="2"/> in (c). Here <i>X</i>@n denotes a tree node of la­bel <i>X </i>at tree address n (Shieber et al., 1995). (The root node has address e, and the first child of node n has address n-1, etc.) Then rule r2 grabs the <i>Büshi </i>subtree and transliterate it into the English word<page local="4" global="276"/></p><doubt alpha="47.8" length="23" tooSmall="False" monospace="0.0">(n)IP (X1:NP X2:VP)X1X2</doubt><doubt alpha="42.9" length="14" tooSmall="False" monospace="0.0">VV AS NP@2.2.3</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">IIIII</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">N</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>_•_</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b>\</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>_•_</p></td><td class="cell"><p></p></td><td class="cell"><p>_</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b>\</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>_•_</p><p>_•_</p><p>•_</p></td><td class="cell"><p></p></td><td class="cell"><p>_</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>\</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>3</p></td><td class="cell"><p>4</p></td><td class="cell"><p>5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="59.6" length="52" tooSmall="False" monospace="0.0">in theory in practice phrase*       O(2nn2• |VO(n2b)</doubt><doubt alpha="56.3" length="71" tooSmall="False" monospace="0.0">tree-to-str O(nc •jV|4(g-1))O(ncb2) this work*   O(nklog2(cr)• |VO(ncb)</doubt><p>Table 2: Summary oftime complexities ofvarious algo­rithms. b is the beam width, V is the English vocabulary, and c is the number of translation rules per node. As a special case, phrase-based decoding with distortion limit dmax is <i>O(nbdmax</i>). *: incremental decoding algorithms.</p><p>"Bush". Similarly, rule r3 shown in Figure 2 is ap­plied to the VP subtree, which swaps the two NPs, yielding the situation in (d). Finally two phrasal rules r4 and r5 translate the two remaining NPs and finish the translation.</p><p>In this framework, decoding without language model (—LM decoding) is simply a linear-time depth-first search with memoization (Huang et al., 2006), since a tree of n words is also of size O(n) and we visit every node only once. Adding a language model, however, slows it down signifi­cantly because we now have to keep track of target-language boundary words, but unlike the phrase-based case in Section 2, here we have to remember both sides the leftmost and the rightmost boundary words: each node is now split into +LM items like (n <i>a </i><i>* </i>b) where n is a tree node, and <i>a </i>and b are left and right English boundary words. For example, a bigram +LM item for node VP®<footnote anchor="2"/> might be</p><doubt alpha="63.2" length="19" tooSmall="False" monospace="0.0">(VP@2held * Sharon)</doubt><p>This is also the case with other syntax-based models like Hiero or GHKM: language model integration overhead is the most significant factor that causes syntax-based decoding to be slow (Chiang, 2007). In theory +LM decoding is <i>O(ncjV</i>|4(g-1)), where V denotes English vocabulary (Huang, 2007). In prac­tice we have to resort to beam search again: at each node we would only allow top-b +LM items. With beam search, tree-to-string decoding with an inte­grated language model runs in time O(ncb<footnote anchor="2"/>), where b is the size of the beam at each node, and c is (max­imum) number of translation rules matched at each node (Huang, 2007). See Table 2 for a summary.</p></subsection><subsection number="3.2" title="Incremental Decoding"><p>Can we borrow the idea of phrase-based decoding, so that we also grow the hypothesis strictly left-to-right, and only need to maintain the rightmost boundary words?</p><p>The key intuition is to adapt the coverage-vector idea from phrase-based decoding to tree-to-string decoding. Basically, a coverage-vector keeps track of which Chinese spans have already been translated and which have not. Similarly, here we might need a "tree coverage-vector" that indicates which sub­trees have already been translated and which have not. But unlike in phrase-based decoding, we can not simply choose any arbitrary uncovered subtree for the next step, since rules already dictate which subtree to visit next. In other words what we need here is not really a tree coverage vector, but more of a derivation history.</p><p>We develop this intuition into an agenda repre­sented as a stack. Since tree-to-string decoding is a top-down depth-first search, we can simulate this re­cursion with a stack of active rules, i.e., rules that are not completed yet. For example we can simulate the derivation in Figure 3 as follows. At the root node IP®e, we choose rule <b>t1, </b>and push its English-side to the stack, with variables replaced by matched tree nodes, here x1 for NP®<footnote anchor="1"/> and x2 for VP®<footnote anchor="2"/>. So we have the following stack where the dot. indicates the next symbol to process in the English word-order. Since node NP®<footnote anchor="1"/> is the first in the English word-order, we expand it first, and push rule r2 rooted at NP to the stack:</p><doubt alpha="29.4" length="17" tooSmall="False" monospace="0.0">s = [. NP®1VP®2],</doubt><p>[. NP®<footnote anchor="1"/> VP®<footnote anchor="2"/> ] [. Bush].</p><p>Since the symbol right after the dot in the top rule is a word, we immediately grab it, and append it to the current hypothesis, which results in the new stack [. NP®<footnote anchor="1"/> VP®<footnote anchor="2"/> ] [Bush . ].</p><p>Now the top rule on the stack has finished (dot is at the end), so we trigger a "pop" operation which pops the top rule and advances the dot in the second-to-top rule, denoting that NP®1 is now completed:</p><p>[NP®1 . VP®2].</p><page local="5" global="277"/><p>Figure 4: Simulation of tree-to-string derivation in Figure 3 in the incremental decoding algorithm. Actions: p, predict; s, scan; c, complete (see Figure 5).</p><doubt alpha="55.2" length="67" tooSmall="False" monospace="0.0">Item I : (s, p) : w;    I: step, s: stack, p: hypothesis, w: weight</doubt><doubt alpha="44.8" length="105" tooSmall="False" monospace="0.0">EquivalenceI: (s, p) ~I: (s', p') iff. s =s'andlastg-1(p)=lastg-1(p')Axiom 0 :([&lt;s&gt;g-1. e&lt;/.&gt;],&lt;s&gt;g-1): 0</doubt><doubt alpha="24.0" length="25" tooSmall="False" monospace="0.0">I: (... [a . n ß], p) : w</doubt><doubt alpha="100.0" length="7" tooSmall="False" monospace="0.0">Predict</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">Scan</doubt><p>Complete Goal</p><doubt alpha="24.7" length="81" tooSmall="False" monospace="0.0">I + |C(r)| : (... [a . n ß] [.f (n,E(r))], p) : w + c(r)I: (... [a . e ß], p) : w</doubt><doubt alpha="41.3" length="46" tooSmall="False" monospace="0.0">I :(... [a e . ß], pe) : w—logPr(e|lastg-1(p))</doubt><doubt alpha="27.7" length="47" tooSmall="False" monospace="0.0">I :(... [a .nß] [Y.],p): wI:(...[an. ß], p) : w</doubt><doubt alpha="27.6" length="29" tooSmall="False" monospace="0.0">|T|:([&lt;s&gt;g-1e&lt;/s&gt;.],p&lt;/s&gt;): w</doubt><doubt alpha="53.3" length="15" tooSmall="False" monospace="0.0">match(n, C (r))</doubt><p>Figure 5: Deductive system for the incremental tree-to-string decoding algorithm. Function <i>last</i>) returns the rightmost g — 1 words (for g-gram LM), and <i>match </i>(n, C (r)) tests matching of rule r against the subtree rooted at node n. C(r) and E(r) are the Chinese and English sides of rule r, and function f (n, E(r)) = [x4 i-&gt; n-var(r) replaces each variable Xj on the English side of the rule with the descendant node n-<i>var </i>(i) under n that matches Xj.</p><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>stack</p></td><td class="cell"><p>hypothesis</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>[     IP@e <b>/ </b>]</p><p><b>1 &lt;s&gt; . </b><b><i>1± </i></b><b><i>&lt;/s&gt;\</i></b></p></td><td class="cell"><p><b>&lt;s&gt;</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[<b>. </b>NP®<footnote anchor="1"/> VP®<footnote anchor="2"/>]</p></td><td class="cell"><p><b>&lt;s&gt;</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[<b>. </b>NP®<footnote anchor="1"/> VP®<footnote anchor="2"/>] [<b>. </b>Bush]</p></td><td class="cell"><p><b>&lt;s&gt;</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP® <b>&lt;/s&gt;] </b>[<b>. </b>NP®<footnote anchor="1"/> VP®<footnote anchor="2"/>] [Bush<b>. </b>]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>c</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP® <b>&lt;/s</b>&gt;][NP@1 <b>. </b>VP®<footnote anchor="2"/>]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [<b>. </b>heklNP®2-2-3 withNP®<footnote anchor="212"/>]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held <b>. </b>NP®<footnote anchor="2"/>-<footnote anchor="2"/>-<footnote anchor="3"/> withNP®<footnote anchor="21"/>'<footnote anchor="2"/>]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held <b>. </b>NP®2-2-3 withNP®<footnote anchor="2"/>'1-2] [<b>. </b>talks]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held <b>. </b>NP®2-2-3 withNP®2-1-2] [talks <b>. </b>]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>c</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [heldNP®2-2-3 <b>. </b>withNP®2-1-2]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held NP®2-2-3 with<b>. </b>NP®2-1-2]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>p</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held NP®2-2-3 with<b>. </b>NP®2-1-2] [<b>. </b>Sharon]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [held NP®2-2-3 with<b>. </b>NP®2-1-2] [Sharon. ]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with Sharon</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>c</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> <b>. </b>VP®<footnote anchor="2"/>] [heldNP®2-2-3 withNP®2-1-2. ]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with Sharon</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>c</p></td><td class="cell"><p><b>[&lt;s&gt;. </b>IP®e <b>&lt;/s&gt;] </b>[NP®<footnote anchor="1"/> VP®<footnote anchor="2"/>. ]</p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with Sharon</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>c</p></td><td class="cell"><p>[    IP®e <b>/ </b>]</p><p><b>1 &lt;s&gt; </b><b><i>1±        . </i></b><b>&lt;/s&gt;|</b></p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with Sharon</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>s</p></td><td class="cell"><p>[   IP®e <b>/ </b>]</p><p><b>1 &lt;s&gt; </b><b><i>i±       </i></b><b>&lt;/s&gt;. I</b></p></td><td class="cell"><p><b>&lt;s&gt; </b>Bush held talks with Sharon <b>&lt;/s&gt;</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="6" global="278"/><p>The next step is to expand VP®<footnote anchor="2"/>, and we use rule r3and push its English-side "VP — held x2 with x1" onto the stack, again with variables replaced by matched nodes:</p><doubt alpha="37.2" length="43" tooSmall="False" monospace="0.0">[NP®1. VP®2] [. held NP®2-2-3with NP®2-1-2]</doubt><p>Note that this is a reordering rule, and the stack al­ways follows the English word order because we generate hypothesis incrementally left-to-right. Fig­ure 4 works out the full example.</p><p>We formalize this algorithm in Figure 5. Each item ( s, p) consists of a stack s and a hypothesis p. Similar to phrase-based dynamic programming, only the last g — 1 words of p are part of the signature for decoding with g-gram LM. Each stack is a list of <i>dotted rules, </i>i.e., rules with dot positions indicting progress, in the style of Earley (1970). We call the last (rightmost) rule on the stack the <i>top rule, </i>which is the rule being processed currently. The symbol af­ter the dot in the top rule is called the <i>next symbol, </i>since it is the symbol to expand or process next. De­pending on the next symbol a, we can perform one of the three actions:</p><p>• if a is a node n, we perform a Predict action which expands n using a rule r that can pattern-match the subtree rooted at n; we push r is to the stack, with the dot at the beginning;</p><p>• if a is an English word, we perform a Scan ac­tion which immediately adds it to the current hypothesis, advancing the dot by one position;</p><p>• if the dot is at the end of the top rule, we perform a Complete action which simply pops stack and advance the dot in the new top rule.</p></subsection><subsection number="3.3" title="Polynomial Time Complexity"><p>Unlike phrase-based models, we show here that incremental decoding runs in average-case polynomial-time for tree-to-string systems.</p><p><b>Lemma 1. </b><i>For an input sentence of </i>n <i>words and its parse tree of depth d, the worst-case complex­ity of our algorithm is </i>f (n,d) = c(cr)d|V|g-1 = O((cr)dng-1), <i>assuming relevant English vocabu­lary </i>| V | = O(n), <i>and where constants c, </i>r <i>and g are the maximum number ofrules matching each tree node, the maximum arityofa rule, andthe language-model order, respectively.</i></p><p><i>Proof. </i>The time complexity depends (in part) on the number of all possible stacks for a tree of depth d. A stack is a list of rules covering a path from the root node to one of the leaf nodes in the following form:</p><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">RiR2Rs</doubt><doubt alpha="8.8" length="34" tooSmall="False" monospace="0.0">[....n1...][....n2...]-..[...^s--^</doubt><p>where n1 = e is the root node and ns is a leaf node, with stack depth s &lt; d. Each rule &gt; 1) ex­pands node ni-1, and thus has c choices by the defi­nition of grammar constant c. Furthermore, each rule in the stack is actually a dotted-rule, i.e., it is associ­ated with a dot position ranging from 0 to r, where r is the arity of the rule (length of English side of the rule). So the total number of stacks is O((cr)d).</p><p>Besides the stack, each state also maintains (g—1) rightmost words of the hypothesis as the language model signature, which amounts to O(|V|g-<footnote anchor="1"/>). So the total number of states is O((cr)d|V|g-<footnote anchor="1"/>). Fol­lowing previous work (Chiang, 2007), we assume a constant number of English translations for each foreign word in the input sentence, so | V| = O(n). And as mentioned above, for each state, there are c possible expansions, so the overall time complexity is f (n, d) = c(cr)d|V |g-<footnote anchor="1"/> = O((cr)dng-1). □</p><p>We do average-case analysis below because the tree depth (height) for a sentence of n words is a random variable: in the worst-case it can be linear in n (degenerated into a linear-chain), but we assume this adversarial situation does not happen frequently, and the average tree depth is O(log n).</p><p><b>Theorem 1. </b><i>Assume for each n, the depth of a parse tree of </i>n <i>words, notated dn, distributes nor­mally with logarithmic mean and variance, i.e., </i>dn ~ N(/xn, ), <i>where </i>/xn = O(log n) <i>and </i>0^ = O(logn), <i>then the average-case complexity ofthe algorithm is </i>h(n) = O(nk log (cr)+g-<footnote anchor="1"/>) <i>for constant k, thus polynomial in n.</i></p><p><i>Proof. </i>From Lemma 1 and the definition of average-case complexity, we have where <i>Ex^D </i>[•] denotes the expectation with respect to the random variable x in distribution D.<page local="7" global="279"/></p><doubt alpha="41.5" length="106" tooSmall="False" monospace="0.0">= Edn^N(,„,,n)[O((cr)dnng-1)], = O(ng-1Edn^N(,„,n)[(cr)dn]), =   O(ng-1Edn^N(M„,ffn)[exp(d„ log(cr))]) (2)</doubt><p>Since dn ~ N(/xn, a<footnote anchor="2"/>) is a normal distribution, dn log(cr) ~ N(/t',a ) is also a normal distribu­tion, where // = /tn log(cr) and <i>a' </i>= <i>an </i>log(cr). Therefore exp(dnlog(cr)) is a log-normal distribu­tion, and by the property of log-normal distribution, its expectation is exp (// + <b>&lt;t</b>'<footnote anchor="2"/>/2). So we have</p><doubt alpha="48.4" length="31" tooSmall="False" monospace="0.0">Edn^N(Mn,^2/2)[exp(d„ log(cr))]</doubt><doubt alpha="23.5" length="17" tooSmall="False" monospace="0.0">= exp(// + a,2/2)</doubt><doubt alpha="42.9" length="35" tooSmall="False" monospace="0.0">= exp (/xnlog(cr) +     log2(cr)/2)</doubt><doubt alpha="50.0" length="46" tooSmall="False" monospace="0.0">= exp (O(log n) log(cr) + O(log n) log2(cr)/2)</doubt><doubt alpha="56.5" length="23" tooSmall="False" monospace="0.0">= exp(O(log n)log2(cr))</doubt><doubt alpha="56.9" length="51" tooSmall="False" monospace="0.0">&lt; exp (k(log n) log2(cr)),      for some constant k</doubt><doubt alpha="59.1" length="22" tooSmall="False" monospace="0.0">= exp (log nklos2(cr))</doubt><doubt alpha="43.8" length="16" tooSmall="False" monospace="0.0">= nklos2(cr).(3)</doubt><p>Plug it back to Equation (2), and we have the average-case complexity</p><doubt alpha="43.2" length="37" tooSmall="False" monospace="0.0">Edn[f (n,d„)]   &lt;   O(ng-1nklog2(cr))</doubt><doubt alpha="28.0" length="25" tooSmall="False" monospace="0.0">=   O(nkl°g2(cr)+5-1).(4)</doubt><p>Since k, c, r and g are constants, the average-case complexity is polynomial in sentence length n. □</p><p>The assumption dn ~ N(O(logn),O(logn)) will be empirically verified in Section 5.</p></subsection><subsection number="3.4" title="Linear-time Beam Search"><p>Though polynomial complexity is a desirable prop­erty in theory, the degree of the polynomial, O(log cr) might still be too high in practice, depend­ing on the translation grammar. To make it linear­time, we apply the beam search idea from phrase-based again. And once again, the only question to decide is the choice of "binning": how to assign each item to a particular bin, depending on their progress?</p><p>While the number of Chinese words covered is a natural progress indicator for phrase-based, it does not work for tree-to-string because, among the three actions, only scanning grows the hypothesis. The prediction and completion actions do not make real progress in terms of <i>words, </i>though they do make progress on the <i>tree.</i><i> </i>So we devise a novel progress indicator natural for tree-to-string translation: the number of tree nodes covered so far. Initially that number is zero, and in a prediction step which ex­pands node n using rule r, the number increments by |C(r)|, the size of the Chinese-side treelet of r. For example, a prediction step using rule r3 in Figure 2 to expand VP®2 will increase the tree-node count by |C(r3)| = 6, since there are six tree nodes in that rule (not counting leaf nodes or variables).</p><p>Scanning and completion do not make progress in this definition since there is no new tree node covered. In fact, since both of them are determin­istic operations, they are treated as "closure" op­erators in the real implementation, which means that after a prediction, we always do as many scan­ning/completion steps as possible until the symbol after the dot is another node, where we have to wait for the next prediction step.</p><p>This method has |T | = O(n) bins where |T | is the size of the parse tree, and each bin holds b items. Each item can expand to c new items, so the overall complexity of this beam search is O(ncb), which is linear in sentence length.</p></subsection></section><section number="4" title="Related Work"><p>The work of Watanabe et al. (2006) is closest in spirit to ours: they also design an incremental decod­ing algorithm, but for the hierarchical phrase-based system (Chiang, 2007) instead. While we leave de­tailed comparison and theoretical analysis to a future work, here we point out some obvious differences:</p><p>1. due to the difference in the underlying trans­lation models, their algorithm runs in O(n<footnote anchor="2"/>b) time with beam search in practice while ours is linear. This is because each prediction step now has O(n) choices, since they need to ex­pand nodes like VP[1, 6] as:</p><doubt alpha="29.6" length="27" tooSmall="False" monospace="0.0">VP[1,6] — PP[1, i] VP[i,6],</doubt><p>where the midpoint i in general has O(n) choices (just like in CKY). In other words, their grammar constant c becomes O(n).</p><p>2. different binning criteria: we use the number of tree nodes covered, while they stick to the original phrase-based idea of number of Chinese words translated;<page local="8" global="280"/></p><p>3. as a result, their framework requires gram­mar transformation into the binary-branching Greibach Normal Form (which is not always possible) so that the resulting grammar always contain at least one Chinese word in each rule in order for a prediction step to always make progress. Our framework, by contrast, works with any grammar.</p><p>Besides, there are some other efforts less closely related to ours. As mentioned in Section 1, while we focus on enhancing syntax-based decoding with phrase-based ideas, other authors have explored the reverse, but also interesting, direction of enhancing phrase-based decoding with syntax-aware reorder­ing. For example Galley and Manning (2008) pro­pose a shift-reduce style method to allow hiearar-chical non-local reorderings in a phrase-based de­coder. While this approach is certainly better than pure phrase-based reordering, it remains quadratic in run-time with beam search.</p><p>Within syntax-based paradigms, cube pruning (Chiang, 2007; Huang and Chiang, 2007) has be­come the standard method to speed up +LM de­coding, which has been shown by many authors to be highly effective; we will be comparing our incre­mental decoder with a baseline decoder using cube pruning in Section 5. It is also important to note that cube pruning and incremental decoding are not mutually exclusive, rather, they could potentially be combined to further speed up decoding. We leave this point to future work.</p><p>Multipass coarse-to-fine decoding is another pop­ular idea (Venugopal et al., 2007; Zhang and Gildea, 2008; Dyer and Resnik, 2010). In particular, Dyer and Resnik (2010) uses a two-pass approach, where their first-pass, —LM decoding is also incremental and polynomial-time (in the style of Earley (1970) algorithm), but their second-pass, +LM decoding is still bottom-up CKY with cube pruning.</p></section><section number="5" title="Experiments"><p>To test the merits of our incremental decoder we conduct large-scale experiments on a state-of-the-art tree-to-string system, and compare it with the stan­dard phrase-based system ofMoses. Furturemore we also compare our incremental decoder with the stan­dard cube pruning approach on the same tree-to-string decoder.</p><subsection number="5.1" title="Data and System Preparation"><p>Our training corpus consists of 1.5M sentence pairs with about 38M/32M words in Chinese/English, re­spectively. We first word-align them by GIZA++ and then parse the Chinese sentences using the Berke­ley parser (Petrov and Klein, 2007), then apply the GHKM algorithm (Galley et al., 2004) to ex­tract tree-to-string translation rules. We use SRILM Toolkit (Stolcke, 2002) to train a trigram language model with modified Kneser-Ney smoothing on the target side of training corpus. At decoding time, we again parse the input sentences into trees, and convert them into translation forest by rule pattern-matching (Mi et al., 2008).</p><p>We use the newswire portion of 2006 NIST MT Evaluation test set (616 sentences) as our develop­ment set and the newswire portion of 2008 NIST MT Evaluation test set (691 sentences) as our test set. We evaluate the translation quality using the BLEU-4 metric, which is calculated by the script mteval-v13a.pl with its default setting which is case-insensitive matching of n-grams. We use the stan­dard minimum error-rate training (Och, 2003) to tune the feature weights to maximize the system's BLEU score on development set.</p><p>We first verify the assumptions we made in Sec­tion 3.3 in order to prove the theorem that tree depth (as a random variable) is normally-distributed with O(log n) mean and variance. Qualitatively, we veri­fied that for most n, tree depth d(n) does look like a normal distribution. Quantitatively, Figure 6 shows that average tree height correlates extremely well with 3.5 log n, while tree height variance is bounded by 5.5 logn.</p></subsection><subsection number="5.2" title="Comparison with Cube pruning"><p>We implemented our incremental decoding algo­rithm in Python, and test its performance on the de­velopment set. We first compare it with the stan­dard cube pruning approach (also implemented in Python) on the same tree-to-string system.<footnote anchor="1"/> Fig-<page local="9" global="281"/></p><footnote label="1">Our implementation of cube pruning follows (Chiang, 2007; Huang and Chiang, 2007) where besides a beam size b of unique +LM items, there is also a hard limit (of 1000) on the</footnote><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">5</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">ino&lt;dW</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">a&gt;ei-ö)</doubt><doubt alpha="50.0" length="8" tooSmall="True" monospace="0.0">TS O O 0</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">Q</doubt><doubt alpha="25.0" length="4" tooSmall="True" monospace="0.0">0 ö)</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">(d</doubt><doubt alpha="14.3" length="7" tooSmall="True" monospace="0.0">—i-1-1-</doubt><p>incremental cube pruning</p><doubt alpha="24.6" length="57" tooSmall="False" monospace="0.0">0     10    20    30    40    50    60 70 Sentence Length</doubt><p>(a) decoding time against sentence length (b) BLEU score against decoding time</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">re</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">o</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">c S</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">LU</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">L</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">m</doubt><doubt alpha="0.0" length="7" tooSmall="False" monospace="0.0">30.1 30</doubt><doubt alpha="0.0" length="24" tooSmall="False" monospace="0.0">29.9 29.8 29.7 29.6 29 5</doubt><doubt alpha="37.0" length="81" tooSmall="False" monospace="0.0">0    0.2   0.4   0.6   0.8    1     1.2 1.4 Avg Decoding Time (secs per sentence)</doubt><p>Figure 7: Comparison with cube pruning. The scatter plot in (a) confirms that our incremental decoding scales linearly with sentence length, while cube pruning super-linearly (b = 50 for both). The comparison in (b) shows that at the same level of translation quality, incremental decoding is slightly faster than cube pruning, especially at smaller beams.</p><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">(n)d(h</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">t</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">p e D</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">ereTr</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">25</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">20</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">15</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">10</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">30</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">40</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">50</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">s)</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">c e(S</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">e</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">im</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">T</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">gindi</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">o c e D</doubt><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">e graer</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">Ave</doubt><doubt alpha="0.0" length="24" tooSmall="False" monospace="0.0">40 35 30 25 20 15 10 5 0</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">n-r</doubt><doubt alpha="20.0" length="5" tooSmall="False" monospace="0.0">M +°°</doubt><doubt alpha="31.2" length="16" tooSmall="False" monospace="0.0">M 10 M 6 M 0 t2s</doubt><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">"TTfT"</doubt><doubt alpha="40.0" length="5" tooSmall="False" monospace="0.0">3 [ft</doubt><p>Sentence Length (n)</p><doubt alpha="34.1" length="41" tooSmall="False" monospace="0.0">0    10   20   30   40 50 Sentence Length</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">60</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">70</doubt><p>Figure 6: Mean and variance of tree depth vs. sentence length. The mean depth clearly scales with 3.5 log n, and the variance is bounded by 5.5 log n.</p><p>Figure 8: Comparison of our incremental tree-to-string decoder with Moses in terms of speed. Moses is shown with various distortion limits (0,6,10, +oo; optimal: 10).</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">0</doubt><p>ure 7(a) is a scatter plot of decoding times versus sentence length (using beam b = 50 for both sys­tems), where we confirm that our incremental de­coder scales linearly, while cube pruning has a slight tendency of superlinearity. Figure 7(b) is a side-by-side comparison of decoding speed versus transla­tion quality (in BLEU scores), using various beam sizes for both systems (b=10-70 for cube pruning, and b= 10— 110 for incremental). We can see that in­cremental decoding is slightly faster than cube prun­ing at the same levels of translation quality, and the difference is more pronounced at smaller beams: for</p><p>number of (non-unique) pops from priority queues.</p><p>example, at the lowest levels of translation quality (BLEU scores around 29.5), incremental decoding takes only 0.12 seconds, which is about 4 times as fast as cube pruning. We stress again that cube prun­ing and incremental decoding are not mutually ex­clusive, and rather they could potentially be com­bined to further speed up decoding.</p></subsection><subsection number="5.3" title="Comparison with Moses"><p>We also compare with the standard phrase-based system of Moses (Koehn et al., 2007), with stan­dard settings except for the ttable limit, which we set to 100. Figure 8 compares our incremental decoder system/decoder BLEU time<page local="10" global="282"/></p><p>Table 3: Final BLEU score and speed results on the test data (691 sentences), compared with Moses and cube pruning. Time is in seconds per sentence, including pars­ing time (0.21s) for the two tree-to-string decoders.</p><p>with Moses at various distortion limits (dmax=0, 6, 10, and +oo). Consistent with the theoretical anal­ysis in Section 2, Moses with no distortion limit (dmax = +oo) scale <i>quadratically, </i>and monotone decoding (dmax = 0) scale linearly. We use MERT to tune the best weights for each distortion limit, and dmax = 10 performs the best on our dev set.</p><p>Table 3 reports the final results in terms of BLEU score and speed on the test set. Our linear-time incremental decoder with the small beam of size b = 10 achieves a BLEU score of 29.54, compara­ble to Moses with the optimal distortion limit of 10 (BLEU score 29.41). But our decoding (including source-language parsing) only takes 0.32 seconds a sentences, which is more than 30 times faster than Moses. With a larger beam of b = 50 our BLEU score increases to 29.96, which is a half BLEU point better than Moses, but still about 15 times faster.</p></subsection></section><section number="6" title="Conclusion"><p>We have presented an incremental dynamic pro­gramming algorithm for tree-to-string translation which resembles phrase-based based decoding. This algorithm is the first incremental algorithm that runs in polynomial-time in theory, and linear-time in practice with beam search. Large-scale experiments on a state-of-the-art tree-to-string decoder confirmed that, with a comparable (or better) translation qual­ity, it can run more than 30 times faster than the phrase-based system of Moses, even though ours is in Python while Moses in C++. We also showed that it is slightly faster (and scale better) than the popular cube pruning technique. For future work we would like to apply this algorithm to forest-based transla­tion and hierarchical system by pruning the first-pass — LM forest. We would also combine cube pruning with our incremental algorithm, and study its perfor­mance with higher-order language models.</p></section><section title="Acknowledgements"><p>We would like to thank David Chiang, Kevin Knight, and Jonanthan Graehl for discussions and the anonymous reviewers for comments. In partic­ular, we are indebted to the reviewer who pointed out a crucial mistake in Theorem 1 and its proof in the submission. This research was supported in part by DARPA, under contract HR0011-06-C-0022 under subcontract to BBN Technologies, and under DOI-NBC Grant N10AP20031, and in part by the National Natural Science Foundation of China, Con­tracts 60736014 and 90920004.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Moses (optimal dmax=10)</p></td><td class="cell"><p>29.41</p></td><td class="cell"><p>10.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tree-to-str: cube pruning (b= 10) tree-to-str: cube pruning (b=20)</p></td><td class="cell"><p>29.51 29.96</p></td><td class="cell"><p>0.65 0.96</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tree-to-str: incremental (b= 10) tree-to-str: incremental (b=50)</p></td><td class="cell"><p>29.54 29.96</p></td><td class="cell"><p>0.32 0.77</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>David Chiang. 2007. Hierarchical phrase-based transla­tion. <i>Computational Linguistics, </i>33(2):201—208.</p><p>Chris Dyer and Philip Resnik. 2010. Context-free re­ordering, finite-state translation. In <i>Proceedings of NAACL.</i></p><p>JayEarley. 1970. An efficient context-free parsing algo­rithm. <i>Communications of the ACM, </i>13(2):94—102.</p><p>Michel Galley and Christopher D. Manning. 2008. A simple and effective hierarchical phrase reordering model. In <i>Proceedings of EMNLP 2008.</i></p><p>Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What's in a translation rule? In <i>Pro­ceedings ofHLT-NAACL, </i>pages 273—280.</p><p>Liang Huang and David Chiang. 2007. Forest rescor-ing: Fast decoding with integrated language models. In <i>Proceedings ofACL, </i>Prague, Czech Rep., June.</p><p>Liang Huang, Kevin Knight, and Aravind Joshi. 2006. Statistical syntax-directed translation with extended domain of locality. In <i>Proceedings ofAMTA, </i>Boston, MA, August.</p><p>Liang Huang. 2007. Binarization, synchronous bina-rization, and target-side binarization. In <i>Proc. NAACL Workshop on Syntax and Structure in Statistical Trans­lation.</i></p><p>Kevin Knight. 1999. Decoding complexity in word-replacement translation models. <i>Computational Lin­guistics, </i>25(4):607—615.</p><p>P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In <i>Proceedings of ACL: demonstration sesion.</i></p><page local="11" global="283"/><p>Philipp Koehn. 2004. Pharaoh: a beam search decoder for phrase-based statistical machine translation mod­els. In <i>Proceedings of 'AMTA, </i>pages 115—124.</p><p>Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-to-string alignment template for statistical machine trans­lation. In <i>Proceedings of COLING-ACL, </i>pages 609— 616.</p><doubt alpha="66.0" length="50" tooSmall="False" monospace="0.0">Haitao Mi, Liang Huang, and Qun Liu. 2008. Forest-</doubt><p>based translation.    In <i>Proceedings of ACL: HLT,</i></p><p>Columbus, OH. Franz Joseph Och. 2003. Minimum error rate training in statistical machine translation. In <i>Proceedings of ACL,</i> for unlexicalized parsing.   In <i>Proceedings of HLT-</i> <i>NAACL.</i><i></i></p><doubt alpha="65.2" length="66" tooSmall="False" monospace="0.0">pages 160—167. Slav Petrov and Dan Klein. 2007. Improved inference</doubt><p>Stuart Shieber, Yves Schabes, and Fernando Pereira. 1995. Principles and implementation of deductive parsing. <i>Journal of Logic Programming, </i>24:3—36.</p><p>Andreas Stolcke. 2002. Srilm - an extensible lan­guage modeling toolkit. In <i>Proceedings ofICSLP, </i>vol­ume 30, pages 901—904.</p><p>Ashish Venugopal, Andreas Zollmann, and Stephen Vo­gel. 2007. An efficient two-pass approach to synchronous-CFG driven statistical MT. In <i>Proceed­ings ofHLT-NAACL.</i></p><p>Taro Watanabe, Hajime Tsukuda, and Hideki Isozaki. 2006. Left-to-right target generation for hierarchical phrase-based translation. In <i>Proceedings ofCOLING-</i> <i>ACL.</i><i></i></p><p>Dekai Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. <i>Computational Linguistics, </i>23(3):377—404.</p><p>Hao Zhang and Daniel Gildea. 2008. Efficient multi­pass decoding for synchronous context free grammars. In <i>Proceedings ofACL.</i></p></references></body></article>