<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="545"/><title>Hierarchical Phrase-Based Translation Grammars Extracted from Alignment Posterior Probabilities</title><pubinfo>Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,pages 545-554, MIT, Massachusetts, USA, 9-11 October 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="de Gispert" givenname="Adrià"><org  name="University of Caen" country="France" city="Caen"/></author><author surname="Pino" givenname="Juan"><org  name="University of Caen" country="France" city="Caen"/></author><author surname="Byrne" givenname="William"><org  name="University of Caen" country="France" city="Caen"/></author></firstpageheader><frontmatter><p><b>Hierarchical Phrase-based Translation Grammars Extracted from Alignment Posterior Probabilities</b></p><p><b>Adria de Gispert, Juan Pino, William Byrne</b></p><p>Machine Intelligence Laboratory Department of Engineering, University of Cambridge Trumpington Street, CB2 1PZ, U.K. {ad4 65|jmp84|wjb3l}@eng.cam.ac.uk</p></frontmatter><abstract>We report on investigations into hierarchi­cal phrase-based translation grammars based on rules extracted from posterior distributions over alignments of the parallel text. Rather than restrict rule extraction to a single align­ment, such as Viterbi, we instead extract rules based on posterior distributions provided by the HMM word-to-wordalignmentmodel. We define translation grammars progressively by adding classes of rules to a basic phrase-based system. We assess these grammars in terms of their expressive power, measured by their ability to align the parallel text from which their rules are extracted, and the quality of the translations they yield. In Chinese-to-English translation, we find that rule extraction from posteriors gives translation improvements. We also find that grammars with rules with only one nonterminal, when extracted from posteri­ors, can outperform more complex grammars extracted from Viterbi alignments. Finally, we show that the best way to exploit source-to-target and target-to-source alignment models is to build two separate systems and combine their output translation lattices. </abstract></header><body><section number="1" title="Introduction"><p>Current practice in hierarchical phrase-based trans­lation extracts regular phrases and hierarchical rules from word-aligned parallel text. Alignment models estimated over the parallel text are used to generate these alignments, but these models are then typically used no further in rule extraction. This is less than ideal because these alignment models, even if they are not suitable for direct use in translation, can still provide a great deal of useful information beyond a single best estimate of the alignment of the parallel text. Our aim is to use alignment models to generate the statistics needed to build translation grammars. The challenge in doing so is to extend the current procedures, which are geared towards the use of a single alignment, to make more of what can be pro­vided by alignment models. The goal is to extract a richer and more robust set of translation rules.</p><p>There are two aspects to hierarchical phrase-based translation grammars which concern us. The first is expressive power, which we take as the ability to generate known reference translations from sen­tences in the source language. This is determined by the degree of phrase movements and the trans­lations allowed by the rules of the grammar. For a grammar with given types of rules, larger rule sets will yield greater expressive power. This motivates studies of grammars based on the rules which are ex­tracted and the movement the grammar allows. The second aspect is of course translation accuracy. If the expressive power is adequate, then the desire is that the grammar assigns a high score to a correct translation.</p><p>We use posterior probabilities over parallel data to address both ofthese aspects. These posteriors allow us to build larger rule sets with improved transla­tion accuracy. Ideally, for a sentence pair we wish to consider all possible alignments between all possi­ble source and target phrases within these sentences. Given a grammar allowing certain types of move­ment, we would then extract all possible parses that are consistent with any alignments of these phrases.</p><page local="2" global="546"/><p>To make this approach feasible, we consider only phrase-to-phrase alignments with a high posterior probability under the alignment models. In this way, the alignment model probabilities guide rule extrac­tion.</p><p>The paper is organized as follows. Section 2 re­views related work on using posteriors to extract phrases, as well as other approaches that tightly in­tegrate word alignment and rule extraction. Sec­tion 3 describes rule extraction based on word and phrase posterior distributions provided by the HMM word-to-word alignment model. In Section 4 we de­fine translation grammars progressively by adding classes of rules to a basic phrase-based system, mo­tivating each rule type by the phrase movement it is intended to achieve. In Section 5 we assess these grammars in terms of their expressive power and the quality of the translations they yield in Chinese-to-English, showing that rule extraction from posteriors gives translation improvements. We also find that the best way to exploit source-to-target and target-to-source alignment models is to build two sepa­rate systems and combine their output translation lattices. Section 6 presents the main conclusions of this work.</p></section><section number="2" title="Related Work"><p>Some authors have previously addressed the limita­tion caused by decoupling word alignment models from grammar extraction. For instance Venugopal et al. (2008) extract rules from n-best lists of align­ments for a syntax-augmented hierarchical system. Alignment n-best lists are also used in Liu et al. (2009) to create a structure called weighted align­ment matrices that approximates word-to-word link posterior probabilities, from which phrases are ex­tracted for a phrase-based system. Alignment pos­teriors have been used before for extracting phrases in non-hierarchical phrase-based translation (Venu-gopal et al., 2003; Kumar et al., 2007; Deng and</p><p>Byrne, 2008).</p><p>In order to simplify hierarchical phrase-based grammars and make translation feasible with rela­tively large parallel corpora, some authors discuss the need for various filters during rule extraction (Chiang, 2007). In particular Lopez (2008) enforces a minimum span of two words per nonterminal,</p><p>Zollmann et al. (2008) use a minimum count thresh­old for all rules, and Iglesias et al. (2009) propose a finer-grained filtering strategy based on rule pat­terns. Other approaches include insisting that target­side rules are well-formed dependency trees (Shen et al., 2008).</p><p>We also note approaches to tighter coupling be­tween translation grammars and alignments. Marcu and Wong (2002) describe a joint-probability phrase-based model for alignment, but the approach is limited due to excessive complexity as Viterbi inference becomes NP-hard (DeNero and Klein, improvement on a phrase-based system where word alignment has been trained with an inversion trans-duction grammar (ITG) rather than IBM models. Pauls et al. (2010) also use an ITG to directly align phrases to nodes in a string-to-tree model. Bayesian methods have been recently developed to induce a grammar directly from an unaligned parallel corpus (Blunsom et al., 2008; Blunsom et al., 2009). Fi­nally, Cmejrek et al. (2009) extract rules directly from bilingual chart parses of the parallel corpus without using word alignments. We take a differ­ent approach in that we aim to start with very strong word alignment models and use them to guide gram­mar extraction.</p><doubt alpha="56.2" length="48" tooSmall="False" monospace="0.0">2008). More recently, Saers et al. (2009) report</doubt></section><section number="3" title="Rule Extraction from Alignment Posteriors"><p>The goal of rule extraction is to generate a set of good-quality translation rules from a parallel cor­pus. Rules are of the form X —(Y,a,~) , where Y, a G {X U <b>T}+ </b>are the source and target sides of the rule, <b>T </b>denotes the set of terminals (words) and ~ is a bijective function<footnote anchor="1"/> relating source and target nonterminals X of each rule (Chiang, 2007). For each Y, the probability over translations a is set by relative frequency over the extracted examples from the corpus.</p><p>We take a general approach to rule extraction, as described by the following procedure. For simplic­ity we discuss the extraction of regular phrases, that is, rules of the form X<i>—(w,w), </i>where w G <b>{T}+. </b>Section 3.3 extends this procedure to rules with nonterminal symbols.<page local="3" global="547"/></p><footnote label="1">This function is defined if there are at least two nontermi­nals, and for clarity of presentation will be omitted in this paper</footnote><p>Given a sentence pair <i>(fJ,</i><i> </i>e{), the extraction al­gorithm traverses the source sentence and, for each sequence of terminals fj.<footnote anchor="2"/>, it considers all possible target-side sequences ei2 as translation candidates. Each target-side sequence that satisfies the align­ment constraints Ca is ranked by the function <i>fR. </i>For practical reasons, a set of selection criteria Cs is then applied to these ranked candidates and defines the set of translations of the source sequence that are extracted as rules. Each extracted rule is assigned a count <i>f</i><i>c</i>.</p><p>In this section we will explore variations of this rule extraction procedure involving alternative def­initions of the ranking and counting functions, fRand <i>f</i><i>c, </i>based on probabilities over alignment mod­els.</p><p>Common practice (Koehn et al., 2003) takes a set of word alignment links <b>L </b>and defines the alignment constraints Ca so that there is a <i>consistency </i>between the links in the (f^<footnote anchor="2"/>, ) phrase pair. This is ex­pressed by V(j,i) G <b>L </b>: (j G [ji <i>,3</i><i>2] </i><i>A </i><i>i</i><i> </i>G [ii <i>,1</i><i>2}) </i><i>V </i><i>(j</i><i> </i><i>G </i><i>[j1,j2</i>] A i G [i1,i2]). If these constraints are met, then alignment probabilities are ignored and <i>f</i><i>R </i>= fc = 1. We call this extraction Viterbi-based, as the set of alignment links is generally obtained after applying a symmetrization heuristic to source-to-target and target-to-source Viterbi alignments.</p><p>In the following section we depart from this ap­proach and apply novel functions to rank and count target-side translations according to their quality in the context of each parallel sentence, as defined by the word alignment models. We also depart from common practice in that we do not use a set of links as alignment constraints. We thus find an increase in the number of extracted rules, and consequently better relative frequency estimates over translations.</p><subsection number="3.1" title="Ranking and Counting Functions"><p>We describe two alternative approaches to modify the functions <i>f</i>R and <i>f</i>c so that they incorporate the probabilities provided by the alignment models.</p><subsubsection number="3.1.1" title="Word-to-word Alignment Posterior Probabilities"><p>Word-to-word alignment posterior probabilities <i>p(l</i><i>ji</i><i>\f</i><i>i </i><i>, e[</i><i>) </i>express how likely it is that the words in source position <i>j </i>and target position <i>i </i>are aligned given a sentence pair. These posteriors can be effi­ciently computed for Model 1, Model 2 and HMM, as described in (Brown et al., 1993; Venugopal et al., 2003; Deng and Byrne, 2008).</p><p>We will use these posteriors in functions to score phrase pairs. For a simple non-disjoint case</p><doubt alpha="56.2" length="16" tooSmall="False" monospace="0.0">(fj2,ei?)we use:</doubt><doubt alpha="50.0" length="4" tooSmall="True" monospace="0.0">j2i2</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">ji</doubt><p>j=j<b>i </b>i=i<b>i</b></p><doubt alpha="45.0" length="20" tooSmall="False" monospace="0.0">P(h\fiJ,e{)i2- ii+ 1</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(1)</doubt><p>which is very similar to the score used for lexical features in many systems (Koehn, 2010), with the link posteriors for the sentence pair playing the role of the Model 1 translation table.</p><p>For a particular source phrase, Equation 1 is not a proper conditional probability distribution over all phrases in the target sentence. Therefore it cannot be used as such without further normalization. Indeed we find that this distribution is too sharp and over­emphasises short phrases, so we use <i>f</i>c = 1. How­ever, it does allow us to rank target phrases as pos­sible translations. In contrast to the common extrac­tion procedure described in the previous section, the ranking approach described here can lead to a much more exhaustive extraction unless selection criteria are applied. These we describe in Section 3.2.</p><p>We note that Equation 1 can be computed us­ing link posteriors provided by alignment models trained on either source-to-target or target-to-source translation directions.</p></subsubsection><subsubsection number="3.1.2" title="Phrase-to-phrase Alignment Posterior Probabilities"><p>Rather than limit ourselves to word-to-word link posteriors we can define alignment proba­bility distributions over phrase alignments. We do this by defining the set of alignments A as A(ji,j2 ; ii ,i2) = <i>{a{ : a</i><i>j </i><i>G </i>[ii, i2 ] iff j G <i>[j1,j2</i>]}, where (j is the random process that de­scribes word-to-word alignments. These are the alignments from which the phrase pair (fjji<footnote anchor="2"/>, eii2i) would be extracted.</p><p>The posterior probability of these alignments given the sentence pair is defined as follows:</p><doubt alpha="46.2" length="26" tooSmall="False" monospace="0.0">r      7J2oJGAP(fl ,alI6!)</doubt><doubt alpha="29.6" length="27" tooSmall="False" monospace="0.0">p{A\e [,!() =^eAl;/'j'f-(2)</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">fR(f</doubt><page local="4" global="548"/><p>Table 1 : Hierarchical phrase-based grammars containing different types of rules. The grammar expressivity is greater as more types of rules are included. In addition to the rules shown in the respective columns, <i>G1, </i>G2 and G3 also contain the rules of G0.</p><p>With IBM models 1 and 2, the numerator and de­nominator in Equation 2 can be computed in terms of posterior link probabilities (Deng, 2005). With the HMM model, the denominator is computed us­ing the forward algorithm while the numerator can be computed using a modified forward algorithm (Deng, 2005).</p><p>These phrase posteriors directly define a proba­bility distribution over the alignments of translation candidates, so we use them both for ranking and scoring extracted rules, that is fR = fc = p. This approach assigns a fractional count to each extracted rule, which allows finer estimation of the forward and backward translation probability distributions.</p></subsubsection></subsection><subsection number="3.2" title="Alignment Constraints and Selection Criteria"><p>In order to keep this process computationally tractable, some extraction constraints are needed. In order to extract a phrase pair <i>(fl? </i>, ), we define the following:</p><p>• Ca requires at least one pair of positions (j, i) :</p><doubt alpha="66.7" length="135" tooSmall="False" monospace="0.0">(jG[j , j2]AiG[i , i2]) with word-to-word link posterior probabilityp(lji\fiJ, e{)&gt;0.5, and that there is no pair ofpositions(j, i):(jG</doubt><doubt alpha="51.6" length="62" tooSmall="False" monospace="0.0">[ji,j2]AiG[ii,i2])V(jG[ji,j2]AiG[ii,i2]) withp(lji\fi7,e{)&gt;0.5</doubt><p>• Cs allows only the k best translation candidates to be extracted. We use k = 3 for regular phrases, and k = 2 for hierarchical rules.</p><p>Note that we do not discard rules according to their scores fc at this point (unlike Liu et al. (2009)), since we prefer to add all phrases from all sentence pairs before carrying out such filtering steps.</p><p>Once all rules over the entire collection of paral­lel sentences have been extracted, we require each rule to occur at least nobs times and with a forward translation probability p(a\Y) &gt; 0.01 to be used for translation.</p></subsection><subsection number="3.3" title="Extraction of Rules with Nonterminals"><p>Extending the procedure previously described to the case of more complex hierarchical rules includ­ing one or even two nonterminals is conceptually straightforward. It merely requires that we traverse the source and target sentences and consider possi­bly disjoint phrase pairs. Optionally, the alignment constraints can also be extended to apply on the non­terminal <i>X</i>.</p><p>Equation 1 is then only modified in the limits of the product and summation, whereas Equation 2 remains unchanged, as long as the set of valid alignments A is redefined. For example, for a rule of the form X —(w X w,w X w), we use A =</p><p>A(j <i>U </i>j2; j3, j4; i i,i2; i3,i4).</p></subsection></section><section number="4" title="Hierarchical Translation Grammar Definition"><p>In this section we define the hierarchical phrase-based synchronous grammars we use for translation experiments. Each grammar is defined by the type of hierarchical rules it contains. The rule type can be obtained by replacing every sequence of terminals by a single symbol 'w', thus ignoring the identity of the words, but capturing its generalized structure and the kind of reordering it encodes (this was defined as rule pattern in Iglesias et al. (2009)).</p><p>A monotonic phrase-based translation grammar Go can be defined as shown in the left-most col­umn of Table 1; it includes all regular phrases, repre­sented by the rule type X—(w,w), and the two glue<page local="5" global="549"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Go</p></td><td class="cell"><p>Gi</p></td><td class="cell"><p><i>G2</i></p></td><td class="cell"><p><i>G3</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>S^(X,X) </i><i>S^(S</i><i> </i><i>X,SX)</i><i> </i><i>X—&gt;(w,w)</i></p></td><td class="cell"><p><i>X^{w </i><i>X,Xw)</i><i> X^(X </i><i>w,w </i><i>X)</i></p></td><td class="cell"><p><i>X^{w </i><i>X,X</i><i> </i><i>w) X^ </i><i>(X</i><i> </i><i>w,w </i><i>X)</i><i> X^(w </i><i>X,w</i><i> </i><i>X)</i></p></td><td class="cell"><p><i>X^{w </i><i>X,X</i><i> </i><i>w) X^ </i><i>(X</i><i> </i><i>w,w </i><i>X)</i><i> X^(w </i><i>X,w</i><i> </i><i>X)</i><i> X^(w X w,w X </i><i>w)</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="39.7" length="126" tooSmall="False" monospace="0.0">(Go)R1: S—(X,X} (Go)R2: X— (s2S3,t2}(Gi)R3: X— (siX,X t3}(Gi)R4: X —(X S4,tiX} (G2)R5: X —(siX,t7X} (G3)R6: X —(siX S4,t5X ta}</doubt><p>Figure 1 : Example of a hierarchical translation grammar and two parsing trees following alternative rule derivations for the input sentence <i>s1s2s3s4.</i></p><p>rules that allow concatenation. Our approach is now simple: we extend this grammar by successively in­corporating sets of hierarchical rules. The goal is to obtain a grammar with few rule types but which is capable of generating a rich set of translation candi­dates for a given input sentence.</p><p>With this in mind, we define the following three grammars, also summarized in Table 1 :</p><doubt alpha="55.6" length="9" tooSmall="False" monospace="0.0">•Gi:= GoU</doubt><p>{ <i>X</i>-^{w <i>X,X </i><i>w)</i><i> </i>, <i>X</i>-^{X <i>w,w X) </i>}. This incorporates reordering capabilities with two rule types that place the unique nonterminal in an opposite position in each language; we call these 'phrase swap rules'. Since all non­terminals are of the same category <i>X,</i><i> </i>nested reordering is possible. However, this needs to happen consecutively, <i>i.e. </i>a swap must apply after a swap, or the rule is concatenated with the glue rule.</p><p>• G2 := Gi LJ{ X-^{w X,w X) }. This adds monotonic concatenation capabilities to the previous translation grammar. The glue rule already allows rule concatenation. However, it does so at the S category, that is, it concate­nates phrases and rules <i>after </i>they have been re­ordered, in order to complete a sentence. With this new rule type, G2 allows phrase/rule con­catenation <i>before </i>reordering with another hier­archical rule. Therefore, nested reordering does not require successive swaps anymore.</p><p>• G3 := G2 U{ X-^{w X w,w X w) }. This adds single nonterminal rules with disjoint ter­minal sequences, which can encode a mono-</p><p>tonic or reordered relationship between them, depending on what their alignment was in the parallel corpus. Although one could expect the movement captured by this phrase-disjoint rule type to be also present in G2 (via two swaps or one concatenation plus one swap), the terminal sequences w may differ.</p><p>Figure 1 shows an example set of rules indicat­ing to which of the previous grammars each rule be­longs, and shows three translation candidates as gen­erated by grammars Gi (left-most tree), G2 (mid­dle tree) and G3 (right-most tree). Note that the middle tree cannot be generated with Gi as it re­quires monotonic concatenation before reordering with rule R<footnote anchor="4"/>.</p><p>The more rule types a hierarchical grammar con­tains, the more different rule derivations and the greater the search space of alternative translation candidates. This is also connected to how many rules are extracted per rule type. Ideally we would like the grammar to be able to generate the correct translation of a given input sentence, without over­generating too many other candidates, as that makes the translation task more difficult.</p><p>We will make use of the parallel data in measuring the ability of a grammar to generate correct transla­tions. By extracting rules from a parallel sentence, we translate them and observe whether the transla­tion grammar is able to produce the parallel target translation. In Section 5.1 we evaluate this for a Chinese-to-English task.</p><page local="6" global="550"/><subsection number="4.1" title="Reducing Grammar Redundancy"><p>Let us discuss grammar G2 in more detail. As de­scribed in the previous section, the motivation for in­cluding rule type X-^{w X,w X) is that the gram­mar be able to carry out monotonic concatenation <i>before </i>applying another hierarchical rule with re­ordering. This movement is permitted by this rule type, but the use of a single nonterminal category X also allows the grammar to apply the concatenation <i>after </i>reordering, that is, immediately before the glue rule is applied. This creates significant redundancy in rule derivations, as this rule type is allowed to act as a glue rule. For example, given an input sentence si s2 and the following simple grammar:</p><doubt alpha="46.2" length="26" tooSmall="False" monospace="0.0">Ro:S— (X,X} Ri: S—(S X,SX}</doubt><doubt alpha="43.9" length="41" tooSmall="False" monospace="0.0">R2: X—(si,ti}R3: X—(s2,t2}R4: X—(siX,tiX}</doubt><p>two derivations are possible: R<footnote anchor="2"/>,Ro,R<footnote anchor="3"/>,Ri and R<footnote anchor="3"/>,R<footnote anchor="4"/>,Ro, and the translation result is identical.</p><p>To avoid this situation we introduce a nonterminal M in the left-hand side of monotonic concatenation rules of G2. All rules are allowed to use nontermi­nals X and M in their right-hand side, except the glue rules, which can only take X. In the context of our example, R<footnote anchor="4"/> is substituted by:</p><doubt alpha="51.4" length="35" tooSmall="False" monospace="0.0">R4a: M —(siX,tiX} R4b: M —(siM,tiM}</doubt><p>so that only the first derivation is possible: R<footnote anchor="2"/>,R<footnote anchor="0"/>,R<footnote anchor="3"/>,R<footnote anchor="1"/>, because applying R<footnote anchor="3"/>,R4&lt;1 yields a non­terminal M that cannot be taken by the glue rule Ro.</p></subsection></section><section number="5" title="Experiments"><p>We report experiments in Chinese-to-English trans­lation. Our system is trained on a subset of the GALE 2008 evaluation parallel text;<footnote anchor="2"/> this is approx­imately 50M words per language. We report trans­lation results on a development set <i>tune-nw </i>and a test set <i>test-nwl. </i>These contain translations pro­duced by the GALE program and portions of the newswire sections of MT02 through MT06. They contain 1,755 sentences and 1,671 sentences respec­tively. Results are also reported on a smaller held-</p><footnote label="2">See http://projects.ldc.upenn.edu/gale/data/catalog.html . We excluded the UN material and the LDC2002E18, LDC2004T08, LDC2007E08 and CUDonga collections.</footnote><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">80</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">60</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">50</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">40</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">30</doubt><p>Figure 2: Percentage of parallel sentences successfully aligned for various extraction methods and grammars.</p><p>out test set <i>test-nw2, </i>containing 60% of the NIST newswire portion of MT06, that is, 369 sentences.</p><p>The parallel texts for both language pairs are aligned using MTTK (Deng and Byrne, 2008). For decoding we use HiFST, a lattice-based decoder im­plemented with Weighted Finite State Transducers (de Gispert et al., 2010). Likelihood-based search pruning is applied if the number of states in the lattice associated with each CYK grid cell exceeds 10,000, otherwise the entire search space is ex­plored. The language model is a 4-gram language model estimated over the English side of the paral­lel text and the AFP and Xinhua portions of the En­glish Gigaword Fourth Edition (LDC2009T13), in­terpolated with a zero-cutoff stupid-backoff (Brants et al., 2007) 5-gram estimated using 6.6B words of English newswire text. In tuning the systems, stan­dard MERT (Och, 2003) iterative parameter estima­tion under IBM BLEU<footnote anchor="3"/> is performed on the devel­opment sets.</p><subsection number="5.1" title="Measuring Expressive Power"><p>We measure the expressive power of the grammars described in the previous section by running the translation system in alignment mode (de Gispert et al., 2010) over the parallel corpus. Conceptually, this is equivalent to replacing the language model by the target sentence and seeing if the system is able to find any candidate. Here the weights assigned to the<page local="7" global="551"/></p><footnote label="3">See ftp://jaguar.ncsl.nist.gov/mt/resources/mteval-v13.pl</footnote><p>Table 2: Chinese-to-English translation results with alternative grammars and extraction methods (lower-cased BLEU shown). Time (secs/word) and prune (times/word) measurements done on <i>tune-nw </i>set.</p><p>rules are irrelevant, as only the ability of the gram­mar to create a desired hypothesis is important.</p><p>We compare the percentage of target sentences that can be successfully produced by grammars G0, Gi , G2 and G3 for the following extraction meth­ods:</p><p>• <b>Viterbi (V). </b>This is the standard extraction method based on a set of alignment links. We distinguish four cases, depending on the model used to obtain the set of links: source-to-target <b>(V-st), </b>target-to-source <b>(V-ts), </b>and two common symmetrization strategies: union <b>(V-union) </b>and grow-diag-final <b>(V-gdf), </b>described</p><p>in (Koehn et al., 2003).</p><p>• <b>Word Posteriors (WP). </b>The extraction method is based on word alignment posteriors de­scribed in Section 3.1.1. These rules can be ob­tained either from the posteriors of the source-to-target <b>(WP-st) </b>or the target-to-source <b>(WP-ts) </b>alignment models. We apply the alignment constraints and selection criteria described in Section 3.2. We do not report alignment per­centages when using phrase posteriors (as de­scribed in Section 3.1.2) as they are roughly identical to the <b>WP </b>case.</p><p>• Finally, in both cases, we also report results when merging the extracted rules in both direc­tions into a single rule set <b>(V-merge </b>and <b>WP-merge).</b></p><p>Figure 2 shows the results obtained for a random selection of 10,000 parallel corpus sentences. As ex­pected, we can see that for any extraction method, the percentage of aligned sentences increases when switching from G0 to Gi , G2 and G3. Posterior-based extraction is shown to outperform standard methods based on a Viterbi set of alignments for nearly all grammars. The highest alignment percent­ages are obtained when merging rules obtained un­der models trained in each direction <b>(WP-merge), </b>approximately reaching 80% for grammar G3.</p><p>The maximum rule span in alignment was al­lowed to be 15 words, so as to be similar to transla­tion, where the maximum rule span is 10 words. Re­laxing this in alignment to 30 words yields approxi­mately 90% coverage for <b>WP-merge </b>under G3.</p><p>We note that if alignment constraints Ca and se­lection criteria Cs were not applied, that is <i>k = </i>oo, then alignment percentages would be 100% even for G0, but the extracted grammar would include many noisy rules with poor generalization power and would suffer from overgeneration.</p></subsection><subsection number="5.2" title="Translation Results"><p>In this section we investigate the translation perfor­mance of each hierarchical grammar, as defined by rules obtained from three rule extraction methods:</p><p>• <b>Viterbi union (V-union). </b>Standard rule extrac­tion from the union of the source-to-target and target-to-source alignment link sets.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Grammar</p></td><td class="cell"><p>Extraction</p></td><td class="cell"><p># Rules</p></td><td class="cell"><p><i>tune-nw</i></p></td><td class="cell"><p><i>test-nwl</i></p></td><td class="cell"><p><i>test-nw2</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>time</i></p></td><td class="cell"><p><i>prune</i></p></td><td class="cell"><p>BLEU</p></td><td class="cell"><p>BLEU</p></td><td class="cell"><p>BLEU</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Gh</i></p></td><td class="cell"><p><b>V-union</b></p></td><td class="cell"><p>979149</p></td><td class="cell"><p>3.7</p></td><td class="cell"><p>0.3</p></td><td class="cell"><p>35.1</p></td><td class="cell"><p>35.6</p></td><td class="cell"><p>37.6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>V-union</b></p></td><td class="cell"><p>613962</p></td><td class="cell"><p>0.4</p></td><td class="cell"><p>0.0</p></td><td class="cell"><p>33.6</p></td><td class="cell"><p>34.6</p></td><td class="cell"><p>36.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Gi</i></p></td><td class="cell"><p><b>WP-st</b></p></td><td class="cell"><p>920183</p></td><td class="cell"><p>0.9</p></td><td class="cell"><p>0.0</p></td><td class="cell"><p>34.3</p></td><td class="cell"><p>34.8</p></td><td class="cell"><p>37.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>PP-st</b></p></td><td class="cell"><p>893542</p></td><td class="cell"><p>1.4</p></td><td class="cell"><p>0.0</p></td><td class="cell"><p>34.4</p></td><td class="cell"><p>35.1</p></td><td class="cell"><p>37.7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>V-union</b></p></td><td class="cell"><p>734994</p></td><td class="cell"><p>1.0</p></td><td class="cell"><p>0.0</p></td><td class="cell"><p>34.5</p></td><td class="cell"><p>35.4</p></td><td class="cell"><p>37.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>G2</i></p></td><td class="cell"><p><b>WP-st</b></p></td><td class="cell"><p>1132386</p></td><td class="cell"><p>5.8</p></td><td class="cell"><p>0.5</p></td><td class="cell"><p>35.1</p></td><td class="cell"><p>36.0</p></td><td class="cell"><p>37.7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>PP-st</b></p></td><td class="cell"><p>1238235</p></td><td class="cell"><p>7.8</p></td><td class="cell"><p>0.7</p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>38.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>V-union</b></p></td><td class="cell"><p>966828</p></td><td class="cell"><p>1.2</p></td><td class="cell"><p>0.0</p></td><td class="cell"><p>34.9</p></td><td class="cell"><p>35.3</p></td><td class="cell"><p>37.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>G3</i></p></td><td class="cell"><p><b>WP-st</b></p></td><td class="cell"><p>2680712</p></td><td class="cell"><p>8.3</p></td><td class="cell"><p>1.1</p></td><td class="cell"><p>35.1</p></td><td class="cell"><p>36.2</p></td><td class="cell"><p>37.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>PP-st</b></p></td><td class="cell"><p>5002168</p></td><td class="cell"><p>10.7</p></td><td class="cell"><p>2.6</p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>38.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="8" global="552"/><p>• <b>Word Posteriors (WP-st). </b>Extraction based on word posteriors as described in Section 3.1.1. The posteriors are provided by the source-to-target alignment model. Alignment constraints and selection criteria of Section 3.2 are applied, with <i>nobs = </i>2.</p><p>• <b>Phrase Posteriors (PP-st). </b>Extraction based on phrase alignment posteriors, as described in Section 3.1.2, with fractional counts pro­portional to the phrase probability under the source-to-target alignment model. Alignment constraints and selection criteria of Section 3.2 are applied, with <i>nobs </i>= 0.2.</p><p>Table 2 reports the translation results, as well as the number of extracted rules in each case. It also shows the following decoding statistics as measured on the <i>tune-nw </i>set: decoding time in seconds per in­put word, and number of instances of search pruning (described in Section 5) per input word.</p><p>As a contrast, we extract rules according to the heuristics introduced in (Chiang, 2007) and apply the filters described in (Iglesias et al., 2009) to gen­erate a standard hierarchical phrase-based grammar GH. This uses rules with up to two nonadjacent non­terminals, but excludes identical rule types such as X-^{w X,w X) or X-^{w X1 w X2,w X1 w X2), which were reported to cause computational difficul­ties without a clear improvement in translation (Igle­sias et al., 2009).</p><p><i>Grammar expressivity. </i>As expected, for the stan­dard extraction method (see rows entitled <b>V-union), </b>grammar Gi is shown to underperform all other grammars due to its structural limitations. On the other hand, grammar G2 obtains much better scores, nearly generating the same translation quality as the baseline grammar GH. Finally, G3 does not prove able to outperform G2, which suggests that the phrase-disjoint rules with one nonterminal are redundant for the translation grammar.</p><p><i>Rule extraction method. </i>For all grammars, we find that the proposed extraction methods based on alignment posteriors outperform standard Viterbi­based extraction, with improvements ranging from 0.5 to 1.1 BLEU points for <i>test-nwl </i>(depending on the grammar) and from 1.0 to 1.5 for <i>test-nw2. </i>In all cases, the use of phrase posteriors <b>PP </b>is the best option. Interestingly, we find that G2 extracted with <b>WP </b>and <b>PP </b>methods outperforms the more complex GH grammar as obtained from Viterbi alignments.</p><p><i>Rule set statistics. </i>For grammar G2 evaluated on the <i>tune-nw </i>set, standard Viterbi-based extrac­tion produces 0.7M rules, whereas the WP and PP extraction methods yield 1.1M and 1.2M rules re­spectively. We further analyse the sets of rules X—&gt;{7,a,~) in terms of the number of distinct source and target sequences 7 and <i>a </i>which are ex­tracted. Viterbi extraction yields 82k distinct source sequences whereas the WP and PP methods yield 116k and 146k sequences, respectively. In terms of the average number of target sequences for each source sequence, Viterbi extraction yields an aver­age of 8.7 while WP and PP yield 9.7 and 8.4 rules on average. This shows that method <b>PP </b>yields wider coverage but with sharper forward rule translation probability distributions than method <b>WP, </b>as the av­erage number of translations per rule is determined by the p(a|7) &gt; 0.01 threshold mentioned in Sec­tion 3.2.</p><p><i>Decoding time and pruning in search. </i>In connec­tion to the previous comments, we find an increased need for search pruning, and subsequently slower decoding speed, as the search space grows larger with methods <b>WP </b>and <b>PP. </b>A larger search space is created by the larger rule sets, which allows the sys­tem to generate new hypotheses of better quality.</p></subsection><subsection number="5.3" title="Rule Concatenation in Grammar G 2"><p>In Section 4.1 we described a strategy to reduce grammar redundancy by introducing an additional nonterminal M for monotonic concatenation rules. We find that without this distinction among nonter­minals, search pruning and decoding time are in­creased by a factor of 1.5, and there is a slight degra­dation in BLEU (~0.2) as more search errors are in­troduced.</p><p>Another relevant aspect of this grammar is the ac­tual rule type selected for monotonic concatenation. We described using type X-^{w X,w X) (con­catenation on the right), but one could also include X-^{X w,X w) (concatenation on the left), or both, for the same purpose. We evaluated the three alter­natives and found that scores are identical when ei­ther including right or left concatenation types, but including both is harmful for performance, as the need to prune and decoding time increase by a factor of 5 and 4, respectively, and we observe again a slight degradation in performance.<page local="9" global="553"/></p><p>Table 3: Translationresults undergrammarG2 withindi-vidual rule sets, merged rule sets, and rescoring and sys­tem combination with lattice-based MBR (lower-cased BLEU shown)</p></subsection><subsection number="5.4" title="Symmetrizing Alignments ofParallel Text"><p>In this section we investigate extraction from align­ments (and posterior distributions) over parallel text which are generated using alignment models trained in the source-to-target <b>(st) </b>and target-to-source <b>(ts) </b>directions. Our motivation is that symmetrization strategies have been reported to be beneficial for Viterbi extraction methods (Och and Ney, 2003;</p><p>Koehn et al., 2003).</p><p>Results are shown in Table 3 for grammar G2. We find that rules extracted under the source-to-target alignment models <b>(V-st, WP-st </b>and <b>PP-st) </b>consis­tently perform better than the <b>V-ts, WP-ts </b>and <b>PP-ts </b>cases. Also, for Viterbi extraction we find that the source-to-target <b>V-st </b>case performs better than any of the symmetrization strategies, which contradicts previous findings for non-hierarchical phrase-based systems(Koehn et al., 2003).</p><p>We use the <b>PP </b>rule extraction method to extract two sets of rules, under the <b>st </b>and <b>ts </b>alignment mod­els respectively. We now investigate two ways of merging these sets into a single grammar for trans­lation. The first strategy is <b>PP-merge </b>and merges both rule sets by assigning to each rule the maximum count assigned by either alignment model. We then extend the previous strategy by adding three binary feature functions to the system, indicating whether the rule was extracted under the <b>'st' </b>model, the <b>'ts' </b>model or both. The motivation is that MERT can weight rules differently according to the alignment model they were extracted from. However, we do not find any improvement with either strategy.</p><p>Finally, we use linearised lattice minimum Bayes-risk decoding (Tromble et al., 2008; Blackwood et al., 2010) to combine translation lattices (de Gis-pert et al., 2010) as produced by rules extracted under each alignment direction (see rows named</p><p>LMBR<b>(V-st,V-ts) </b>and LMBR<b>(PP-st,PP-ts)). </b>Gains are consistent when comparing this to applying LMBR to each of the best individual systems (rows named LMBR<b>(V-st) </b>and LMBR<b>(PP-st)).</b><b> </b>Overall, the best-performing strategy is to extract two sets of translation rules under the phrase pair posteriors in each translation direction, and then to perform trans­lation twice and merge the results.</p></subsection></section><section number="6" title="Conclusion"><p>Rule extraction based on alignment posterior proba­bilities can generate larger rule sets. This results in grammars with more expressive power, as measured by the ability to align parallel sentences. Assign­ing counts equal to phrase posteriors produces bet­ter estimation of rule translation probabilities. This results in improved translation scores as the search space grows.</p><p>This more exhaustive rule extraction method per­mits a grammar simplification, as expressed by the phrase movement allowed by its rules. In particular a simple grammar with rules of only one nontermi­nal is shown to outperform a more complex gram­mar built on rules extracted from Viterbi alignments. Finally, we find that the best way to exploit align­ment models trained in each translation direction is to extract two rule sets based on alignment posteri­ors, translate the input independently with each rule set and combine translation output lattices.</p></section><section title="Acknowledgments"><p>This work was supported in part by the GALE pro­gram of the Defense Advanced Research Projects<page local="10" global="554"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Rule Extraction</p></td><td class="cell"><p><i>tune-nw</i></p></td><td class="cell"><p><i>test-nwl</i></p></td><td class="cell"><p><i>test-nw2</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>V<b>-st</b></p></td><td class="cell"><p>34.7</p></td><td class="cell"><p>35.6</p></td><td class="cell"><p>37.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>V<b>-ts</b></p></td><td class="cell"><p>34.0</p></td><td class="cell"><p>34.8</p></td><td class="cell"><p>36.6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>V<b>-union</b></p></td><td class="cell"><p>34.5</p></td><td class="cell"><p>35.4</p></td><td class="cell"><p>37.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>V-gdf</p></td><td class="cell"><p>34.4</p></td><td class="cell"><p>35.3</p></td><td class="cell"><p>37.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>WP-st</b></p></td><td class="cell"><p>35.1</p></td><td class="cell"><p>36.0</p></td><td class="cell"><p>37.7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>WP-ts</b></p></td><td class="cell"><p>34.5</p></td><td class="cell"><p>35.0</p></td><td class="cell"><p>37.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>PP-st</b></p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>38.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>PP-ts</b></p></td><td class="cell"><p>34.8</p></td><td class="cell"><p>35.3</p></td><td class="cell"><p>37.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>PP</b>-merge</p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>38.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>PP</b>-merge<b>-MERT</b></p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>38.3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LMBR(V<b>-st)</b></p></td><td class="cell"><p>35.0</p></td><td class="cell"><p>35.8</p></td><td class="cell"><p>38.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LMBR(V<b>-st</b>,V<b>-ts)</b></p></td><td class="cell"><p>35.5</p></td><td class="cell"><p>36.3</p></td><td class="cell"><p>38.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LMBR<b>(PP-st)</b></p></td><td class="cell"><p>36.1</p></td><td class="cell"><p>36.8</p></td><td class="cell"><p>38.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LMBR<b>(PP-st,PP-ts)</b></p></td><td class="cell"><p>36.4</p></td><td class="cell"><p>36.9</p></td><td class="cell"><p>39.3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="59.2" length="49" tooSmall="False" monospace="0.0">Agency, Contract No. HR0011- 06-C-0022.References</doubt><p>Graeme Blackwood, Adria de Gispert, and William Byrne. 2010. Efficient Path Counting Transducers for Minimum Bayes-Risk Decoding of Statistical Ma­chine Translation Lattices. In <i>Proceedings of ACL, Short Papers, </i>pages 27-32. Phil Blunsom, Trevor Cohn, and Miles Osborne. 2008. Bayesian Synchronous Grammar Induction. In <i>Ad­vances in Neural Information ProcessingSystems, </i>vol­ume 21, pages 161-168. Phil Blunsom, Trevor Cohn, Chris Dyer, and Miles Os-borne. 2009. A gibbs sampler for phrasal synchronous grammar induction. In <i>Proceedings ofthe ACL, </i>pages 782-790.</p><p>Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, and Jeffrey Dean. 2007. Large language models in machine translation. In <i>Proceedings ofEMNLP-ACL,</i> pages 858-867.</p><p>P. Brown, S. Della Pietra, V. Della Pietra, and R. Mercer. 1993. The mathematics of statistical machine trans­lation: Parameter estimation. <i>Computational Linguis­tics, </i>19(2):263-311. David Chiang. 2007. Hierarchical phrase-based transla­tion. <i>Computational Linguistics, </i>33(2):201-228. Martin Cmejrek, Bowen Zhou, and Bing Xiang. 2009. Enriching SCFG Rules Directly From Efficient Bilin­gual Chart Parsing. In <i>Proceedings ofIWSLT, </i>pages 136-143.</p><p>Adria de Gispert, Gonzalo Iglesias, Graeme Blackwood, Eduardo R. Banga, and William Byrne. 2010. Hier­archical phrase-based translation with weighted finite state transducers and shallow-n grammars. <i>Computa­tional Linguistics, </i>36(3). John DeNero and Dan Klein. 2008. The complexity of phrase alignment problems. In <i>Proceedings ofACL-HLT, Short Papers, </i>pages 25-28. Yonggang Deng and William Byrne. 2008. HMM word and phrase alignment for statistical machine transla­tion. <i>IEEE Transactions on Audio, Speech, and Lan­guage Processing, </i>16(3):494-507. Yonggang Deng.  2005.  <i>Bitext Alignment for Statisti­cal Machine Translation. </i>Ph.D. thesis, Johns Hopkins University.</p><p>Gonzalo Iglesias, Adria de Gispert, Eduardo R. Banga, and William Byrne. 2009. Rule filtering by pattern for efficient hierarchical translation. In <i>Proceedings of</i> <i>the EACL, </i>pages 380-388.</p><p>Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In <i>Proceedings of HLT/NAACL, </i>pages 48-54.</p><p>Philipp Koehn. 2010. <i>Statistical Machine Translation.</i></p><p>Cambridge University Press. Shankar Kumar, Franz J. Och, and Wolfgang Macherey. 2007.   Improving word alignment with bridge lan­guages. In <i>Proceedings ofEMNLP-CoNLL, </i>pages 42­50.</p><p>Yang Liu, Tian Xia, Xinyan Xiao, and Qun Liu. 2009. Weighted alignment matrices for statistical machine translation. In <i>Proceedings ofEMNLP, </i>pages 1017­1026.</p><p>Adam Lopez. 2008. Tera-scale translation models via pattern matching. In <i>Proceedings </i><i>ofCOLING,</i><i> </i>pages 505-512.</p><p>Daniel Marcu and William Wong. 2002. A phrase-based, joint probability model for statistical machine transla­tion. In <i>Proceedings of EMNLP, </i>pages 133-139. Franz J. Och and Hermann Ney.  2003. A systematic comparison of various statistical alignment models. <i>Computational Linguistics, </i>29(1):19-51. Franz J. Och. 2003. Minimum error rate training in sta­tistical machine translation. In <i>Proceedings </i><i>ofACL,</i><i> </i>pages 160-167. Adam Pauls, Dan Klein, David Chiang, and Kevin Knight. 2010. Unsupervised syntactic alignment with inversion transduction grammars. In <i>Proceedings of the HLT-NAACL, </i>pages 118-126. Markus Saers and Dekai Wu. 2009. Improving phrase-based translation via word alignments from stochastic inversion transduction grammars. In <i>Proceedings of the HLT-NAACL Workshop on Syntax and Structure in Statistical Translation, </i>pages 28-36. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algo­rithm with a target dependency language model. In <i>Proceedings ofACL-HLT, </i>pages 577-585. Roy Tromble, Shankar Kumar, Franz J. Och, and Wolf­gang Macherey. 2008. Lattice Minimum Bayes-Risk decoding for statistical machine translation. In <i>Pro­ceedings ofEMNLP, </i>pages 620-629. Ashish Venugopal, Stephan Vogel, and Alex Waibel. 2003.   Effective phrase translation extraction from alignment models. In <i>Proceedings of ACL, </i>pages 319­326.</p><p>Ashish Venugopal, Andreas Zollmann, Noah A. Smith, and Stephan Vogel. 2008. Wider pipelines: N-best alignments and parses in mt training. In <i>Proceedings ofAMTA, </i>pages 192-201. Andreas Zollmann, Ashish Venugopal, Franz J. Och, and Jay Ponte. 2008. A systematic comparison of phrase-based, hierarchical and syntax-augmented statistical MT. In <i>Proceedings of COLING, </i>pages 1145-1152.</p></section></body></article>