<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="658"/><title>Can Chinese Phonemes Improve Machine Transliteration?: A Comparative Study of English-to-Chinese Transliteration Models</title><author surname="Oh" givenname="Jong-Hoon"><org  name="National Institute of Information and Communications Technology" country="Japan" city="Kyoto"/></author><author surname="Uchimoto" givenname="Kiyotaka"><org  name="National Institute of Information and Communications Technology" country="Japan" city="Kyoto"/></author><author surname="Torisawa" givenname="Kentaro"><org  name="National Institute of Information and Communications Technology" country="Japan" city="Kyoto"/></author></firstpageheader><frontmatter><p><b>Can Chinese Phonemes Improve Machine Transliteration?: A Comparative Study of English-to-Chinese Transliteration Models</b></p><p><b>Jong-Hoon Oh, Kiyotaka Uchimoto, and Kentaro Torisawa</b></p><p>Language Infrastructure Group, MASTAR Project, National Institute of Information and Communications Technology (NICT) 3-5 Hikaridai Seika-cho, Soraku-gun, Kyoto 619-0289 Japan {rovellia,uchimoto,torisawa}@nict.go.jp</p></frontmatter><abstract>Inspired by the success of English grapheme-to-phoneme research in speech synthesis, many researchers have pro­posed phoneme-based English-to-Chinese transliteration models. However, such ap­proaches have severely suffered from the errors in Chinese phoneme-to-grapheme conversion. To address this issue, we propose a new English-to-Chinese transliteration model and make system­atic comparisons with the conventional models. Our proposed model relies on the joint use of Chinese phonemes and their corresponding English graphemes and phonemes. Experiments showed that Chinese phonemes in our proposed model can contribute to the performance im­provement in English-to-Chinese translit­eration. </abstract></header><body><section number="1" title="Introduction 1.1 Motivation"><p>Transliteration, <i>i.e.,</i><i> </i><i>phonetic</i><i> </i><i>translation,</i><i> </i>is com­monly used to translate proper names and techni­cal terms across languages. A variety of English-to-Chinese machine transliteration models has been proposed in the last decade (Meng et al., 2001; Gao et al, 2004; Jiang et al, 2007; Lee and Chang, 2003; Li et al., 2004; Li et al, 2007; Wan and Verspoor, 1998; Virga and Khudanpur, 2003). They can be categorized into those based on Chinese phonemes (Meng et al., 2001; Gao et al, 2004; Jiang et al, 2007; Lee and Chang, 2003; Wan and Verspoor, 1998; Virga and Khu­danpur, 2003) and those that don't rely on Chinese phonemes (Li et al., 2004; Li et al, 2007).</p><p>Inspired by the success of English grapheme-to-phoneme research in speech synthesis, many re­searchers have proposed phoneme-based Englishto-Chinese transliteration models. In these ap­proaches, Chinese phonemes are generated from English graphemes or phonemes, and then the Chinese phonemes are converted into Chinese graphemes (or characters), where Chinese Pinyin strings<footnote anchor="1"/> are used for representing a syllable-level Chinese phoneme sequence. Despite its high ac­curacy in generating Chinese phonemes from En­glish, this approach has severely suffered from er­rors in Chinese phoneme-to-grapheme conversion, mainly caused by Chinese homophone confusion - one Chinese Pinyin string can correspond to sev­eral Chinese characters (Li et al., 2004). For ex­ample, the Pinyin string "LI" corresponds to such different Chinese characters as ^IJ, ftj, and JL For this reason, it has been reported that English-to-Chinese transliteration without Chinese phonemes outperforms that with Chinese phonemes (Li et al, 2004).</p><p>Then "Can Chinese phonemes improve English-to-Chinese transliteration, if we can re­duce the errors in Chinese phoneme-to-grapheme conversion?" Our research starts from this question.</p><subsection number="1.2" title="Our Approach"><p>Previous approaches using Chinese phonemes have relied only on Chinese phonemes in Chi­nese phoneme-to-grapheme conversion. However, the simple use of Chinese phonemes doesn't al­ways provide a good clue to reduce the ambi­guity in Chinese phoneme-to-grapheme conver­sion. Let us explain with an example, the Chinese transliteration of <i>Greeley</i><i> </i>in Table 1, where Chi­nese phonemes are represented in terms of Chi­nese Pinyin strings and English phonemes are rep­resented by <i>ARPAbet</i><i> </i>symbols<footnote anchor="2"/>.</p><p>In Table 1, Chinese Pinyin string "LI" corre­sponds to two different Chinese characters, ÜL and</p><footnote label="1">Pinyin, the most commonly used Romanization sys­tem for Chinese characters, faithfully represents Chinese</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">658</doubt><p><i>Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, </i>pages 658-667, Singapore, 6-7 August 2009. ©2009 ACL and AFNLP<page local="2" global="659"/></p><p>Table 1 : Chinese Pinyin string "LI" and its corre­sponding Chinese characters in Chinese transliter­ation of <i>Greeley</i> ^IJ. It seems difficult to find evidence for select­ing the correct Chinese character corresponding to each Chinese Pinyin string "LI" by just looking at the sequence of Chinese Pinyin strings "GE LI LI." However, English graphemes <i>(ree</i><i> </i>and <i>ley)</i><i> </i>or phonemes ("R IY" and "L IY") corresponding to Chinese Pinyin string "LI", especially their conso­nant parts (r and / in the English graphemes and "R" and "L" in the English phonemes), provide strong evidence to resolve the ambiguity. Thus, we can easily find rules for the conversion from Chinese Pinyin string "LI" to ÜL and ^IJ as follows:</p><doubt alpha="50.0" length="14" tooSmall="False" monospace="0.0">• ("RIY",LI)ÜL</doubt><doubt alpha="28.6" length="21" tooSmall="False" monospace="0.0">• ( "L IY", LI ) -»■M</doubt><p>Based on the observation, we propose an English-to-Chinese transliteration model based on the joint use of Chinese phonemes and their corre­sponding English graphemes and phonemes. We define a set of English-to-Chinese transliteration models and categorize them into the following three classes:</p><p>• Mi: Models Independent of Chinese phonemes</p><p>• Ms : Models based on Simple use of Chinese phonemes</p><p>• Mj: Models based on Joint use of Chi­nese phonemes and English graphemes and phonemes that correspond to our proposed model.</p><p>Our comparison among the three types of translit­eration models can be summarized as follows.</p><p>• The Mi models relying on either English graphemes or phonemes could not outper­form those based on both English graphemes and phonemes.</p><p>phonemes and syllables (Yin and Felley, 1990).</p><footnote label="2">http :// www.cs.emu.edu/~laura/pages/ arpabet.ps</footnote><p>• The Ms models always showed the worst performance due to the severe error rate in Chinese phoneme-to-grapheme conversion.</p><p>• The Mj models significantly reduced er­rors in Chinese phoneme-to-grapheme con­version; thus they achieved the best perfor­mance.</p><p>The rest of this paper is organized as follows. Section 2 introduces the notations used through­out this paper. Section 3 describes the translitera­tion models we compared. Section 4 describes our tests and results. Section 5 concludes the paper with a summary.</p></subsection></section><section number="2" title="Preliminaries"><p>Let <i>Eg </i>be an English word composed of « English graphemes, and let <i>Ep </i>be a sequence of English phonemes that represents the pronunciation of <i>Eg-</i>Let <i>Cq </i>be a sequence of Chinese graphemes cor­responding to the Chinese transliteration of <i>Eg, </i>and let <i>Cp </i>be a sequence of Chinese phonemes that represents the pronunciation <i>oïCg-</i></p><p><i>Cp</i><i> </i>corresponds to a sequence of the Chinese Pinyin strings of <i>Cg- </i>Because a Chinese Pinyin string represents the pronunciation of a sylla­ble consisting of consonants and vowels, we di­vide a Chinese Pinyin string into consonant and vowel parts like "L+I", "L+I+N", and "SH+A." In this paper, we define a <i>Chinese</i><i> </i><i>phoneme</i><i> </i>as the vowel and consonant parts in a Chinese Pinyin string (e.g., "L", "SH", and "I"). A Chi­nese character usually corresponds to multiple English graphemes, English phonemes, and Chi­nese phonemes (i.e., ÜL corresponds to English graphemes <i>ree,</i><i> </i>English phonemes "R IY", and Chinese phonemes "L I" in Table 1). To repre­sent these many-to-one correspondences, we use the well-known BIO labeling scheme to represent a Chinese character, where B and I represent the beginning and inside/end of the Chinese charac­ters, respectively, and O is not used. Each Chi­nese phoneme corresponds to a Chinese character with B and I labels. For example, Chinese charac­ter "II" in Table 1 can be represented as "J|:B" and "11:1", where "J|:B" and "11:1" correspond to Chinese phonemes "L" and "I", respectively. In this paper, we define a <i>Chinese</i><i> </i><i>grapheme</i><i> </i>as a Chi­nese character represented with a BIO label, e.g., "JL:B" and "ML"<page local="3" global="660"/></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">659</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>English grapheme</p></td><td class="cell"><p><i>g</i></p></td><td class="cell"><p><i>ree</i></p></td><td class="cell"><p><i>ley</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>English phoneme</p></td><td class="cell"><p>G</p></td><td class="cell"><p>RIY</p></td><td class="cell"><p>LIY</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Chinese Pinyin</p></td><td class="cell"><p>GE</p></td><td class="cell"><p>LI</p></td><td class="cell"><p>LI</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Chinese character</p></td><td class="cell"><p></p></td><td class="cell"><p>M</p></td><td class="cell"><p><i>M</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Table 2: <i>egi</i><i> </i>and its corresponding <i>epi,</i><i> </i><i>cpi,</i><i> </i>and <i>cgi</i><i> </i>in <i>Greeley</i><i> </i>and its corresponding Chinese translit­eration "fèmf ü"</p><p>Then <i>Ep, Cp, </i>and Cg can be segmented into a series of sub-strings, each of which corresponds to an English grapheme in <i>Eg- </i>We can thus write where <i>egi,</i><i> </i><i>epi,</i><i> </i><i>cpi,</i><i> </i>and <i>cgi</i><i> </i>represent the <i>i</i><i> </i>English grapheme, English phonemes, Chinese phonemes, and Chinese graphemes corresponding to <i>egi,</i><i> </i>respectively.</p><p>Based on the definition, we model English-to-Chinese transliteration so that each English grapheme is tagged with its corresponding En­glish phonemes, Chinese phonemes, and Chinese graphemes. Table 2 illustrates <i>egi,</i><i> </i><i>epi,</i><i> </i><i>cpi,</i><i> </i>and <i>cgi</i><i> </i>with the same example listed in Table 1 (En­glish word <i>Greeley</i><i> </i>and its corresponding Chinese transliteration "fê-JlW)<footnote anchor="3"/>, where <i>&lt;fi </i>represents an empty string.</p></section><section number="3" title="Transliteration Model"><p>We defined eighteen transliteration models to be compared. These transliteration models are clas­sified into three classes, Mi, Ms, and Mj as de­scribed in Section 1.2; each class has three basic transliteration models and three hybrid ones. In this section, we first describe the basic translit­eration models in each class by focusing on the main difference among the three classes and then describe the hybrid transliteration models.</p><footnote label="3">We performed alignment between Eg  and Ep  and be­tween Ep  and Cp  in a similar manner presented in Li et al. (2004). Then the two alignment results were merged using Ep  as a pivot. Finally, we made a correspondence relation among egi, epi, cpi, and cgi using the merged alignment re­sult and the Pinyin table.</footnote><subsection number="3.1" title="Basic Transliteration Models"><p>The basic transliteration models in each class are denoted as <i>M(x, y).</i></p><doubt alpha="50.0" length="12" tooSmall="False" monospace="0.0">• (x, y)gXxY</doubt><doubt alpha="58.8" length="17" tooSmall="False" monospace="0.0">•xeX= {EG,Ep,EGp}</doubt><doubt alpha="55.6" length="18" tooSmall="False" monospace="0.0">•yeY= {cf),Cp,JCp}</doubt><p><i>x</i><i> </i>is an English-side parameter representing En­glish grapheme <i>(Eg), </i>English phoneme <i>(Ep), </i>and the joint use of English grapheme and phoneme <i>(Egp = (Eg,Ep)) </i>that contributes to generat­ing Chinese phonemes or Chinese graphemes in a transliteration model, y is a Chinese-phoneme parameter that represents a way of using Chinese phonemes to generate Chinese graphemes in a transliteration model. Since <i>M(x,&lt;fi) </i>represents a transliteration model that does not rely on Chi­nese phonemes, it falls into Mi, while <i>M(x, Cp) </i>corresponds to a transliteration model in Ms that only uses Chinese phonemes in Chinese phoneme-to-grapheme conversion. <i>M(x, JCp) </i>is a translit­eration model in the Mj class that generates Chi­nese transliterations based on joint use of <i>x</i><i> </i>and Chinese phoneme <i>Cp, </i>where <i>x </i>g <i>X. </i>Thus, <i>M(x,JCp) </i>can be rewritten as <i>M(x, (x,Cp)), </i>where the joint representation of <i>x </i>and <i>Cp, (x, Cp), </i>is used in Chinese phoneme-to-grapheme conversion. The three basic models in Mj can be interpreted as follows:</p><doubt alpha="40.6" length="32" tooSmall="False" monospace="0.0">• M (EG, JCp) = M (Eg, {Eg, Cp))</doubt><doubt alpha="48.1" length="27" tooSmall="False" monospace="0.0">• M(EP,JCp)= M(EP, (Ep,Cp))</doubt><doubt alpha="53.3" length="30" tooSmall="False" monospace="0.0">• M(Egp,JCp) = M(Egp,{Egp,Cp))</doubt><p><i>M (</i><i>Eg</i><i>, JCp) </i>directly converts English graphemes into Chinese phonemes without the help of English phonemes and then gener­ates Chinese transliterations based on the joint representation of English graphemes and Chi­nese phonemes. The main difference between <i>M(Ep,JCP) </i>and <i>M(EGp,JCP) </i>lies in the use of English graphemes to generate Chinese phonemes and graphemes. English graphemes are only used in English grapheme-to-phoneme conversion, and English phonemes play a crucial role for generating Chinese transliteration in <i>M(Ep, JCp). </i>Chinese phoneme-to-grapheme conversion that relies on the joint use of English graphemes,  English phonemes,  and Chinese <i>Pm(eg,jcp)</i> phonemes is the key feature of <i>M(Egp</i><i>, JCp).</i><page local="4" global="661"/><i> </i>Because M(a;, <i>JCp) </i>can be interpreted as <i>M(x,(x,Cp)), </i>English-side parameter <i>x </i>de­termines the English graphemes and phonemes, or both jointly used with Chinese phonemes in Chinese phoneme-to-grapheme conversion. Then we can represent the three basic transliteration models as in Eqs. (l)-(3), where <i>P(Cg\Eg</i><i>, Cp), </i><i>P</i><i>{CG\Ep, Cp), </i>and <i>P</i><i>{CG\EG, </i><i>EP, </i><i>CP) </i>are the key points in our proposed models, Mj.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">660</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>i</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>3</p></td><td class="cell"><p>4</p></td><td class="cell"><p>5</p></td><td class="cell"><p>6</p></td><td class="cell"><p>7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Eg</i></p></td><td class="cell"><p><i>g</i></p></td><td class="cell"><p>r</p></td><td class="cell"><p><i>e</i></p></td><td class="cell"><p><i>e</i></p></td><td class="cell"><p>/</p></td><td class="cell"><p><i>e</i></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Ep</i></p></td><td class="cell"><p>G</p></td><td class="cell"><p>R</p></td><td class="cell"><p>IY</p></td><td class="cell"><p><b><i>&lt;\&gt;</i></b></p></td><td class="cell"><p>L</p></td><td class="cell"><p>IY</p></td><td class="cell"><p><b>&lt;/&gt;</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Cp</i></p></td><td class="cell"><p>GE</p></td><td class="cell"><p>L</p></td><td class="cell"><p>I</p></td><td class="cell"><p></p></td><td class="cell"><p>L</p></td><td class="cell"><p>I</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>GE</p></td><td class="cell"><p>LI</p></td><td class="cell"><p></p></td><td class="cell"><p>LI</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>CG</i></p></td><td class="cell"><p></p></td><td class="cell"><p>M:B</p></td><td class="cell"><p>M:I</p></td><td class="cell"><p></p></td><td class="cell"><p>flJ:B</p></td><td class="cell"><p>flJ:I</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>M</p></td><td class="cell"><p></p></td><td class="cell"><p><i>m</i></p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>•</p></td><td class="cell"><p><i>Eg</i></p></td><td class="cell"><p><i>= egi, ■ ■</i></p></td><td class="cell"><p><i>■</i><i> </i><i>,egn</i></p></td><td class="cell"><p><i>= &amp;9i</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>•</p></td><td class="cell"><p><i>Ep</i></p></td><td class="cell"><p><i>= epi,--</i></p></td><td class="cell"><p><i>■</i><i> </i><i>,epn</i></p></td><td class="cell"><p><i>=</i><i> </i><i>ep"</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>•</p></td><td class="cell"><p><i>Cp</i></p></td><td class="cell"><p><i>= cpi,--</i></p></td><td class="cell"><p><i>■</i><i> </i><i>,cpn</i></p></td><td class="cell"><p><i>=</i><i> </i><i>cpi</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>•</p></td><td class="cell"><p><i>CG</i></p></td><td class="cell"><p><i>= cgi, ■ ■</i></p></td><td class="cell"><p><i>■</i><i> </i><i>,cgn</i></p></td><td class="cell"><p><i>=</i><i> </i><i>CQi</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="55.9" length="34" tooSmall="False" monospace="0.0">(Cg\Eg) =YjP{Cp\Eg)xP(Cg\Eg,Cp)(1)</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">VCp</doubt><doubt alpha="57.4" length="54" tooSmall="False" monospace="0.0">Pm(ep,jcp)(Cg\Eg)= E EP(ep\eg)xP(CP\Ep)xP(Cg|£p,Cp)(2)</doubt><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">VCp VEp</doubt><doubt alpha="51.8" length="56" tooSmall="False" monospace="0.0">Pm(egp,JCp)(Cg\Eg)= E ExP(Cp|£g, £P) xP(CG|£G,Ep, Cp)(3)</doubt><doubt alpha="50.5" length="93" tooSmall="False" monospace="0.0">Pm(eg,cp)(Cg\Eg) =E^(CpI^g)xP{Cg\Cp)(4)^m(Ep,Cp)(Cg|£g) = E EP(Ep\eg)x ^(CpI^p)x P(CG|Cp) (5)</doubt><doubt alpha="53.7" length="54" tooSmall="False" monospace="0.0">Pm(egp,cp)(Cg\Eg) =E EPOM^g)x P(Cp|£G,£P) xP(Cg\Cp)(6)</doubt><p>The three basic transliteration models in Ms <i>- M(EG, </i><i>Cp</i><i>), M{EP, </i><i>Cp), </i>and <i>M{EGP, </i><i>CP) -</i>are formulated as Eqs. (4)-(6). Chinese phoneme-based transliteration models in the literature fall into either <i>M{EG, CP) </i>or <i>M{EP, CP) </i>(Meng et al., 2001; Gao et al, 2004; Jiang et al, 2007; Lee and Chang, 2003; Wan and Verspoor, 1998; Virga and Khudanpur, 2003). The three basic transliter­ation models in Ms are identical as those in Mj, except for the Chinese phoneme-to-grapheme con­version method. They only depend on Chinese phonemes in Chinese phoneme-to-grapheme con­version represented as <i>P(Cg\Cp) </i>in Eqs. (4)-(6).</p><doubt alpha="47.8" length="69" tooSmall="False" monospace="0.0">PM(eg^(Cg\Eg) = P(Cg\Eg)(7)Pm(ep,4&gt;)(.Cg\Eg)(8)= J2p(ep\eg) xP(Cg\Ep)</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">VEp</doubt><doubt alpha="51.1" length="45" tooSmall="False" monospace="0.0">Pm(egp,4&gt;)(.Cg\Eg)(9)= Y,P(ep\eg)xP{CG\EG,Ep)</doubt><p>The three basic transliteration models in Mi are represented in Eqs. (7)-(9). Because the Mi models are independent of Chinese phonemes, they are the same as the transliteration models in the lit­erature used for machine transliteration from En­glish to other languages without relying on target-language phonemes (Karimi et al., 2007; Malik, 2006; Oh et al, 2006; Sherif and Kondrak, 2007; Yoon et al., 2007). Note that <i>M(EG,4&gt;) </i>is the same transliteration model as the one proposed by Li et al. (2004).</p></subsection><subsection number="3.2" title="Hybrid Transliteration Models"><p>The hybrid transliteration models in each class are defined by discrete mixture between the prob­ability distribution of the two basic transliter­ation models, as in Eq. (10) (Al-Onaizan and Knight, 2002; Oh et al, 2006), where 0 &lt; <i>a &lt; </i>1. We denote a hybrid transliteration model be­tween two basic transliteration models <i>M(xi,y) </i>and <i>M</i><b><i>(x2</i></b><i>,y) </i>as <i>M(x\ </i>+ <b><i>X2</i></b><i>,y,a), </i>where <i>y </i>g <i>Y </i><i>= {&lt;p,Cp, JCp}, </i><i>x\</i><i> </i>/ <b><i>X2, </i></b>and <i>x</i><b><i>\,X2 </i></b>g <i>X = {Eg,Ep,Egp}. </i>In this paper, we define three types of hybrid transliteration models in each class: <i>M(EG </i>+ <i>EP, </i><i>y, a), M(EG + EGp,y, a), </i>and <i>M(Ep </i>+ <i>Egp</i><i>, y, a).</i></p><doubt alpha="50.0" length="50" tooSmall="False" monospace="0.0">(Cg\Eg)(10)= axPm^Xi^(Cg\Eg)+(l-a)xPM(x2,y)(CG\EG)</doubt></subsection><subsection number="3.3" title="Probability Estimation"><p>Because Eqs. (l)-(9) can be estimated in a similar way, we limit our focus to Eq. (3) in this section. Assuming that <i>P</i><i>{EP\EG), </i><i>P</i><i>{CP\EG,Ep), </i>and <i>P(Cg\Eg</i><i>, Ep, Cp) </i>in Eq. (3) depend on the size of the context window, <i>k (k = </i>3 in this paper), they can be simplified into a series of products in Eqs.<page local="5" global="662"/> (11)-(13).</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">661</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">i</doubt><table caption="Table 3: Feature functions forP(cgi\cgl_l, (eg, ep, cp)1^)with an example in Table 2, wherei =2"></table><p>The maximum entropy model is used to esti­mate the probabilities in Eqs. (11)—(13) (Berger et al, 1996). Generally, a conditional maxi­mum entropy model is an exponential model that gives the conditional probability, as described in Eq. (14), where Ai is the parameter to be estimated and <i>fi(a, b) </i>is a feature function corresponding to A» (Berger et al, 1996; Ratnaparkhi, 1997):</p><doubt alpha="51.7" length="29" tooSmall="False" monospace="0.0">P(EP\EG) ^\\P(eVl\ejp\-_\,egl</doubt><doubt alpha="53.1" length="32" tooSmall="False" monospace="0.0">P(CP\EG,Ep) ~X\P{cPiW~\, (eg,ep)</doubt><doubt alpha="64.3" length="14" tooSmall="False" monospace="0.0">P(CG\EG,Ep,CP)</doubt><doubt alpha="44.4" length="9" tooSmall="False" monospace="0.0">i+k\ i-k&gt;</doubt><doubt alpha="62.1" length="29" tooSmall="False" monospace="0.0">IlP{cgi\cgli_lk, (eg, ep, cp)</doubt><doubt alpha="61.1" length="18" tooSmall="False" monospace="0.0">exp(Y;i Xifi(a,b))</doubt><doubt alpha="55.6" length="9" tooSmall="False" monospace="0.0">i+k\ i-kl</doubt><doubt alpha="37.5" length="8" tooSmall="False" monospace="0.0">P(b\a) =</doubt><doubt alpha="65.0" length="20" tooSmall="False" monospace="0.0">T,b'exvÇLihfi(a,b'))</doubt><doubt alpha="0.0" length="9" tooSmall="False" monospace="0.0">(11) (12)</doubt><doubt alpha="0.0" length="9" tooSmall="False" monospace="0.0">(13) (14)</doubt><p><i>fi(a,b) </i>is a binary function returning TRUE or FALSE based on context <i>a </i>and output <i>b. </i>If <i>fi(a,b)=\, </i>its corresponding model parame­ter Ai contributes toward conditional probability <i>P(b\a) </i>(Berger et al, 1996; Ratnaparkhi, 1997). The feature functions used here are defined in terms of context predicates — a function return­ing TRUE or FALSE that depends on the presence of the information in the current context (Ratna­parkhi, 1997). Context predicates and their de­scriptions used are given in Table 4.</p><p>N-gram includes <i>gram\(uj), gram2(uj), </i>and <i>granis(uj) </i>corresponding to a unigram, a bigram, and a trigram, respectively. Pair includes a pair of unigrams <i>(pair\\), </i>unigram and bigram <i>(pair\2), </i>and bigrams <i>(pair22). </i>Triple includes a triple of three unigrams <i>(triplex) </i>and a triple of two uni­grams and one bigram <i>(triple2). </i>Note that if dif­ferent context predicates represent the same con­text, we accept one of them and ignore the others (e.g., <i>pair\2(uj,Uj+i) = trigram(uj) </i><i>= u^+2).</i><i> </i>Table 3 represents the examples of feature func­tions for <i>P</i><i>(cgi\cgl~_\, (eg, ep, cp)<footnote anchor="1"/>^).</i></p><table caption="Table 4: Context predicates and their descriptions Category   Context predicates Description"></table><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">N-gram</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">Pair</doubt><doubt alpha="100.0" length="6" tooSmall="False" monospace="0.0">Triple</doubt><p>We used the "Maximum Entropy Modeling Toolkit"<footnote anchor="4"/> to estimate the probabilities and the LBFGS algorithm to find Ai in Eq. (14). For each transliteration model, we produced n-best transliterations using a stack decoder (Schwartz and Chow, 1990).</p></subsection><subsection number="3.4" title="Summary"><p>In this paper, we defined eighteen transliteration models to be compared. There are six translitera­tion models, three basic and three hybrid ones, in each class, Mi, Ms, and Mj. We compared the transliteration models from the viewpoint of Chi­nese phonemes or the class of transliteration mod­els in our experiments.</p></subsection></section><section number="4" title="Testing and Results"><p>We used the same test set used in Li et al. (2004) for our testing<footnote anchor="5"/>. It contains 37,694 pairs of English words and their official Chinese transliterations</p><p>Available at http://homepages.inf.ed.ac. uk/s0450736/maxent_toolkit.html<page local="6" global="663"/></p><footnote label="5">This test set was also used in"NEWS09 machine translit­eration shared task" for English-to-Chinese transliteration (Li et al., 2009)</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">662</doubt><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>fl</i></p></td><td class="cell"><p><i>gram;i(egi)</i></p></td><td class="cell"><p></p></td><td class="cell"><p>1-2</p></td><td class="cell"><p>= "ree"</p></td><td class="cell"><p>= "S</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>h</i></p></td><td class="cell"><p><i>pairn(cpi-i,cgi-i)</i></p></td><td class="cell"><p><i>CPi-</i></p></td><td class="cell"><p>-1</p></td><td class="cell"><p>= "G", cft_! = "fê:B"</p></td><td class="cell"><p><i>cgi = </i>"M</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>h</i></p></td><td class="cell"><p><i>pairi2(cgi-i,cpi-i)</i></p></td><td class="cell"><p><i>cp\_</i></p></td><td class="cell"><p>-1</p></td><td class="cell"><p>= "GE L", <i>cgi-\= </i>"#r:B"</p></td><td class="cell"><p>= "S</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>h</i></p></td><td class="cell"><p><i>pair</i><b><i>22</i></b><i>(cpi-i,cgi-2)</i></p></td><td class="cell"><p><i>egl</i></p></td><td class="cell"><p>-1</p></td><td class="cell"><p>= "gr", <i>ePi-i = </i>"G R"</p></td><td class="cell"><p>= "S</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>h</i></p></td><td class="cell"><p><i>triplei(egi,cpi,cgi-i)</i></p></td><td class="cell"><p></p></td><td class="cell"><p>=</p></td><td class="cell"><p>'r", <i>cpi-i </i>= "GE", <i>cgi-\= </i>"|§-:B"</p></td><td class="cell"><p><i>cgi = </i>"M</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>h</i></p></td><td class="cell"><p><i>triple2(egi-i,cgi-i,cpi-i)</i></p></td><td class="cell"><p><i>egi-</i></p></td><td class="cell"><p>-1</p></td><td class="cell"><p>= "g", <i>cv\-i= </i>"GE L", <i>cgi-\= </i>"|§-:B"</p></td><td class="cell"><p>= "S</p></td><td class="cell"><p>B</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>gram\(uj)</i></p></td><td class="cell"><p><i>Uj</i></p><p>i+i</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>gram2(uj)</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>gram;i(uj)</i></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>pairn(uj</i><i>,vk) </i><i>pain2(uj,vk) pair22(uj,vk)</i></p></td><td class="cell"><p><i>Uj, vk</i></p><p><i>v vk+1</i>J' <i>uk</i></p><p><i>vj+l vk+ u'j   </i><b>' </b><i>uk</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>triplei(uj</i><i>,v</i><i>k,wi)</i></p></td><td class="cell"><p><i>Uj, Vk, Wl</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>triple</i><i>2(uj,v</i><i>k,wi)</i></p></td><td class="cell"><p><i>Uj,</i><i> vk, wl</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>extracted from the "Chinese Transliteration of For­eign Personal Names" (Xinhua News Agency, 1992), which includes names in English, French, German, and many other foreign languages (Li et al., 2004). We used the same test data as in Li et al. (2004). But we randomly selected 90% of the training data used in Li et al. (2004) as our training data and the remainder as the development data, as shown in Table 5.</p><p>Table 5 : Number of English-Chinese translitera<u>tion pairs in each data set</u>_</p><doubt alpha="40.0" length="25" tooSmall="False" monospace="0.0">Ours     Li et al. (2004)</doubt><doubt alpha="44.4" length="27" tooSmall="False" monospace="0.0">Training data 31,299 34,777</doubt><doubt alpha="60.7" length="28" tooSmall="False" monospace="0.0">Development data   3,478 N/A</doubt><doubt alpha="39.4" length="33" tooSmall="False" monospace="0.0">Blind test data       2,896 2,896</doubt><p>We used the training data for training the transliteration models. For each model, we tuned the parameters including the number of iterations for training the maximum entropy model and a Gaussian prior for smoothing the maximum en­tropy model using the development data. Further, the development data was used to select param­eter <i>a </i>of the hybrid transliteration models. We varied parameter <i>a </i>from 0 to 1 in 0.1 intervals (i.e., <i>a</i><i>=0, </i>0.1, 0.2, • • •,!) and tested the perfor­mance of the hybrid models with the development data. Then we chose <i>a </i>that showed the best per­formance in each hybrid model. The blind test data was used for evaluating the performance of each transliteration model. <i>The</i><i> </i><i>CMU</i><i> </i><i>Pronounc­</i><i></i><i>ing</i><i> </i><i>Dictionary<footnote anchor="6"/>,</i><i> </i>which contains about 120,000 English words and their pronunciations, was used for estimating <i>P</i><i>(</i><i>Ep\Eg).</i></p><p>We conducted two experiments. First, we com­pared the overall performance of the translitera­tion models. Second, we investigated the effect of training data size on the performance of each transliteration model.</p><p>The evaluation was done for word accuracy in top-1 (ACC), Chinese pronunciation accuracy (CPA) and a mean reciprocal rank (MRR) met­ric (Kantor and Voorhees, 2000; Li et al, 2009; Chang et al, 2009). ACC measures how many correct transliterations appeared in the top-1 re­sult of each system. CPA measures the Chinese pronunciation accuracy in the top-1 of the n-best Chinese pronunciation. We used CPA for comparing the performance between systems based on Chinese phonemes. MRR, mean reciprocal ranks of n-best results of each system over the test en­tries, is an evaluation measure for n-best translit­erations. If a transliteration generated by a system matches a reference transliteration<footnote anchor="7"/> at the <i>rth </i>posi­tion of the n-best results, its reciprocal rank equals 1/r; otherwise its reciprocal rank equals 0, where 1 &lt; <i>r &lt; n. </i>We produced 10-best Chinese translit­erations for each English word in our experiments.</p><footnote label="6">Available at http : //www. speech. cs . emu. edu/ cgi-bin/cmudict</footnote><subsection number="4.1" title="Comparison of the Overall Performance"><p>Table 6 represents the overall performance of one system in a previous work (Li et al., 2004) and eighteen systems based on the transliteration mod­els defined in this paper. ACC, MRR, and CPA represent the evaluation results for each model trained by our training data. To test transliteration models without the errors introduced by incorrect Chinese phonemes, we carried out the experiments with the correct Chinese pronunciation (or the correct Chinese phoneme sequence) in Chinese phoneme-to-grapheme conversion. In the exper­iment, we put the correct Chinese pronunciation into the top-1 of the n-best Chinese pronunciation with the highest probability, say <i>P(Cp\Eg</i><i>)=1; </i>thus CPA was assumed to be 100%. The ACC of the transliteration models under this condition is denoted as ACC in Table 6. Train represents the evaluation results of the transliteration mod­els trained by our training data. To compare Li et al. (2004) and transliteration models defined in this paper under the same condition, we also car­ried out experiments with the same training data in Li et al. (2004). Since the training data used in Li et al. (2004) is identical as the union of our training and development data, we denoted it as Train+Dev in Table 6. In both Train and Train+Dev, we used the same parameter setting that was obtained by using the development data.</p><p>LI04 represents a system in Li et al. (2004), and its ACC in Train+Dev is taken from the literature. The systems based on the translitera­tion models defined in our paper are represented from the second row in Table 6. The phoneme-based transliteration models in the literature cor­respond to either <i>M(Eg</i><i>, Cp) </i>(Wan and Verspoor, 1998; Lee and Chang, 2003; Jiang et al., 2007) or <i>M{Ep, Cp) </i>(Meng et al, 2001; Gao et al, 2004;<page local="7" global="664"/></p><footnote label="7">In our test set, an English word corresponds to one refer­ence Chinese transliteration.</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">663</doubt><table caption="Table 6: Comparison of the overall performance"></table><p>Virga and Khudanpur, 2003).</p><p>A comparison between the basic and hybrid transliteration models showed that the hybrid ones usually performed better (the exception was <i>M(Ep+Egp</i><i>, y, a) </i>but the performance still com­parable to the basic ones in each class). Es­pecially, the hybrid ones based on the best two basic transliteration models, <i>M(EG+EGp,y,a), </i>showed the best performance.</p><p>A comparison among the Mi, Ms, and Mj models showed that Chinese phonemes did contribute to the performance improvement of English-to-Chinese transliteration when Chinese phonemes were used together with their corre­sponding English graphemes and phonemes in Chinese phoneme-to-grapheme conversion. A one-tail paired t-test between the Mi and Mj models showed that the results of the Mj mod­els were always significantly better than those of the Mi models if the Mi and Mj models shared the same English-side parameter, <i>x </i>g <i>{Eg</i><i>, Ep, </i><i>Egp} </i>(level of significance = 0.001). In the results obtained by the Ms and Mj mod­els, the figures in CPA are the same when the Ms and our Mj models share the same English-side parameter. Moreover, the difference between the figures in ACC and CPA can be interpreted as the error rate of Chinese phoneme-to-grapheme conversion. Our proposed Mj models gener­ated Chinese transliterations with a very low er­ror rate in Chinese phoneme-to-grapheme conver­sion, while the Ms models suffered from a signif­icant error rate in Chinese phoneme-to-grapheme conversion. ACC showed that the Mj models still outperformed the Ms models even without errors in generating Chinese pronunciation from the English words. These results indicate that the joint use of Chinese phonemes and their corre­sponding English graphemes and phonemes sig­nificantly improved the performance in Chinese phoneme-to-grapheme conversion and English-to-Chinese transliteration.</p><p>Table 7 shows the Chinese transliterations gen­erated by <i>M (EG, </i><i>&lt;j&gt;),</i><i> </i><i>M (Egp</i><i>, &lt;f&gt;), </i><i>M (Eg, JCp), </i>and <i>M (</i><i>Egp</i><i>, JCP) </i>where English or Chinese phonemes contributed to the correct translitera­tion. In this table, the first column show the English words and their English phonemes, and the second and third columns represent the Chi­nese transliterations and their phonemes. Note that the Chinese phonemes in the second and third columns of the Mi models are not used in translit­eration. They are shown in the table to indicate the difference in the Chinese phonemes of Chinese<page local="8" global="665"/></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">664</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Class</p></td><td class="cell"><p>Model</p></td><td class="cell"><p>Train</p></td><td class="cell"><p>Train<i>+</i>Dev</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>acc</p></td><td class="cell"><p>mrr</p></td><td class="cell"><p>cpa</p></td><td class="cell"><p>acc</p></td><td class="cell"><p>acc</p></td><td class="cell"><p>mrr</p></td><td class="cell"><p>cpa</p></td><td class="cell"><p>acc</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LI04</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>70.1</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG, </i><i>JCp)</i></p></td><td class="cell"><p>71.9</p></td><td class="cell"><p>80.4</p></td><td class="cell"><p>72.3</p></td><td class="cell"><p>88.2</p></td><td class="cell"><p>72.3</p></td><td class="cell"><p>80.7</p></td><td class="cell"><p>73.1</p></td><td class="cell"><p>88.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EP, JCp)</i></p></td><td class="cell"><p>61.1</p></td><td class="cell"><p>70.3</p></td><td class="cell"><p>62.4</p></td><td class="cell"><p>82.8</p></td><td class="cell"><p>61.1</p></td><td class="cell"><p>70.6</p></td><td class="cell"><p>63.1</p></td><td class="cell"><p>83.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mj</p></td><td class="cell"><p><i>M(EGP, </i><i>JCp)</i></p></td><td class="cell"><p>72.3</p></td><td class="cell"><p>80.9</p></td><td class="cell"><p>73.2</p></td><td class="cell"><p>89.6</p></td><td class="cell"><p>73.5</p></td><td class="cell"><p>81.5</p></td><td class="cell"><p>73.9</p></td><td class="cell"><p>90.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(Eg+Ep</i><i>, JCp, </i>0.7)</p></td><td class="cell"><p>72.8</p></td><td class="cell"><p>80.7</p></td><td class="cell"><p>73.8</p></td><td class="cell"><p>89.7</p></td><td class="cell"><p>73.2</p></td><td class="cell"><p>81.0</p></td><td class="cell"><p>74.7</p></td><td class="cell"><p>90.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG+EGP, JCp, </i>0.6)</p></td><td class="cell"><p><b>73.5</b></p></td><td class="cell"><p><b>81.7</b></p></td><td class="cell"><p><b>74.2</b></p></td><td class="cell"><p><b>90.6</b></p></td><td class="cell"><p><b>73.7</b></p></td><td class="cell"><p><b>81.8</b></p></td><td class="cell"><p><b>74.8</b></p></td><td class="cell"><p><b>91.2</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(Ep+Egp</i><i>, JCp, </i>0.1)</p></td><td class="cell"><p>71.6</p></td><td class="cell"><p>80.3</p></td><td class="cell"><p>73.3</p></td><td class="cell"><p>89.8</p></td><td class="cell"><p>72.5</p></td><td class="cell"><p>80.8</p></td><td class="cell"><p>73.8</p></td><td class="cell"><p>90.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG,&lt;j&gt;)</i></p></td><td class="cell"><p>70.0</p></td><td class="cell"><p>78.5</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>70.6</p></td><td class="cell"><p>79.0</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EP, </i><i>&lt;/&gt;)</i></p></td><td class="cell"><p>58.5</p></td><td class="cell"><p>69.3</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>59.4</p></td><td class="cell"><p>70.1</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mi</p></td><td class="cell"><p><i>M(EGP,&lt;f&gt;)</i></p></td><td class="cell"><p>71.2</p></td><td class="cell"><p>79.9</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>72.3</p></td><td class="cell"><p>80.7</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG+EP,(f</i><i>),0</i><i>.7)</i></p></td><td class="cell"><p>70.7</p></td><td class="cell"><p>79.1</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>72.0</p></td><td class="cell"><p>80.0</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG+EGP,&lt;f</i><i>),0A)</i></p></td><td class="cell"><p>72.0</p></td><td class="cell"><p>80.3</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>72.8</p></td><td class="cell"><p>80.9</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(Ep+EGp,(f</i><i>),0</i><i>.l)</i></p></td><td class="cell"><p>71.0</p></td><td class="cell"><p>79.6</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>72.0</p></td><td class="cell"><p>80.4</p></td><td class="cell"><p>N/A</p></td><td class="cell"><p>N/A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG,Cp)</i></p></td><td class="cell"><p>58.9</p></td><td class="cell"><p>70.2</p></td><td class="cell"><p>72.3</p></td><td class="cell"><p>78.4</p></td><td class="cell"><p>59.1</p></td><td class="cell"><p>70.4</p></td><td class="cell"><p>73.1</p></td><td class="cell"><p>78.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EP, </i><i>Cp)</i></p></td><td class="cell"><p>50.2</p></td><td class="cell"><p>62.3</p></td><td class="cell"><p>62.4</p></td><td class="cell"><p>78.4</p></td><td class="cell"><p>50.4</p></td><td class="cell"><p>62.6</p></td><td class="cell"><p>63.1</p></td><td class="cell"><p>78.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Ms</p></td><td class="cell"><p><i>M(EGP,Cp)</i></p></td><td class="cell"><p>59.1</p></td><td class="cell"><p>70.4</p></td><td class="cell"><p>73.2</p></td><td class="cell"><p>78.4</p></td><td class="cell"><p>59.3</p></td><td class="cell"><p>70.5</p></td><td class="cell"><p>73.9</p></td><td class="cell"><p>78.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(EG+Ep,CP, </i>0.8)</p></td><td class="cell"><p>59.7</p></td><td class="cell"><p>71.3</p></td><td class="cell"><p>73.8</p></td><td class="cell"><p>79.0</p></td><td class="cell"><p>60.3</p></td><td class="cell"><p>71.7</p></td><td class="cell"><p>74.7</p></td><td class="cell"><p>79.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(Eg+Egp</i><i>,</i><i>Cp, </i>0.6)</p></td><td class="cell"><p>59.8</p></td><td class="cell"><p>71.7</p></td><td class="cell"><p>74.2</p></td><td class="cell"><p>78.9</p></td><td class="cell"><p>60.6</p></td><td class="cell"><p>72.1</p></td><td class="cell"><p>74.8</p></td><td class="cell"><p>78.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><i>M(Ep+EGp,CP, </i>0.1)</p></td><td class="cell"><p>58.8</p></td><td class="cell"><p>70.4</p></td><td class="cell"><p>73.3</p></td><td class="cell"><p>78.9</p></td><td class="cell"><p>59.4</p></td><td class="cell"><p>70.7</p></td><td class="cell"><p>73.8</p></td><td class="cell"><p>78.8</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Table 7: Top-1 results <i>oïM(EG, </i><i>&lt;j&gt;),</i><i> M(EGp, </i><i>&lt;j&gt;),</i><i> M(EG, </i><i>JCp), </i>and <i>M(EGP, </i><i>JCP), </i>where * rep­resents incorrect transliterations transliterations between the Mi and Mj models.</p><p>For <i>Emily</i><i> </i>and <i>Reinhardt,</i><i> </i>the Mj models gen­erated correct Chinese transliterations, but the Mi models did not. Figure 1 shows the probabil­ity distribution when a transliteration model gen­erates the first Chinese character in the Chinese transliteration of <i>Reinhardt</i><i> </i>with and without Chi­nese phonemes. Two Chinese characters, H and <i>Wi, </i>were strong candidates and ^ is the correct one in this case. Without Chinese phonemes, <i>M (Eg</i><i>, &lt;fi), </i>which is based on <i>V(cg\Reinhardt)</i><i> </i>in Figure 1(a) preferring H to <i>W,</i><i> </i>generated the incorrect transliteration as shown in Table 7. How­ever, Figure 1(b) shows that ^ can be selected if the correct Chinese phoneme sequence "LAI YIN ..." is given. Three Chinese phoneme se­quences starting with "LAI YIN ..." "LAI NA ...", and "LAI NEI ..." were generated from <i>Rein­</i><i></i><i>hardt,</i><i> </i>where "LAI YIN ..." was the best Chinese phoneme sequence based on the probability distri­bution in Figure 1(c). As a result, <i>M(EG, JCp), </i>which jointly used Chinese phonemes with En­glish graphemes, generated the correct Chinese transliteration of <i>Reinhardt</i><i> </i>based on two probabil­ity distribution in Figures 1(b) and 1(c). In the case of <i>Ivy,</i><i> </i>English phonemes contributed to generat­ing the correct transliteration in the <i>M(EGp,(f)) </i>and <i>M(EGp, JCp) </i>models.</p><p>Chinese transliterations sometimes reflect the English word's pronunciation as well as the Chi­nese character's meaning (Li et al, 2007). Li</p><doubt alpha="0.0" length="10" tooSmall="False" monospace="0.0">0.6 -0.4 -</doubt><doubt alpha="0.0" length="8" tooSmall="False" monospace="0.0">0.2 -0 -</doubt><p>POffi I Reinhardt)  P(H | Reinhardt)</p><p>(a) Probability distribution when Chi­nese phonemes are not given (b) Probability distribution when Chinese phonemes are given (c) Probability distribution for Chinese phoneme se­quence "LAI YIN ..." and others</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">»</doubt><doubt alpha="58.3" length="96" tooSmall="False" monospace="0.0">MP(cg IReinhardt, "LAI YIN ..") ■ P(cg| Reinhardt, "LAI NA ..") □ P(cg IReinhardt, "LAI NEI ..")</doubt><p>Figure 1 : Probability distribution for the first Chi­nese character in the Chinese transliteration of et al. (2007) defined such a Chinese transliter­ation as a phonetic-semantic transliteration (se­mantic transliteration) to distinguish it from a usual phonetic transliteration. One fact that affects semantic transliteration is gender asso­ciation (Li et al., 2007). For example, ftj (meaining <i>jasmine)</i><i> </i>is frequently used in Chi­nese transliterations of female names but sel­dom in common person names. Because <i>Emily</i><i> </i>is often used in female names, the results ob­tained by the <i>M(EG,JCP) </i>and <i>M(EGP,JCP) </i>models are acceptable. This indicates that Chi­nese phonemes coupled with English graphemes or those coupled with English graphemes and phonemes could provide evidence required for se­mantic transliteration as well as phonetic translit­eration. As a result, <i>M</i><i>(EGP, </i><i>(/)), M (Eg</i><i>, JCP),</i> and <i>M (Egp</i><i>, JCp), </i>which used phonemes cou­pled with English graphemes, achieved higher per­formance than <i>M(Eg</i><i>,4&gt;), </i>which relied only on English graphemes.<page local="9" global="666"/></p><doubt alpha="65.6" length="32" tooSmall="False" monospace="0.0">Reinhardt:M(EG,&lt;f&gt;)vs.M(EG, JCP)</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">665</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>M, models</p></td><td class="cell"><p>M(EG,0)</p></td><td class="cell"><p>M(EGP,0)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Emily</i></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(EH M IH L IY)</p></td><td class="cell"><p>(AI MI LI)</p></td><td class="cell"><p>(AI MI LI)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Ivy</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>um</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(AY V IY)</p></td><td class="cell"><p>(YIWEI)</p></td><td class="cell"><p>(AI WEI)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Reinhardt</i></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(R AI N HH AA R T)</p></td><td class="cell"><p>(LAI YIN HA TE)</p></td><td class="cell"><p>(LAI YIN HATE)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mj models</p></td><td class="cell"><p>M(EG,JCP)</p></td><td class="cell"><p>M(EGP,JCp)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Emily</i></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(EH M IH L IY)</p></td><td class="cell"><p>AI MI LI</p></td><td class="cell"><p>AI MI LI</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Ivy</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>um</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(AY V IY)</p></td><td class="cell"><p>YI WEI</p></td><td class="cell"><p>AI WEI</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Reinhardt</i></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(R AI N HH AA R T)</p></td><td class="cell"><p>LAI YIN HA TE</p></td><td class="cell"><p>LAI YIN HA TE</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>0.8</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>0.6</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>0.4</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>0.2</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>0</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>P("LAI YIN .." 1 Reinhardt)   P(V'LAI YIN .." | Reinhardt)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></subsection><subsection number="4.2" title="Effect of Training Data Size"><doubt alpha="0.0" length="11" tooSmall="False" monospace="0.0">20 40 60 80</doubt><p>Training Data Size (%)</p><p>(a) Basic transliteration models</p><p>S M(EG+Ep,Cp,0.8) .......s.......</p><doubt alpha="50.0" length="16" tooSmall="False" monospace="0.0">M(EG+EGP,Cp,0.6)</doubt><doubt alpha="45.0" length="20" tooSmall="False" monospace="0.0">M(Ep+EGP,Cp,0.1)—g —</doubt><doubt alpha="50.0" length="16" tooSmall="False" monospace="0.0">M(EG+Ep,JCp,0.7)</doubt><footnote label="30">" M(E G +E GP ,JCp,0.6) -- a.- "</footnote><doubt alpha="48.1" length="27" tooSmall="False" monospace="0.0">M(Ep+EGP,JCp,0.1) _i_i_i_i_</doubt><p>(b) Hybrid transliteration models</p><figure caption="Figure 2: Performance of each system with differ­ent training data size"></figure><p>We investigated the effect of training data size on the performance of each transliteration model. We randomly selected training data with ratios from 10 to 90% and compared the performance of each system trained by different sizes of train­ing data. The results for the basic translitera­tion models in Figure 2(a) can be categorized into three groups. <i>M(EGp</i><i>,4&gt;) </i>and <i>M(EGp,JCP) </i>fall into the best group, where they showed the best performance regardless of training data size. <i>M</i><i>(EG, </i><i>4&gt;) </i>and <i>M(EG, JCP) </i>belong to the mid­dle group, where they showed lower performance than the best group if the training data size is small, but their performance is comparable to the best group if the size of the training data is large enough. The others always showed lower perfor­mance than both the best and middle groups. Figure 2(b) shows that hybrid transliteration models, on average, were less sensitive to the training data size than the basic ones, because the two differ­ent basic transliteration models used in the hybrid ones boosted transliteration performance by com­plementing each other's weak points.</p></subsection></section><section number="5" title="Conclusion"><p>We proposed a new English-to-Chinese transliter­ation model based on Chinese phonemes and their corresponding English graphemes and phonemes. We defined eighteen English-to-Chinese translit­eration models including our proposed model and classified them into three classes based on the role of Chinese phonemes in the transliteration mod­els. Experiments showed that Chinese phonemes in our proposed model can contribute to the performance improvement in English-to-Chinese transliteration.</p><p>Now we can answer <i>Yes</i><i> </i>to this paper's key ques­tion, "Can Chinese phonemes improve machine transliteration?" Actually, this is the second time the same question has been answered. The pre­vious answer, which was unfortunately reported as <i>No</i><i> </i>by Li et al. (2004), has been accepted as true for the last five years; the research issue has been considered closed. In this paper, we found a new answer that contradicts the previous an­swer. We hope that our answer promotes research on phoneme-based English-to-Chinese translitera­tion.</p></section><references><p>Y. Al-Onaizan and Kevin Knight. 2002. Translating named entities using monolingual and bilingual re­sources. In <i>Proc. of ACL '02, </i>pages 400-408.</p><p>A. L. Berger, S. D. Pietra, and V J. D. Pietra. 1996. A maximum entropy approach to natural language pro­cessing. <i>Computational Linguistics, </i>22(1):39-71.</p><p>M. Chang, D. Goldwasser, D. Roth, and Y. Tu. 2009. Unsupervised constraint driven learning for translit­eration discovery. In <i>Proceedings of NAACL HLT' 09.</i></p><p>Wei Gao, Kam-Fai Wong, and Wai Lam. 2004. Phoneme-based transliteration of foreign names for OOV problem.   In <i>Proc. of IJCNLP 2004, </i>pages 110-119.</p><p>Long Jiang, Ming Zhou, Lee-Feng Chien, and Cheng Niu. 2007. Named entity translation with web min­ing and transliteration. In <i>Proc. of IJCAI '07, </i>pages 1629-1634.</p><p>PaulB. Kantor and Ellen M.Voorhees. 2000. Thetrec-5 confusion track: Comparing retrieval methods for scanned text. <i>Information Retrieval, </i>2:165-176.</p><p>Sarvnaz Karimi, Falk Scholer, and Andrew Turpin. 2007. Collapsed consonant and vowel models: New approaches for English-Persian transliteration and back-transliteration. In <i>Proceedings ofACL '07, </i>pages 648-655.</p><p>Chun-Jen Lee and Jason S. Chang. 2003. Acqui­sition of English-Chinese transliterated word pairs from parallel-aligned texts using a statistical ma­chine transliteration model. In <i>Proc. ofHLT-NAACL 2003 Workshop on Building and Using Parallel Texts, </i>pages 96-103.</p><p>Haizhou Li, Min Zhang, and Su Jian. 2004. A joint source-channel model for machine transliteration.</p><p>In <i>Proceedings of the 42th Annual Meeting ofthe As­sociation ofComputational Linguistics, </i>pages 160­167.</p><p>Haizhou Li, Khe Chai Sim, Jin-Shea Kuo, and Minghui Dong. 2007. Semantic transliteration of personal names. In <i>Proceedings of the 45th Annual Meeting ofthe Association ofComputational Linguistics.</i></p><p>Haizhou Li, A Kumaran, Min Zhang, and Vladimir machine transliteration shared task. In <i>Proc. of ACL-IJCNLP 2009 Named Entities Workshop.</i></p><doubt alpha="64.3" length="42" tooSmall="False" monospace="0.0">Pervouchine. 2009. Whitepaper of NEWS 2009</doubt><p>M.G. Abbas Malik. 2006. Punjabi machine translit­eration. In <i>Proceedings of the COLING/ACL 2006, </i>pages 1137-1144.</p><p>H.M. Meng, Wai-Kit Lo, Berlin Chen, and K. Tang. 2001. Generating phonetic cognates to handle named entities in English-Chinese cross-language spoken document retrieval. In <i>Proc. of Auto­matic Speech Recognition and Understanding, 2001. ASRU </i>'01, pages 311-314.</p><p>Jong-Hoon Oh, Key-Sun Choi, and Hitoshi Isahara. 2006. A comparison of different machine transliter­ation models. <i>Journal of Artificial Intelligence Re­search (JAIR), </i>27:119-151.</p><p>Adwait Ratnaparkhi. 1997. A linear observed time sta­tistical parser based on maximal entropy models. In <i>Proceedings ofthe Second Conference on Empirical Methods in Natural Language Processing, </i>pages 1­10.</p><p>Richard Schwartz and Yen-Lu Chow. 1990. The N-</p><p>Best algorithm: an efficient procedure for finding top N sentence hypotheses. In <i>Proc. ofICASSP '90, </i>pages 81-84.</p><p>Tarek Sherif and Grzegorz Kondrak. 2007. Substring-based transliteration. In <i>Proceedings ofACL '07, </i>pages 944-951.</p><p>Paola Virga and Sanjeev Khudanpur. 2003. Translit­eration of proper names in cross-lingual information retrieval. In <i>Proc. of ACL 2003 Workshop on Multi­lingual and Mixed-language Named Entity Recogni­tion, </i>pages 57-64.</p><p>Stephen Wan and Cornelia Maria Verspoor. 1998. Au­tomatic English-Chinese name transliterationforde­velopment of multilingual resources. In <i>Proc. of COLING '98, </i>pages 1352-1356.</p><p>Xinhua News Agency. 1992. <i>Chinese transliteration offoreign personal names. </i>The Commercial Press.</p><p>Binyong Yin and Mary Felley. 1990. <i>Chinese Roman-ization: Pronunciation and Orthography. </i>Sinolingua.</p><p>Su-Youn Yoon, Kyoung-Young Kim, and Richard Sproat. 2007. Multilingual transliteration using feature based phonetic method. In <i>Proceedings of ACL'07, </i>pages 112-119.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">667</doubt></references><appendix title="Appendix"><p>(b) Ms models<page local="10" global="667"/></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">666</doubt><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b>i ^.......</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>70</p></td><td class="cell"><p></p></td><td class="cell"><p>sis'''''</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>60</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b>4--------■       * "</b></p><p>M(E<b>G,fl —</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>50</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>M(EPil;&gt;) <b>—x-—</b></p><p>M(EGP,0) --»-</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>40</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>M(EG,CP) .......a....... -</p><p>M(Ep,Cp)  - . -</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>M(EGP,Cp) -<b>g- -</b>M(EG,JCP)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>20</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>M(Ep,JCp) -M(EGP,JCp)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></appendix></body></article>