<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1"/><title>Lexical Choice via Topic Adaptation for Paraphrasing Written Language to Spoken Language</title><pubinfo>R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 981-992,2005. © Springer-Verlag Berlin Heidelberg 2005</pubinfo><author surname="Kaji" givenname="Nobuhiro"><org  name="University of Tokyo" country="Japan" city="Tokyo"/></author><author surname="Kurohashi" givenname="Sadao"><org  name="University of Tokyo" country="Japan" city="Tokyo"/></author></firstpageheader><frontmatter><p><b>Lexical Choice via Topic Adaptation for Paraphrasing Written Language to Spoken</b></p><p><b>Language</b></p><p>Nobuhiro Kaji<footnote anchor="1"/> and Sadao Kurohashi<footnote anchor="2"/></p><p>Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan kaji@tkl.iis.u-tokyo.ac.jp Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan kuro@kc.t.u-tokyo.ac.jp</p></frontmatter><abstract><b>Abstract. </b>Our research aims at developing a system that paraphrases written language text to spoken language style. In such a system, it is important to distinguish between appropriate and inappropriate words in an input text for spoken language. We call this task lexical choice for paraphrasing. In this paper, we describe a method of lexical choice that considers the topic. Basically, our method is based on the word probabilities in written and spoken language corpora. The novelty of our method is topic adaptation. In our framework, the corpora are classified into topic categories, and the probability is estimated using such corpora that have the same topic as input text. The result of evaluation showed the effectiveness of topic adaptation. </abstract></header><body><section number="1" title="Introduction"><p>Written language is different from spoken language. That difference has various aspects. For example, spoken language is often ungrammatical, or uses simplified words rather than difficult ones etc. Among these aspects this paper examines difficulty. Difficult words are characteristic of written language and are not ap­propriate for spoken language.</p><p>Our research aims at developing a system that paraphrases written language text into spoken language style. It helps text-to-speech generating natural voice when the input is in written language. In order to create such a system, the following procedure is required: (1) the system has to detect inappropriate words in the input text for spoken language, (2) generate paraphrases of inappropriate words, and (3) confirm that the generated paraphrases are appropriate. This paper examines step (1) and (3), which we call lexical choice for paraphrasing written language to spoken language.</p><p>Broadly speaking, lexical choice can be defined as binary classification task: the input is a word and a system outputs whether it is appropriate for spoken language or not.<page local="2"/> This definition is valid if we can assume that the word diffi­culty is independent of such factors as context or listeners. However, we think such assumption is not always true. One example is business jargon (or technical term). Generally speaking, business jargon is difficult and inappropriate for spo­ken language. Notwithstanding, it is often used in business talk. This example implies that the word difficulty is dependent on the topic of text/talk.</p><p>In this paper, we define the input of lexical choice as a word and text where it occurs (= the topic). Such definition makes it possible for a system to consider the topic. We think the topic plays an important role in lexical choice, when dealing with such words that are specific to a certain topic, e.g., business jargon. Hereafter, those words are called <i>topical words, </i>and others are called non-topical words. Of course, in addition to the topic, we have to consider other factors such as listeners and so on. But, the study of such factors lies outside the scope of this paper.</p><p>Based on the above discussion, we describe a method of lexical choice that considers the topic. Basically, our method is based on the word probabilities in written and spoken language corpora. It is reasonable to assume that these two probabilities reflect whether the word is appropriate or not. The novelty of the method is topic adaptation. In order to adapt to the topic of the input text, the corpora are classified into topic categories, and the probability is estimated using such corpora that have the same topic category as the input text. This process enables us to estimate topic-adapted probability. Our method was evalu­ated by human judges. Experimental results demonstrated that our method can accurately deal with topical words.</p><p>This paper is organized as follows. Section 2 represents method overview. Section 3 and Section 4 describe the corpora construction. Section 5 represents learning lexical choice. Section 6 reports experimental results. Section 7 describes related works. We conclude this paper in Section 8.</p><p><b>2Method</b><b> Overview</b></p><p>Our method uses written and spoken language corpora classified into topic cat­egories. They are automatically constructed from the WWW. The construction procedure consists of the following two processes (Figure 1).</p></section><section number="1." title="Style Classification"><p>Web pages are downloaded from the WWW, and are classified into written and spoken language style. Those pages classified as written/spoken language are referred as written/spoken language corpus. In this process, we discarded ambiguous pages that are difficult to classify.</p></section><section number="2." title="Topic Classification"><p>The written and spoken language corpora are classified into 14 topic cate­gories, such as arts, computers and so on.</p><p>Both classification methods are represented in Section 3 and Section 4.</p><p>Given an input word and a text where it occurs, it is decided as follows whether the input word is appropriate or inappropriate for spoken language.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">982</doubt><page local="3"/><p>1. The topic category of the input text is decided by the same method as the one used to classify Web pages into topic categories.</p><p>2. We estimate the probabilities of the input word in the written and spoken language corpora. We use such corpora that have the same topic as the input text.</p><p>3. Using the two probabilities, we decide whether the input word is appropriate or not. Section 5 describes this method.</p><p>Style classification    Written    Spoken Ambiguous ' Section ;ii</p><p>Web Pages</p><p>Arts category Written Spoken</p><p>Topic classification (Section 4)</p><p>Recreation category Written Spoken</p><p>Science category Written Spoken</p><p><b>Fig. 1. </b>Written and spoken language corpora construction</p></section><section number="3" title="Style Classification"><p>In order to construct written and spoken language corpora classified into topic categories, first of all, Web pages are classified into written and spoken language pages (Figure 1). Note that what is called spoken language here is not real utterance but chat like texts. Although it is not real spoken language, it works as a good substitute, as some researchers pointed out [2,11].</p><p>We follow a method proposed by Kaji et al (2004). Their method classifies Web pages into three types: (1) written language page, (2) spoken language page, and (3) ambiguous page. Then, Web pages classified into type (1) or (2) are used. Ambiguous pages are discarded because classification precision decreases if such pages are used. This Section summarizes their method. See [11] for detail. Note that for this method the target language is Japanese, and its procedure is dependent on Japanese characteristics.</p><subsection number="3.1" title="Basic Idea"><p>Web pages are classified based on interpersonal expressions, which imply an at­titude of a speaker toward listeners, such as familiarity, politeness, honor or con­tempt etc. Interpersonal expressions are often used in spoken language, although not frequently used in written language.<page local="4"/> For example, when spoken language is used, one of the most basic situations is face-to-face communication. On the other hand, such situation hardly happens when written language is used.</p><p>Therefore, Web pages containing many interpersonal expressions are classi­fied as spoken language, and vice versa. Among interpersonal expressions, such expressions that represent familiarity or politeness are used, because:</p><p><b>— </b>Those two kinds of interpersonal expressions frequently appear in spoken language,</p><p><b>— </b>They are represented by postpositional particle in Japanese and, therefore, are easily recognized as such.</p><p>Hereafter, interpersonal expression that represents familiarity/politeness is called familiarity/politeness expression.</p></subsection><subsection number="3.2" title="Style Classification Procedure"><p>Web pages are classified into the three types based on the following two ratios:</p><p><b>— </b>Familiarity ratio (F-ratio): '# of sentences including familiarity expressions' divided by '# of all the sentences in the page'.</p><p><b>— </b>Politeness ratio (P-ratio): '# of sentences including politeness expressions' divided by '# of all the sentences in the page'.</p><p>The procedure is as follows. First, Web pages are processed by Japanese morphological analyzer JUMAN3. And then, in order to calculate F-ratio and P-ratio, sentences which include familiarity or politeness expressions are recognized in the following manner. A sentence is considered to include the familiarity expression, if it has one of the following six postpositional particles: <i>ne, yo, wa, sa, ze, na. </i>A sentence is considered to include the politeness expression, if it has one of the following four postpositional particles: <i>desu, masu, kudasai, gozaimasu.</i></p><p>After calculating the two ratios, the page is classified according to the rules illustrated in Figure 2. If F-ratio and P-ratio are equal to 0, the page is classified as written language page. If F-ratio is more than 0.2, or if F-ratio is more than 0.1 and P-ratio is more than 0.2, the page is classified as spoken language page. The other pages are regarded as ambiguous and are discarded.</p></subsection><subsection number="3.3" title="The Result"><p>Table 1 shows the number of pages and words (noun, verb, and adjective) in the corpora constructed from the WWW. About 8,680k pages were downloaded from the WWW, and 994k/1,338k were classified as written/spoken language. The rest were classified as ambiguous page and they were discarded. The precision of this method was reported by Kaji et al (2004). According to their experiment, the precision was 94%.</p><footnote label="3"> http://www.kct.u-tokyo.ac.jp/nl-resource/juman-e.html</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">984</doubt><page local="5" global="985"/><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">F-ratio</doubt><doubt alpha="0.0" length="7" tooSmall="False" monospace="0.0">0.2 0.1</doubt><p>Spoken : language:</p><doubt alpha="100.0" length="1" tooSmall="True" monospace="0.0">I</doubt><p>Ambiguous</p></subsection></section><section number="02" title="P-ratio Written language"><p>F<b>ig. 2. </b>Style classification rule</p><p>T<b>able 2. </b>The size of the training and test data</p></section><section number="4" title="Topic Classification"><p>The written and spoken language corpora are classified into 14 topic categories (Figure 1). This task is what is called text categorization. We used Support Vec­tor Machine because it is reported to achieve high performance in this task.The training data was automatically built from Yahoo! Japan4.</p><p>The category provided by Yahoo! Japan have hierarchy structure. For exam­ple, there are Arts and Music categories, and Music is one of the subcategories of Arts. We used 14 categories located at the top level of the hierarchy. We downloaded Web pages categorized in one of the 14 categories. Note that we did not use Web pages assigned more than one categories. And then, the Web pages were divided them into 20 segments. One of them was used as the test data, and the others were used as the training data (Table 2). In the Table, the<page local="6" global="986"/></p><footnote label="4"> http://www.yahoo.co.jp/</footnote><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Topic category</p></td><td class="cell"><p>Training Test</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Arts</p></td><td class="cell"><p>2,834</p></td><td class="cell"><p>150</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Business &amp; Economy</p></td><td class="cell"><p>5,475</p></td><td class="cell"><p>289</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>T<b>able 1. </b>Thesizeof the written and spoken</p></td><td class="cell"><p>Computers &amp; Internet</p></td><td class="cell"><p>6,156</p></td><td class="cell"><p>325</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>language corpora</p></td><td class="cell"><p>Education</p></td><td class="cell"><p>2,943</p></td><td class="cell"><p>155</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Entertainment</p></td><td class="cell"><p>6,221</p></td><td class="cell"><p>328</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p># of pages # of words</p></td><td class="cell"><p>Government</p></td><td class="cell"><p>3,131</p></td><td class="cell"><p>165</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Written language          989k 432M</p></td><td class="cell"><p>Health</p></td><td class="cell"><p>1,800</p></td><td class="cell"><p>95</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Spoken language        1,337k 907M</p></td><td class="cell"><p>News</p></td><td class="cell"><p>2,888</p></td><td class="cell"><p>152</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Recreation</p></td><td class="cell"><p>4,352</p></td><td class="cell"><p>230</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Reference</p></td><td class="cell"><p>1,099</p></td><td class="cell"><p>58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Regional</p></td><td class="cell"><p>4,423</p></td><td class="cell"><p>233</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Science</p></td><td class="cell"><p>3,868</p></td><td class="cell"><p>204</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Social Science</p></td><td class="cell"><p>5,410</p></td><td class="cell"><p>285</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Society &amp; Culture</p></td><td class="cell"><p>5,208</p></td><td class="cell"><p>275</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Topic category□ Written language page ÜSpoken language page</p><doubt alpha="0.0" length="48" tooSmall="False" monospace="0.0">400000 350000 300000 250000 200000 150000 100000</doubt><doubt alpha="0.0" length="7" tooSmall="False" monospace="0.0">50000 0</doubt><p><b>Fig. 3. </b>The size of written and spoken language corpora in each topic category first column shows the name of the 14 topic categories. The second/third column shows the number of pages in the training/test data.</p><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">0)</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">gpa</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">o</doubt><p>SVM was trained using the training data. In order to build multi-class clas­sifier, we used One-VS-Rest method. Features of SVM are probabilities of nouns in a page. Kernel function was linear. After the training, it was applied to the test data. The macro-averaged accuracy was 86%.</p><p>The written and spoken language corpora constructed from the WWW were classified into 14 categories by SVM. Figure 3 depicts the number of pages in each category.</p></section><section number="5" title="Learning Lexical Choice"><p>We can now construct the written and spoken language corpora classified into topic categories. The next step is discrimination between inappropriate and ap­propriate words for spoken language using the probabilities in written and spoken language corpora (Section 2). This paper proposes two methods: one is based on Decision Tree (DT), and the other is based on SVM. This Section first de­scribes the creation of gold standard data, which is used for both training and evaluation. Then, we describe the features given to DT and SVM.</p><subsection number="5.1" title="Creation of Gold Standard Data"><p>We prepared data consisting of pairs of a word and binary tag. The tag represents whether that word is inappropriate or appropriate for spoken language. This data is referred as gold standard data. Note that the gold standard is created for each topic category.</p><p>Gold standard data of topic T is created as follows.</p><p>1. Web pages in topic T are downloaded from Yahoo! Japan, and we sampled words (verbs, nouns, and adjectives) from those pages at random.</p><p>2. Three human judges individually mark each word as INAPPROPRIATE, APPROPRIATE or NEUTRAL. NEUTRAL tag is used when a judge cannot mark a word as INAPPROPRIATE or APPROPRIATE with certainty.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">986</doubt><page local="7" global="987"/><p>3. The three annotations are combined, and single gold standard data is cre­ated. A word is marked as INAPPROPRIATE/APPROPRIATE in the gold standard, if <b>— </b>All judges agree that it is INAPPROPRIATE/APPROPRIATE, or</p><p><b>— </b>Two judges agree that it is INAPPROPRIATE/APPROPRIATE and the other marked it as NEUTRAL.</p><p>The other words are not used in the gold standard data. <b>5.2   The Features</b></p><p>Both DT and SVM use the same three features: the word probability in written language corpus, the word probability in spoken language corpus, and the ratio of the word probability in spoken language corpus to that in written language. Note that when DT and SVM are trained on the gold standard of topic T, the probability is estimated using the corpus in topic T.</p></subsection></section><section number="6" title="Evaluation"><p>This Section first reports the gold standard creation. Then, we show that DT and</p><p>SVM can successfully classify INAPPROPRIATE and APPROPRIATE words in the gold standard. Finally, the effect of topic adaptation is represented. <b>6.1   The Gold Standard Data</b></p><p>The annotation was performed by three human judges (Judge1, Judge2 and Judge3) on 410 words sampled from Business category, and 445 words sampled from Health category. Then, we created the gold standard data in each category (Table 3). The average Kappa value [3] between the judges was 0.60, which corresponds to substantial agreement.</p><p>T<b>able 3. </b>Gold standard data <b>Table 4. </b># of words in Business and Health categories corpora</p><subsection number="6.2" title="Lexical Choice Evaluation"><p>DT and SVM were trained and tested on the gold standard data using Leave-One-Out (LOO) cross validation. DT and SVM were implemented using C4.55and TinySVM6 packages. The kernel function of SVM was Gaussian RBF. Table <b>Table 5.</b><page local="8" global="988"/><b> </b>The result of LOO cross validation 4 shows the number of words in Business and Health categories corpora. Three features described in Section 5 were used.</p><footnote label="5"> http://www.rulequest.com/Personal/</footnote><footnote label="6"> http://chasen.org/ taku/software/TinySVM/</footnote><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>-<b> -----0 —----■ ! —</b></p><p>Business Health</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>INAPPROPRIATE</p></td><td class="cell"><p>49</p></td><td class="cell"><p>38</p></td><td class="cell"><p></p></td><td class="cell"><p>Business</p></td><td class="cell"><p>Health</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>APPROPRIATE</p></td><td class="cell"><p>267</p></td><td class="cell"><p>340</p></td><td class="cell"><p>Written language</p></td><td class="cell"><p>29,891k</p></td><td class="cell"><p>30,778k</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Total</p></td><td class="cell"><p>316</p></td><td class="cell"><p>378</p></td><td class="cell"><p>Spoken language</p></td><td class="cell"><p>9,018k</p></td><td class="cell"><p>32,235k</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>The result is summarized in Table 5. For example, in Business category, the accuracy of DT was 91.5%. 289 out of 316 words were classified successfully, and the 289 consists of 31 INAPPROPRIATE and 258 APPROPRIATE words. The last two columns show the precision and recall of INAPPROPRIATE words. MCB is Majority Class Baseline, which marks every word as APPROPRIATE.</p><p>Judging from the accuracy in Health category, one may think that our method shows only a little improvement over MCB. However, considering other evaluation measures such as recall of INAPPROPRIATE words, it is obvious that the proposed method overwhelms MCB. We would like to emphasize the fact that MCB is not at all practical lexical choice method. If MCB is used, all words in the input text are regarded as appropriate for spoken language and the input is never paraphrased.</p><p>One problem of our method is that the recall of INAPPROPRIATE words is low. We think that the reason is as follows. The number of INAPPROPRIATE words in the gold standard is much smaller than that of APPROPRIATE words. Hence, we think a system that is biased to classify words as APPROPRIATE often achieves high accuracy. It is one of future works to improve the recall while keeping high accuracy.</p><p>We examined discrimination rules learned by DT. Figure 4 depicts the rules learned by DT when the whole gold standard data of Business category is used as a training data. In the Figure, the horizontal/vertical axis corresponds to the probability in the written/spoken language corpus. Words in the gold standard can be mapped into this two dimension space. INAPPROPRIATE/ APPROPRI­ATE words are represented by a cross/square. The line represents discrimination rules. Words below the line are classified as INAPPROPRIATE, and the others are classified as APPROPRIATE.</p></subsection><subsection number="6.3" title="Effect of Topic Adaptation"><p>Finally, we investigated the effect of topic adaptation by comparing our method to a baseline method that does not consider topic.</p><p>Our method consists of two steps: (1) mapping from a word to features, and (2) applying discrimination rules to the features. In the step (1), the probability is estimated using the written and spoken language corpora in a certain topic<page local="9" global="989"/></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">988</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Topic Method        Accuracy # of correct answers</p></td><td class="cell"><p>Precision Recall</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Business</p></td><td class="cell"><p>DT .915 (289/316)</p></td><td class="cell"><p>31 + 258 =</p></td><td class="cell"><p>289</p></td><td class="cell"><p></p></td><td class="cell"><p>.775</p></td><td class="cell"><p>.660</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>SVM .889 (281/316)</p></td><td class="cell"><p>21 + 260 =</p></td><td class="cell"><p>281</p></td><td class="cell"><p></p></td><td class="cell"><p>.750</p></td><td class="cell"><p>.429</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>MCB .845 (267/316)</p></td><td class="cell"><p>0 + 267 =</p></td><td class="cell"><p>267</p></td><td class="cell"><p></p></td><td class="cell"><p>—</p></td><td class="cell"><p>.000</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Health</p></td><td class="cell"><p>DT .918 (347/378)</p></td><td class="cell"><p>21 + 326 =</p></td><td class="cell"><p>"347</p></td><td class="cell"><p></p></td><td class="cell"><p>.600</p></td><td class="cell"><p>.552</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>SVM .918 (347/378)</p></td><td class="cell"><p>13 + 334 =</p></td><td class="cell"><p>347</p></td><td class="cell"><p></p></td><td class="cell"><p>.684</p></td><td class="cell"><p>.342</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>MCB .899 (340/378)</p></td><td class="cell"><p>0 + 340 =</p></td><td class="cell"><p>340</p></td><td class="cell"><p></p></td><td class="cell"><p>—</p></td><td class="cell"><p>.000</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Fi<b>g. 4. </b>Decision tree rules in Business cat<b>Fig.</b><b> 5. </b>Examples in Business category</p><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">egory</doubt><doubt alpha="0.0" length="6" tooSmall="True" monospace="0.0">0.0001</doubt><doubt alpha="0.0" length="6" tooSmall="True" monospace="0.0">0.0004</doubt><doubt alpha="0.0" length="6" tooSmall="True" monospace="0.0">0.0002</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">ni</doubt><doubt alpha="90.3" length="31" tooSmall="True" monospace="0.0">Probability in written language</doubt><doubt alpha="16.7" length="6" tooSmall="True" monospace="0.0">U.0002</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">0</doubt><doubt alpha="0.0" length="20" tooSmall="True" monospace="0.0">0.0002 0.0004 0.0006</doubt><doubt alpha="0.0" length="6" tooSmall="True" monospace="0.0">0.0008</doubt><p>T. In the step (2), discrimination rules are learned by DT using the whole gold standard data of topic T. We used DT rather than SVM because rules are easy for humans to understand. On the other hand, the baseline uses the same dis­crimination rules as our method, but uses the whole written and spoken language corpora to map a word to features. Hereafter, the two methods are referred as Proposed and Baseline. Both methods use the same discrimination rules but map a word to features in a different way. Therefore, there are such words that are classified as INAPPROPRIATE by Proposed and are classified as AP­PROPRIATE by Baseline, and vice versa. In the evaluation, we compared the classification results of such words.</p><p>We evaluated the results of topical words and non-topical words separately. This is because we think Proposed is good at dealing with topical words and hence we can clearly confirm the effectiveness of topic adaptation. Here, a word is regarded as topical word in topic T, if its probabilities in the written and spoken language corpora assigned topic category T are larger than those in the whole corpora with statistical significance (the 5% level). Otherwise it is regarded as non-topical word in topic T. As a statistical test log-likelihood ratio [4] was used. The evaluation procedure is as follows.</p><p>1. Web pages in Business category were downloaded from Yahoo! Japan, and words in those pages were classified by the two methods. If the results of the two methods disagree, such words were stocked.</p><p>2. From the stocked words, we randomly sampled 50 topical words in Business and 50 non-topical words. Note that we did not use such words that are contained in the gold standard.</p><p>3. Using Web pages in Health category, we also sampled 50 topical words in Health and 50 non-topical words in the same manner.</p><p>4. As a result, 100 topical words and 100 non-topical words were prepared. For each word, two judges (Judge-A and Judge-B) individually assessed which method successfully classified the word. Some classification results were dif­ficult even for human judges to assess. In such cases, the results of the both methods were regarded as correct.</p><page local="10" global="990"/><p>Table 6 represents the classification accuracy of the 100 topical words. For example, according to assessment by Judge-A, 75 out of 100 words were classified successfully by Proposed. Similarly, Table 7 represents the accuracy of the 100 non-topical words. The overall agreement between the two judges according to the Kappa value was 0.56. We compared the result of the two methods using McNemar's test [8], and we found statistically significant difference (the 5% level) in the results. There was no significant difference in the result of non-topical words assessed by the Judge-A.</p><p>Ta<b>ble 6. </b>Accuracy of topical words classi­fication <b>Table 7.</b><b> </b>Accuracy of non-topical words classification</p><doubt alpha="62.3" length="53" tooSmall="False" monospace="0.0">Judge     Method Accuracy Judge-AProposed75% (75/100)</doubt><doubt alpha="38.1" length="21" tooSmall="False" monospace="0.0">_Baseline52% (52/100)</doubt><doubt alpha="45.8" length="48" tooSmall="False" monospace="0.0">Judge-B Proposed72% (72/100)Baseline53% (53/100)</doubt><doubt alpha="62.3" length="53" tooSmall="False" monospace="0.0">Judge     Method Accuracy Judge-AProposed48% (48/100)</doubt><doubt alpha="38.1" length="21" tooSmall="False" monospace="0.0">_Baseline66% (66/100)</doubt><doubt alpha="45.8" length="48" tooSmall="False" monospace="0.0">Judge-B Proposed38% (38/100)Baseline78% (78/100)</doubt></subsection><subsection number="6.4" title="Discussion and Future Work"><p>Proposed outperformed Baseline in topical words classification. This result indicates that the difficulty of topical words depends on the topic and we have to consider the topic. On the other hand, the result of Proposed was not good when applied to non-topical words. We think this result is caused by two reasons: (1) the difficulty of non-topical words is independent of the topic, and (2) Base­line uses larger corpora than Proposed (see Table 1 and Table 4). Therefore, we think this result does not deny the effectiveness of topic adaptation. These results mean that Proposed and Baseline are complementary to each other, and it is effective to combine the two methods: Proposed/Baseline is applied to topical/non-topical words. It is obvious from the experimental results that such combination is effective.</p><p>We found that Baseline is prone to classify topical words as inappropriate and such bias decreases the accuracy. Figure 5 depicts typical examples sam­pled from topical words in Business. Both judges regarded 'management' and 'customer'7 as appropriate for spoken language in Business topic. The white tri­angle and diamond in the Figure represent their features when the probability is estimated using the corpora in Business category. They are located above the line, which corresponds to discrimination rules, and are successfully classified as appropriate by Proposed. However, if the probability is estimated using the whole corpora, the features shift to the black triangle and diamond, and Base­line wrongly classified the two as inappropriate. In Health category, we could observe similar examples such as 'lung cancer' or 'metastasis'.</p><footnote label="7">Our target language is Japanese. Examples illustrated here are translation of the original Japanese words.</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">990</doubt><page local="11" global="991"/><p>These examples can be explained in the following way. Consider topical words in Business. When the probability is estimated using the whole corpora, it is influenced by the topic but Business, where topical words in Business are often inappropriate for spoken language. Therefore, we think that Baseline is biased to classify topical words as inappropriate.</p><p>Besides the lexical choice method addressed in this paper, we proposed lex­ical paraphrase generation method [10]. Our future direction is to apply these methods to written language texts and evaluate the output of text-to-speech. So far, the methods were tested on a small set of reports.</p><p>Although the main focus of this paper is lexical paraphrases, we think that it is also important to deal with structural paraphrases. So far, we implemented a system that paraphrases compound nouns into nominal phrases. It is our future work to build a system that generates other kinds of structural para­phrases.</p></subsection></section><section number="7" title="Related Work"><p>Lexical choice has been widely discussed in both paraphrasing and natural lan­guage generation (NLG). However, to the best of our knowledge, no researches address topic adaptation. Previous approaches are topic-independent or specific to only certain topic.</p><p>Lexical choice has been one of the central issues in NLG. However, the main focus is mapping from concepts to words, (e.g., [1]). In NLG, a work by Edmonds and Hirst is related to our research [5]. They proposed a computational model that represents the connotation of words.</p><p>Some paraphrasing researches focus on lexical choice. Murata and Isahara addressed paraphrasing written language to spoken language. They used only probability in spoken language corpus [12]. Kaji et al. also discussed paraphras­ing written language to spoken language, and they used the probabilities in written and spoken language corpora [11]. On the other hand, Inkpen et al. ex­amined paraphrasing positive and negative text [9]. They used the computational model proposed by Edmonds and Hirst [5].</p><p>The proposed method is based on the probability, which can be considered as a simple language model. In language model works, many researchers have discussed topic adaptation in order to precisely estimate the probability of topical words [6,7,13]. Our work can be regarded as one application of such language model technique.</p><p><b>8Conclusion</b></p><p>This paper proposed lexical choice method that considers the topic. The method utilizes written and spoken language corpora classified into topic categories, and estimate the word probability that is adapted to the topic of the input text. From the experimental result we could confirm the effectiveness of topic adap­tation.</p><page local="12" global="992"/></section><references><p>1. Berzilay, R., Lee, L.: Bootstrapping Lexical Choice via Multiple-Sequence Align­ment. Proceedings of EMNLP. (2002) 50-57</p><p>2. Bulyko, I., Ostendorf, M., and Stolcke, A.: Getting More Mileage from Web Text Sources for Conversational Speech Language Modeling using Class-Dependent Mix­tures. Proceedings of HLT-NAACL (2003) 7-9</p><p>3. Carletta, J.: Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics. <b>22 </b>(2). (1996) 249-255</p><p>4. Dunning, T.: Accurate Methods for the Statistics of Surprise and Coincidence. Computational Linguistics. <b>19 </b>(1). (1993) 61-74 6. Florian, R., Yarowsky, D.: Dynamic Nonlocal Language Modeling via Hierarchical</p><doubt alpha="60.6" length="109" tooSmall="False" monospace="0.0">5. Edmonds, P., Hirst, G.: Near-Synonymy and Lexical Choice. Computational Lin­guistics.28(2). (2002) 105-144</doubt><doubt alpha="62.1" length="58" tooSmall="False" monospace="0.0">Topic-Based Adaptation: Proceedings of ACL. (1999) 167-174</doubt><doubt alpha="63.3" length="109" tooSmall="False" monospace="0.0">7. Gildea, D., Hofmann, T.; TOPIC-BASED LANGUAGE MODELS USING EM. Proceedings of EUROSPEECH. (1999) 2167-2170</doubt><p>8. Gillick, L., Cox, S.: Some Statistical Issues in the Comparison of Speech Recogni­tion Algorithms. Proceedings of ICASSP. (1989) 532-535</p><p>9. Inkpen, D., Feiguina, O., and Hirst, G.: Generating more-positive and more-negative text. Proceedings of AAAI Spring Symposium on Exploring Attitude and Affect in Text. (2004)</p><doubt alpha="63.4" length="142" tooSmall="False" monospace="0.0">10. Kaji, N., Kawahara, D., Kurohashi, S., and Satoshi, S. : Verb Paraphrase based on Case Frame Alignment. Proceedings of ACL. (2002) 215-222</doubt><p>11. Kaji, N., Okamoto, M., and Kurohasih, S.: Paraphrasing Predicates from Written Language to SpokenLanguage Using the Web. Proceedings ofHLT-NAACL. (2004)</p><doubt alpha="0.0" length="7" tooSmall="False" monospace="0.0">241-248</doubt><p>12. Murata, M., Isahara, H.: Automatic Extraction of Differences Between Spoken and Written Languages, and Automatic Translation from the Written to the Spoken Language. Proceedings of LREC. (2002)</p><p>13. Wu, J., Khudanpur, S.: BUILDING A TOPIC-DEPENDENT MAXIMUM EN­TROPY MODEL FOR VERY LARGE CORPORA. Proceedings of ICASSP. (2002) 777-780</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">992</doubt></references></body></article>