<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="165"/><title>Description of the HKU Chinese Word Segmentation System for Sighan Bakeoff 2005</title><author surname="Fu" givenname="Guohong"><org  name="University of Hong Kong" country="Hong Kong" city="Pokfulam"/></author><author surname="Luke" givenname="Kang-Kwong"><org  name="University of Hong Kong" country="Hong Kong" city="Pokfulam"/></author><author surname="WONG" givenname="Percy Ping-Wai"><org  name="University of Hong Kong" country="Hong Kong" city="Pokfulam"/></author></firstpageheader><frontmatter><p><b>Description of the HKU Chinese Word Segmentation System</b></p><p><b>for Sighan Bakeoff 2005</b></p><p><b>Guohong Fu Kang-Kwong Luke Percy Ping-Wai WONG</b></p><p>Department of Linguistics       Department of Linguistics       Department of Linguistics</p><p>The University of Hong Kong The University of Hong Kong The University of Hong Kong</p><p>Pokfulam Road, Hong Kong Pokfulam Road, Hong Kong Pokfulam Road, Hong Kong</p><p>ghfu@hkucc.hku.hk kkluke@hkusua.hku.hk wongpw@hkusua.hku.hk</p></frontmatter><abstract>In this paper, we describe in brief our system for the Second International Chinese Word Segmentation Bakeoff sponsored by the ACL-SIGHAN. We participated in all tracks at the bakeoff. The evaluation results show our system can achieve an F measure of 0.940­0.967 for different testing corpora. </abstract></header><body><section number="1" title="Introduction"><p>Word segmentation is very important for Chi­nese text processing, which is aiming at recog­nizing the implicit word boundaries in plain Chinese text. Over the past decades, great pro­gress has been made with Chinese word segmentation technology. However, two difficulties still face us while developing a practical segmentation system for large open applications, i.e. the resolution of ambiguous segmentation and the identification of unknown or out-of-vocabulary (OOV) words.</p><p>In order to resolve the above two problems, we developed a purely statistical Chinese word segmentation system using a two-stage strategy. We participated in eight tracks at the Second International Chinese Word Segmentation Bakeoff sponsored by the ACL-SIGHAN, and tested our system on different testing corpora. The scored results show that our system is effec­tive for most of ambiguous segmentation and unknown words in Chinese text. In this paper, we make a summary of this work and give some analysis on the results.</p><p>The rest of this paper is organized as follows: First in Section 2, we describe in brief a two-stage strategy for Chinese word segmentation. Then in Section 3, we give details about the set­tings or configuration of our system for different testing tracks, particularly the training data and the dictionaries used in our system. Finally, we report the results of our system at this bakeoff in Section 4, and give our conclusions on this work in Section 5.</p></section><section number="2" title="Overview of the System"><p>In practice, our system works in two major steps as follows:</p><p>The first step is a process of known word segmentation, which aims to segment an input sequence of Chinese characters into a sequence of known words that are listed in the system dic­tionary. In our current system, we apply a known word bigram model to perform this task (Fu and Luke, 2003; Fu and Luke, 2004; Fu and</p><p>Luke, 2005).</p><p>Actually, known word segmentation is a process of disambiguation. Given a Chinese character string <i>c </i><i>= c1c1- -cn </i>, there are usually multiple possible segmentations of known words <i>W = wiw2---wm </i>according to the system diction­ary. The task of known word segmentation is to find a proper segmentation <i>W = wlw1--wm </i>that maximizes the probability <b>jj</b><i>p(w </i>| <i>w </i>^, i.e.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">1=1</doubt><doubt alpha="52.3" length="44" tooSmall="False" monospace="0.0">W = arg max P(W |C) » arg maxjjP(wi|wt_1)(1)</doubt><doubt alpha="57.1" length="7" tooSmall="True" monospace="0.0">W W j=i</doubt><p>The second step is actually a tagging task on the sequence of known words acquired in the first step, which intends to detect unknown words or out-of-vocabulary (OOV) words in the input.<page local="2" global="166"/> In this process, each known word yielded in the first step will be further assigned a proper tag that indicates whether the known word is an independent segmented word by itself or a be­ginning/middle/ending component of an OOV word (Fu and Luke, 2004). In order to improve our system, part-of-speech information is also introduced in some tracks such as the PKU open test and the AS open test. Furthermore, a lexicalized HMM tagger is developed to per­form this task (Fu and Luke, 2004).</p><p>Given a sequence of known words <i>W = w1w2---wn </i>, the lexicalized HMM tagger attempt to find an appropriate sequence of tags <i>T = t1t2 •• -tn </i>that maximizes the conditional prob­ability <i>P(T </i>| <i>W </i>), namely <i>T = </i>argmax P(T | <i>W </i>) three criteria: (1) Each character in the training corpus is taken as an independent entry and col­lected into the relevant system dictionary. (2) A standard Chinese word in the training corpus will enter to the relevant dictionary if it has four or less Chinese characters within it, and at the same time, its counts of occurrence in the corpus is observed to be larger than a threshold. In our current system, the threshold is set to 10 for the AS closed test and 5 for other closed tests. (3) For non-standard Chinese words such as nu­meral expressions, English words and punctua­tions, if they consist of multiple characters, they will be not included in the system dictionary.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(2)</doubt><doubt alpha="51.2" length="41" tooSmall="False" monospace="0.0">&gt; arg maxjjP(w,|wt_,,tt)P(t,. |wt_,,tt_i)</doubt><p>As for the open test, some other dictionaries are applied. As can be seen from Table 2, <i>the CKIP Lexicon and Chinese Grammar </i>is used in both AS and CityU open test, and the <i>Gram­matical Knowledge-base of Contemporary Chi­nese </i>developed by the Peking University is utilized in both PKU and MSR open test.</p></section><section number="3" title="Settings for Different Tracks"><p>Table 1. Training corpora for different tracks Table 1 presents the corpora used to train our system for different tracks. In the Academia Sinica (AS) open test and the Peking University (PKU) open test, our system is trained respec­tively using the <i>Sinica Corpus </i>(3.0) and <i>the PFR Corpus. </i>In all other tests, including all closed tests, City University of Hong Kong (CityU) open test and Microsoft Research (MSR) open test, we trained our system using the relevant training corpora provided for the bakeoff.</p><p>In the closed test, the system dictionaries are derived automatically from the relevant training corpora for this bakeoff by using the following</p><p>It should be noted that part-of-speech infor­mation is also utilized in the AS open test and the PKU open test, because part-of-speech in­formation proved to be informative in identify­ing OOV words in Chinese text (Fu and Luke, 2004). Therefore, the training corpora for the two tests are tagged with part-of-speech, and entries in the relevant dictionaries are defined with their potential part-of-speech categories.</p><table caption="Table 2. System dictionaries for different tracks" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Track</p></td><td class="cell"><p>System dictionary</p></td><td class="cell"><p># of entries</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-O</p></td><td class="cell"><p><i>The CKIP Lexicon and Chinese Grammar</i></p></td><td class="cell"><p>84K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-C</p></td><td class="cell"><p>Automatically extracted from AS corpus for Bakeoff 2005</p></td><td class="cell"><p>30K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-O</p></td><td class="cell"><p><i>The CKIP Lexicon and Chinese Grammar </i>(without part-of-speech)</p></td><td class="cell"><p>84K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-C</p></td><td class="cell"><p>Automatically extracted from CityU corpus for Bakeoff 2005</p></td><td class="cell"><p>22K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-O</p></td><td class="cell"><p><i>The Grammatical Knowledge­base of Contemporary Chinese </i>(without part-of-speech)</p></td><td class="cell"><p>65K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-C</p></td><td class="cell"><p>Automatically extracted from MSR corpus for Bakeoff 2005</p></td><td class="cell"><p>17K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-O</p></td><td class="cell"><p><i>The Grammatical Knowledge­base of Contemporary Chinese</i></p></td><td class="cell"><p>65K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-C</p></td><td class="cell"><p>Automatically extracted from PKU corpus for Bakeoff 2005</p></td><td class="cell"><p>17K</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 2 shows all the dictionaries used in our system for different tracks." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Track</p></td><td class="cell"><p>Training Corpus</p></td><td class="cell"><p>Word counts</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-O</p></td><td class="cell"><p><i>The Sinica Corpus 3.0</i></p></td><td class="cell"><p>5,692,150</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-C</p></td><td class="cell"><p>AS corpus for Bakeoff 2005</p></td><td class="cell"><p>5,449,698</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-O</p></td><td class="cell"><p>CityU corpus for Bakeoff 2005</p></td><td class="cell"><p>1,455,629</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-C</p></td><td class="cell"><p>CityU corpus for Bakeoff 2005</p></td><td class="cell"><p>1,455,629</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-O</p></td><td class="cell"><p>MSR corpus for Bakeoff 2005</p></td><td class="cell"><p>2,368,391</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-C</p></td><td class="cell"><p>MSR corpus for Bakeoff 2005</p></td><td class="cell"><p>2,368,391</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-O</p></td><td class="cell"><p><i>The PFR Corpus</i></p></td><td class="cell"><p>7,286,960</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-C</p></td><td class="cell"><p>PKU corpus for Bakeoff 2005</p></td><td class="cell"><p>1,109,947</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="3" global="167"/></section><section number="4" title="The Scored Results"><p>In Bakeoff 2005, six measures are employed to score the performance of a word segmentation system, namely recall (R), precision (P), the evenly-weighted F-measure (F), out-of-vocabulary (OOV) rate for the test corpus, recall with respect to OOV words (ROOV) or in-vocabulary words (Riv).</p><p>In order to achieve a consistent evaluation of our system in both the closed test and the open test, OOV is defined in this paper as the set of words in the test corpus but not occurring in both the training corpus and the system diction­ary. Furthermore, the additional two rates, i.e. OOV-C and OOV-D are used to denote the out-of-vocabulary rate with respect to the training corpus and the out-of-vocabulary rate with respect to the system dictionary, respectively. At the same time, the precision with regard to in-vocabulary words (Piv) and OOV words (Poov) are also computed in this paper to give a more complete evalution of our system in unknown word identification.</p><p>able 4. Scores for different tracks</p><p>The OOV rates and scores of our system are summarized respectively in Table 3 and Table 4. The results show that our system can achieve a F-measure of 0.940-0.967 for different testing corpora while the relevant OOV rates are from 0.023 to 0.074.</p><p>Although our system has achieved a promis­ing performance, there is still much to be done to improve it. Firstly, our system is purely statis­tics-based, it cannot yield correct segmentations for all non-standard words (NSWs) such as nu­meral expressions and English strings in Chi­nese text. Secondly, known word segmentation and unknown word identification are taken as two independent stages in our system. This strategy is obviously simple and more easily applicable (Fu and Luke, 2003). Although the known word bigram model can partly resolve this problem, it is not always effective for some complicated strings that contains a mixture of ambiguities and unknown words, such as "31 R and the fragment "^fi^H" in the sen­tence ^^^raiTiilll".</p></section><section number="5" title="Conclusions"><p>This paper presents a two-stage statistical word segmentation system for Chinese. We partici­pated in all testing tracks at the second Sighan bakeoff. The scored results show that our system can achieve a F-measure of 0.940-0.967 as a whole for different corpora. This indicates that the proposed system is effective for most am­biguous segmentations and unknown words in Chinese test. For future work, we hope to im­prove our system by incorporating some pattern rules to handle complicated ambiguous frag­ments and non-standard words in Chinese text.</p><table caption="Table 3. OOV rates for different tracks" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Track</p></td><td class="cell"><p>OOV-C</p></td><td class="cell"><p>OOV-D</p></td><td class="cell"><p>OOV</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-O</p></td><td class="cell"><p>0.043</p></td><td class="cell"><p>0.096</p></td><td class="cell"><p>0.039</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-C</p></td><td class="cell"><p>0.043</p></td><td class="cell"><p>0.096</p></td><td class="cell"><p>0.043</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-O</p></td><td class="cell"><p>0.074</p></td><td class="cell"><p>0.140</p></td><td class="cell"><p>0.049</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-C</p></td><td class="cell"><p>0.074</p></td><td class="cell"><p>0.127</p></td><td class="cell"><p>0.074</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-O</p></td><td class="cell"><p>0.026</p></td><td class="cell"><p>0.076</p></td><td class="cell"><p>0.023</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-C</p></td><td class="cell"><p>0.026</p></td><td class="cell"><p>0.087</p></td><td class="cell"><p>0.026</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-O</p></td><td class="cell"><p>0.038</p></td><td class="cell"><p>0.070</p></td><td class="cell"><p>0.033</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-C</p></td><td class="cell"><p>0.058</p></td><td class="cell"><p>0.091</p></td><td class="cell"><p>0.058</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Track</p></td><td class="cell"><p>F</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p></p></td><td class="cell"><p>Piv</p></td><td class="cell"><p>ROOV</p></td><td class="cell"><p>POOV</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-O</p></td><td class="cell"><p>0.946</p></td><td class="cell"><p>0.955</p></td><td class="cell"><p>0.938</p></td><td class="cell"><p>0.972</p></td><td class="cell"><p>0.947</p></td><td class="cell"><p>0.532</p></td><td class="cell"><p>0.660</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AS-C</p></td><td class="cell"><p>0.940</p></td><td class="cell"><p>0.947</p></td><td class="cell"><p>0.934</p></td><td class="cell"><p>0.966</p></td><td class="cell"><p>0.949</p></td><td class="cell"><p>0.523</p></td><td class="cell"><p>0.566</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-O</p></td><td class="cell"><p>0.941</p></td><td class="cell"><p>0.944</p></td><td class="cell"><p>0.938</p></td><td class="cell"><p>0.962</p></td><td class="cell"><p>0.956</p></td><td class="cell"><p>0.592</p></td><td class="cell"><p>0.599</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CityU-C</p></td><td class="cell"><p>0.939</p></td><td class="cell"><p>0.944</p></td><td class="cell"><p>0.933</p></td><td class="cell"><p>0.969</p></td><td class="cell"><p>0.952</p></td><td class="cell"><p>0.626</p></td><td class="cell"><p>0.677</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-O</p></td><td class="cell"><p>0.967</p></td><td class="cell"><p>0.969</p></td><td class="cell"><p>0.966</p></td><td class="cell"><p>0.978</p></td><td class="cell"><p>0.977</p></td><td class="cell"><p>0.586</p></td><td class="cell"><p>0.537</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSR-C</p></td><td class="cell"><p>0.962</p></td><td class="cell"><p>0.962</p></td><td class="cell"><p>0.962</p></td><td class="cell"><p>0.972</p></td><td class="cell"><p>0.977</p></td><td class="cell"><p>0.592</p></td><td class="cell"><p>0.499</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-O</p></td><td class="cell"><p>0.962</p></td><td class="cell"><p>0.959</p></td><td class="cell"><p>0.965</p></td><td class="cell"><p>0.963</p></td><td class="cell"><p>0.970</p></td><td class="cell"><p>0.835</p></td><td class="cell"><p>0.816</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU-C</p></td><td class="cell"><p>0.944</p></td><td class="cell"><p>0.943</p></td><td class="cell"><p>0.944</p></td><td class="cell"><p>0.961</p></td><td class="cell"><p>0.958</p></td><td class="cell"><p>0.656</p></td><td class="cell"><p>0.700</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Guohong Fu, and Kang-Kwong Luke. 2003. A two-stage statistical word segmentation system for Chinese. In: <i>Proceedings of the 2nd SIGHAN Workshop on Chinese Language Processing, </i>Sap­poro, Japan, 156-159.</p><p>Guohong Fu, and Kang-Kwong Luke. 2004. Chinese unknown word identification as known word tag­ging. In: <i>Proceedings of the Third IEEE Interna­tional Conference on Machine Learning and Cybernetics (ICMLC 2004), </i>Shanghai, China, 2612-2617.</p><p>Guohong Fu, and Kang-Kwong Luke. 2005. Chinese unknown word identification using class-based</p><p>LM. <i>Lecture Notes in Computer Science (IJCNLP 2004), </i>3248: 704-713.</p></references></body></article>