<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="69"/><title>The Fourth International Chinese Language Processing Bakeoff: Chinese Word Segmentation, Named Entity Recognition and Chinese POS Tagging</title><pubinfo>Sixth SIGHAN Workshop on Chinese Language Processing</pubinfo><author surname="Jin" givenname="Guangjin"><org  name="City University of Hong Kong" country="Hong Kong" city="Kowloon"/></author><author surname="Chen" givenname="Xiao"><org  name="University of Hong Kong" country="Hong Kong" city="Pokfulam"/></author></firstpageheader><frontmatter><p><b>The Fourth International Chinese Language Processing Bakeoff: Chinese Word Segmentation, Named Entity Recognition and Chinese POS Tagging</b></p><p><b>Guangjin Jin Xiao Chen</b></p><p>Institute of Applied linguistics Dept of Chinese, Translation &amp; Linguistics</p><p>M.O.E., P.R.C. City University of Hong Kong</p><p>No.51, Chaonei Nanxiaojie 83 Tat Chee Avenue</p><p>Dong Cheng District, Beijing, China Kowloon, Hong Kong, China</p><p>guangjin2 0 0 0@163.com cxiao2@student.cityu.edu.hk</p></frontmatter><abstract>The Fourth International Chinese Language Processing Bakeoff was held in 2007 to as­sess the state of the art in three important tasks: Chinese word segmentation, named entity recognition and Chinese POS tagging. Twenty-eight groups submitted result sets in the three tasks across two tracks and a total of seven corpora. Strong results have been found in all the tasks as well as continuing challenges. </abstract></header><body><section number="1" title="Introduction"><p>Chinese is a kind of language which does not use word delimiters in its writing system. Now a days, under the background of information explosion, many application oriented natural language process­ing task become more and more important, such as parsing and machine translation. Chinese tokeniza-tion, as the foundation of many downstream pro­cessing tasks, has attracted lots of research interest. However, it is still a significant challenge for all the researchers.</p><p>SIGHAN, the Special Interest Group for Chinese Language Processing of the Association for Compu­tational Linguistics, conducted three prior word seg­mentation bakeoffs, in 2003, 2005 and 2006(Sproat and Emerson, 2003; Emerson, 2005; Levow, 2006), which established benchmarks for word segmenta­tion and named entity recognition. The bakeoff pre­sentations at SIGHAN workshops highlighted new approaches in this field.</p><p>The fourth bakeoff was jointly held with the First CIPS Chinese Language Processing Evaluation in the summer of 2007, and co-organized by SIGHAN, Chinese LDC, and the Verifying Center of Chinese Language and Character Standards of the State Lan­guage Commission of P.R.C. In this bakeoff, we continue the Chinese word segmentation and named entity recognition tasks. Furthermore, a new evalu­ation task has been augmented, the task for Chinese POS tagging. In this evaluation task, a participating system will take a given segmented corpus as the in­put, and only the POS tagging performance will be evaluated. Both closed and open track are available for this task.</p></section><section number="2" title="Details of the Evaluation 2.1 Corpora"><p>Seven corpora were provided for the evaluation: five in Simplified characters and two in traditional characters. The Simplified character corpora were provided by Microsoft Research Asia (MSRA) for NER, by University of Pennsylvania/University of Colorado (CTB) for WS and POS tagging, by Peking University for NER and POS tagging, by Shanxi University for WS. The Traditional char­acter corpora were provided by City University of Hong Kong (CITYU) for WS, NER and POS tag­ging, by the Chinese Knowledge Information Pro­cessing Laboratory (CKIP) of the Academia Sinica, Taiwan for WS and POS tagging. Each data provider offered separate training and test corpora. Statistical information for each corpus appears in Table1. All data providers were requested to supply the training and test corpora in both the standard local encoding and in Unicode (UTF-16).<page local="2" global="70"/> For all providers, missing encodings were transcoded by the organizers using the appropriate software. Primary training and truth data for word segmentation were generated by the organizers via a C++ program by uniforming sen­tence end tags and delimiters. For test data, all tags removed except sentence end tags.</p><p>Comparable XML format data was also provided for all corpora and all tasks. Except as noted above, no additional changes were made to the data fur­nished by the providers.</p><table caption="Table 1: Corpora for Bakeoff-4"></table><p>aTAG:Chinese POS tagging</p><p>In the closed track, participants could only use information found in the provided training data. Groups were required to submit fully automatic runs and were prohibited from testing on corpora which they had previously used.</p><p>Scoring was performed automatically using a C++ program. In cases where naming errors or mi­nor divergences from required file formats arose, a mix of manual intervention and automatic conver­sion was employed to enable scoring. The primary scoring program was made available to participants for follow up experiments.</p></section><section number="3" title="Participating sites"><p>A total of 42 sites registered, and 28 submitted re­sults for scoring. A summary of participating groups with task and track information appears in Table 2. A total of 263 official runs were scored: 166 for word segmentation, 33 for named entity recognition and 64 for POS tagging.</p><subsection number="2.2" title="Rules and Procedures"><p>The fourth Bakeoff followed the structure of the for­mer three word segmentation bakeoffs. The only difference is that participating groups ("sites") reg­istered online and for those who could not access our web site, email registration is acceptable; On registration, all the groups are asked to identify the corpora and tasks of interest. Training data was re­leased for download from the online registration sys­tem on August 25, 2007. Test data was released on September 25, 2007 and results were due 12:00 Bei­jing Time on September 28,2007. Scores for all sub­mitted runs were emailed to the individual groups on October 15, and were made available to all groups on a web page a few days later.</p><p>Groups could participate in either or both of two tracks for each task and corpus:</p><p>In the open track, participants could use any ex­ternal data they chose in addition to the provided training data. Groups were required to specify this information in their system descriptions.</p></subsection></section><section number="4" title="Results and Discussion"><subsection number="4.1" title="Word Segmentation Results &amp; Discussion"><p>There are five corpus provided in the CWS track. The statistics for these corpora are in Table 3. We introduce a type-token ration(TTR) to indicate the vocabulary diversity in each corpus.</p><p>To provide a basis for comparison, we computed baseline and possible topline scores for each of the corpora. The baseline was constructed by left-to-right maximal match algorithm, using the train­ing corpus vocabulary. The topline employed the same procedure, but instead used the test vocabu­lary. These results are shown in Tables 5 and 6. For the CWS task, we computed the following mea­sures: recall (R), precision (P), equally weighted F-measure <i>(F</i><i> = 2PR/</i>(P + R)), the recall, preci­sion and F-measure on OOV (Roov, Poov, Foov), and recall, precision and F-measure on in vocabu­lary words (R/y, P/y, F/y). In and out of vocabu­lary status are defined relative to the training corpus. Following previous bakeoffs, we employ the Central Limit Theorem for Bernoulli trials (Grinstead and<page local="3" global="71"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>Encoding</p></td><td class="cell"><p>CWS</p></td><td class="cell"><p>NER</p></td><td class="cell"><p>TAG"</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>BIG5HKSCS/UTF-16</p></td><td class="cell"><p></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p>BIG5/UTF-16</p></td><td class="cell"><p>/</p></td><td class="cell"><p></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p>GB/UTF-16</p></td><td class="cell"><p><i>V</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSRA</p></td><td class="cell"><p>GB/UTF-16</p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p>GB/UTF-16</p></td><td class="cell"><p><i>V</i></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU</p></td><td class="cell"><p>GB/UTF-16</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>V</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SXU</p></td><td class="cell"><p>GB/UTF-16</p></td><td class="cell"><p><i>V</i></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">UGAPT</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">p</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">C</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">o</doubt><doubt alpha="80.0" length="5" tooSmall="False" monospace="0.0">bo bo</doubt><doubt alpha="60.0" length="10" tooSmall="True" monospace="0.0">cd  cd TO1</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">b</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">CG</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">NCGAT</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">p o p</doubt><doubt alpha="100.0" length="6" tooSmall="False" monospace="0.0">BGTACT</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">p o</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">-9-9</doubt><p>IPG IGA CT no og iey Hin</p><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">U Y</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">ITYGA</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">CT</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">A</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">bp o</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">P</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">es</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">u o</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">O</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">CO</doubt><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">&amp;</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">u</doubt><doubt alpha="50.0" length="8" tooSmall="False" monospace="0.0">t/3 oc_e</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">Oh</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">cn</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">I</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">U</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">TYR</doubt><doubt alpha="100.0" length="6" tooSmall="False" monospace="0.0">ITECIN</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">pp o</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">US</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">XWSC</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">13"</doubt><doubt alpha="66.7" length="9" tooSmall="True" monospace="0.0">cd  cd cd</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">u uq0</doubt><doubt alpha="44.4" length="9" tooSmall="False" monospace="0.0">p   p p o</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">CS</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">CWNC</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">n-t^T</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">O u</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">1?</doubt><doubt alpha="54.5" length="11" tooSmall="False" monospace="0.0">p   p bo bo</doubt><doubt alpha="50.0" length="2" tooSmall="True" monospace="0.0">n-</doubt><doubt alpha="60.0" length="5" tooSmall="True" monospace="0.0">CS C3</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">CC</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">tET2T1?</doubt><doubt alpha="75.0" length="8" tooSmall="True" monospace="0.0">cd cd cd</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">IPS</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">KIW</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">P P</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">YS</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">ITW</doubt><doubt alpha="66.7" length="3" tooSmall="True" monospace="0.0">o c</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">e c</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">3^</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">Un</doubt><doubt alpha="30.8" length="13" tooSmall="False" monospace="0.0">ja og   ^ ^ ^</doubt><doubt alpha="7.1" length="14" tooSmall="False" monospace="0.0">Ü ±£    $ ^ =«</doubt><doubt alpha="33.3" length="24" tooSmall="False" monospace="0.0">O °OOin^  &gt;y      -9 'SO</doubt><doubt alpha="9.1" length="11" tooSmall="False" monospace="0.0">&lt; -S3 ^ ^ ^</doubt><doubt alpha="42.9" length="7" tooSmall="False" monospace="0.0">o ^ J g</doubt><doubt alpha="18.2" length="11" tooSmall="False" monospace="0.0">^ !^ ^ ^ Ja</doubt><doubt alpha="28.6" length="7" tooSmall="False" monospace="0.0">^3&amp;^eg^</doubt><doubt alpha="100.0" length="4" tooSmall="True" monospace="0.0">ODOS</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">nin</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">N</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">g n</doubt><doubt alpha="80.0" length="5" tooSmall="True" monospace="0.0">CS CS</doubt><doubt alpha="0.0" length="11" tooSmall="False" monospace="0.0">"= ^ '-^ '3</doubt><doubt alpha="50.0" length="12" tooSmall="True" monospace="0.0">tu o -SS .J1</doubt><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">cnoui,P</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">cieiC</doubt><doubt alpha="40.0" length="5" tooSmall="False" monospace="0.0">H -'S</doubt><doubt alpha="40.0" length="5" tooSmall="True" monospace="0.0">o c ^</doubt><doubt alpha="11.1" length="9" tooSmall="False" monospace="0.0">&gt;, 2 3 c«</doubt><doubt alpha="66.7" length="6" tooSmall="True" monospace="0.0">n es c</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">sT ^</doubt><doubt alpha="60.0" length="10" tooSmall="False" monospace="0.0">.aCD  Og o</doubt><doubt alpha="40.0" length="10" tooSmall="False" monospace="0.0">uö «I ^'3s</doubt><doubt alpha="25.0" length="8" tooSmall="False" monospace="0.0">^ ^ tu ^</doubt><doubt alpha="0.0" length="9" tooSmall="True" monospace="0.0">^^.^'^^^^</doubt><doubt alpha="22.2" length="9" tooSmall="False" monospace="0.0">•505S°S5^</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">ni</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">g</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">n c</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">eT</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">d</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">nu</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">toes</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">iencM</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">cieC</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">ScES</doubt><doubt alpha="80.0" length="5" tooSmall="False" monospace="0.0">V inS</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">og-a</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">Ineec</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">IBsee</doubt><doubt alpha="50.0" length="4" tooSmall="True" monospace="0.0">^ tu</doubt><p>n n <b>o </b>rl <b>tu</b></p><doubt alpha="33.3" length="9" tooSmall="False" monospace="0.0">^   -Bu;j</doubt><doubt alpha="41.7" length="12" tooSmall="False" monospace="0.0">^  H   tdo S</doubt><doubt alpha="100.0" length="7" tooSmall="False" monospace="0.0">CseniTF</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">3^5^</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">eni</doubt><doubt alpha="100.0" length="1" tooSmall="True" monospace="0.0">n</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">J3</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">e</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">d n</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">in</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">S3</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">H</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">ecT</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">r</doubt><doubt alpha="44.4" length="9" tooSmall="True" monospace="0.0">t^ p t^ n</doubt><doubt alpha="75.0" length="4" tooSmall="True" monospace="0.0">n tS</doubt><doubt alpha="42.9" length="7" tooSmall="False" monospace="0.0">&lt;^ la S</doubt><doubt alpha="50.0" length="6" tooSmall="True" monospace="0.0">n -n O</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">2'tS</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">tu</doubt><doubt alpha="100.0" length="3" tooSmall="True" monospace="0.0">nJo</doubt><doubt alpha="75.0" length="4" tooSmall="True" monospace="0.0">c tu</doubt><doubt alpha="62.5" length="8" tooSmall="False" monospace="0.0">tu -'JcP</doubt><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">Ic</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">-q</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">c</doubt><doubt alpha="25.0" length="4" tooSmall="False" monospace="0.0">■a 3</doubt><doubt alpha="25.0" length="4" tooSmall="False" monospace="0.0">&gt;&lt; a</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">INin</doubt><doubt alpha="0.0" length="5" tooSmall="False" monospace="0.0">"•^■^</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">c§ 3o3</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">Kc</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">%'S</doubt><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">1</doubt><doubt alpha="31.6" length="76" tooSmall="True" monospace="0.0">\^rt     fr)ko     bo    b\        (N en ht    kor^-00o     o^     m htr^- o</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">RR</doubt><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">SEMN</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">e­r</doubt><p><b>o     </b><i>&lt;z .</i></p><doubt alpha="22.2" length="9" tooSmall="False" monospace="0.0">£&gt;,=q33 a</doubt><doubt alpha="9.1" length="11" tooSmall="False" monospace="0.0">^.^^^ :=^S§</doubt><page local="4" global="72"/><p>Snell, 1997) to compute 95% confidence interval as</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">Vn</doubt><p>Chinese Word Segmentation results for all runs grouped by corpus and track appear in Tables 6-15; all tables are sorted by F-score.</p><p>Across all corpora, the best closed track F-score was achieved in the SXU corpus at 0.9623.In the open track, two systems that has exceeded the topline in the CTB corpus, and there are also three runs approaching the topline. This might because of the overlapping of testing data in this bakeoff and the training data in the last bakeoff.</p><p>According to the statistics on all the corpus for this bakeoff, there is no clear negative linear cor­relation between the OOV rate of a corpus and the highest score achieved on it, since the OOV words are not the only obstacle for segmentation systems to overcome.</p><p>There are some difference in the segmentation scoring system between this bakeoff and the for­mer ones. The precision and F-measure for both IV and OOV are appended. It could be observed that, from the result tables in every corpus, the highest total F-measure is always coming up with the high­est OOV and IV F-measure rather than the recall of them. So, we consider the F-measure of both IV and OOV words a more powerful indicator for the performance of the segmentation systems in some sense.</p></subsection><subsection number="4.2" title="Named Entity Recognition Results &amp; Discussion"><p>There are only two corpus CITYU and MSRA for named entity recognition task in this bakeoff. For statistics, we compute the OOV rate of named en­tities for each corpus, which denotes the proportion of named entities in testing data that are not seen in training corpus.</p><p>For each submission for named entity recogni­tion, like the former bakeoff, we compute over­all phrase precision (P), recall(R), and F-measure (F), as well as the F-measure for each entity type (PER,ORG,LOC). The only difference is the recall and precision for each entity type is appended.</p><p>We compute a baseline for each corpus as in the bakeoff-3. A left-to-right maximum match algo­rithm was applied on the testing data with a named entity list generated from the training data. This al­gorithm only detects those named entities with one unique tag in training data, others are considered as incorrectly tagged. These scores for all NER corpora are found in Table 18.</p><p>Named entity recognition results for all runs grouped by corpus and track appear in Tables 19-22; all tables are sorted by F-score.</p><p>It is shown in the result table that the baseline and the system performance for MSRA corpus are better than those for CITYU corpus. However,the statistics is showing that the number of named en­tities in CITYU training corpus is twice as large as the number in MSRA corpus. The system per­formance for these two corpus are consist with the OOV rate for these two corpora. Therefore,it seems that OOV named entities is a principal challenge for named entity recognition systems. Furthermore, the F-measure of organization name recognition is the lowest one in every participant's result on ev­ery corpus. This phenomenon is potentially imply­ing that the organization name is the most difficult one among the three categories of named entities.</p><p>There are several systems participating both the closed and open track on the same corpus. All of them perform better in the open track. This phe­nomenon is implying that proper external informa­tion can strongly affect the performance of named entity recognition system.</p><p>since the testing data MSRA is a subset of the training data for last bakeoff, two sites have achieved novelly high scores in the open track.</p></subsection><subsection number="4.3" title="POS Tagging Result &amp; Discussion"><p>There are five corpora in the Chinese POS tagging task, each of them is built on different tag set and tagging standard. For statistics and evaluation, we define several terms for this task:</p><p>• Multi-tag words: the words that been assigned more than one POS-tag in either the training corpus or testing corpus. For instance, if an IV</p><page local="5" global="73"/><table caption="Table 3: Chinese Word Segmentation Training and Truth data statistics"></table><p>aWT: word type.</p><p>bTTR: type-token ratio = type count / token count. cOOV: number of OOV. dROOV: OOV Rate<page local="6" global="74"/><page local="7" global="75"/><page local="8" global="76"/></p><table caption="Table 4: Chinese Word Segmentation Baseline" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>Token</p></td><td class="cell"><p>Training WT"</p></td><td class="cell"><p>TTRb</p></td><td class="cell"><p>Token</p></td><td class="cell"><p>WT</p></td><td class="cell"><p>Truth TTR</p></td><td class="cell"><p>OOVc</p></td><td class="cell"><p>Roov d</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>1092687</p></td><td class="cell"><p>43639</p></td><td class="cell"><p>0.0399</p></td><td class="cell"><p>235631</p></td><td class="cell"><p>23303</p></td><td class="cell"><p>0.0989</p></td><td class="cell"><p>19382</p></td><td class="cell"><p>0.0823</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p>721549</p></td><td class="cell"><p>48114</p></td><td class="cell"><p>0.0667</p></td><td class="cell"><p>90678</p></td><td class="cell"><p>14662</p></td><td class="cell"><p>0.1617</p></td><td class="cell"><p>6718</p></td><td class="cell"><p>0.0741</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p>642246</p></td><td class="cell"><p>42159</p></td><td class="cell"><p>0.0656</p></td><td class="cell"><p>80700</p></td><td class="cell"><p>12188</p></td><td class="cell"><p>0.1510</p></td><td class="cell"><p>4480</p></td><td class="cell"><p>0.0555</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p>913466</p></td><td class="cell"><p>58592</p></td><td class="cell"><p>0.0641</p></td><td class="cell"><p>152354</p></td><td class="cell"><p>21352</p></td><td class="cell"><p>0.1401</p></td><td class="cell"><p>7218</p></td><td class="cell"><p>0.0474</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SXU</p></td><td class="cell"><p>528238</p></td><td class="cell"><p>32484</p></td><td class="cell"><p>0.0614</p></td><td class="cell"><p>113527</p></td><td class="cell"><p>12428</p></td><td class="cell"><p>0.1095</p></td><td class="cell"><p>5815</p></td><td class="cell"><p>0.0512</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 6: CITYU: Word Segmentation: Closed Track" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>.9006</p></td><td class="cell"><p>.8225</p></td><td class="cell"><p>.8598</p></td><td class="cell"><p>.0970</p></td><td class="cell"><p>.2262</p></td><td class="cell"><p>.1358</p></td><td class="cell"><p>.9727</p></td><td class="cell"><p>.8424</p></td><td class="cell"><p>.9029</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p>.8978</p></td><td class="cell"><p>.8232</p></td><td class="cell"><p>.8589</p></td><td class="cell"><p>.0208</p></td><td class="cell"><p>.0678</p></td><td class="cell"><p>.0319</p></td><td class="cell"><p>.9680</p></td><td class="cell"><p>.8393</p></td><td class="cell"><p>.8990</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p>.8864</p></td><td class="cell"><p>.8427</p></td><td class="cell"><p>.8640</p></td><td class="cell"><p>.0283</p></td><td class="cell"><p>.0769</p></td><td class="cell"><p>.0414</p></td><td class="cell"><p>.9369</p></td><td class="cell"><p>.8579</p></td><td class="cell"><p>.8956</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p>.9200</p></td><td class="cell"><p>.8716</p></td><td class="cell"><p>.8951</p></td><td class="cell"><p>.0273</p></td><td class="cell"><p>.1858</p></td><td class="cell"><p>.0476</p></td><td class="cell"><p>.9644</p></td><td class="cell"><p>.8761</p></td><td class="cell"><p>.9181</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SXU</p></td><td class="cell"><p>.9238</p></td><td class="cell"><p>.8679</p></td><td class="cell"><p>.8949</p></td><td class="cell"><p>.0251</p></td><td class="cell"><p>.0867</p></td><td class="cell"><p>.0389</p></td><td class="cell"><p>.9723</p></td><td class="cell"><p>.8789</p></td><td class="cell"><p>.9232</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 5: Chinese Word Segmentation Topline</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>.9787</p></td><td class="cell"><p>.9840</p></td><td class="cell"><p>.9813</p></td><td class="cell"><p>.9917</p></td><td class="cell"><p>.9678</p></td><td class="cell"><p>.9796</p></td><td class="cell"><p>.9775</p></td><td class="cell"><p>.9855</p></td><td class="cell"><p>.9815</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p>.9823</p></td><td class="cell"><p>.9880</p></td><td class="cell"><p>.9852</p></td><td class="cell"><p>.9932</p></td><td class="cell"><p>.9642</p></td><td class="cell"><p>.9784</p></td><td class="cell"><p>.9815</p></td><td class="cell"><p>.9900</p></td><td class="cell"><p>.9857</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p>.9710</p></td><td class="cell"><p>.9825</p></td><td class="cell"><p>.9767</p></td><td class="cell"><p>.9920</p></td><td class="cell"><p>.9707</p></td><td class="cell"><p>.9812</p></td><td class="cell"><p>.9698</p></td><td class="cell"><p>.9832</p></td><td class="cell"><p>.9764</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p>.9735</p></td><td class="cell"><p>.9817</p></td><td class="cell"><p>.9776</p></td><td class="cell"><p>.9933</p></td><td class="cell"><p>.9203</p></td><td class="cell"><p>.9554</p></td><td class="cell"><p>.9725</p></td><td class="cell"><p>.9850</p></td><td class="cell"><p>.9787</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SXU</p></td><td class="cell"><p>.9820</p></td><td class="cell"><p>.9867</p></td><td class="cell"><p>.9844</p></td><td class="cell"><p>.9942</p></td><td class="cell"><p>.9480</p></td><td class="cell"><p>.9705</p></td><td class="cell"><p>.9813</p></td><td class="cell"><p>.9890</p></td><td class="cell"><p>.9851</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9526</p></td><td class="cell"><p>.000875</p></td><td class="cell"><p>.9493</p></td><td class="cell"><p>.000903</p></td><td class="cell"><p>.9510</p></td><td class="cell"><p>.7495</p></td><td class="cell"><p>.7912</p></td><td class="cell"><p>.7698</p></td><td class="cell"><p>.9708</p></td><td class="cell"><p>.9626</p></td><td class="cell"><p>.9667</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9513</p></td><td class="cell"><p>.000887</p></td><td class="cell"><p>.9430</p></td><td class="cell"><p>.000955</p></td><td class="cell"><p>.9471</p></td><td class="cell"><p>.7339</p></td><td class="cell"><p>.7752</p></td><td class="cell"><p>.7540</p></td><td class="cell"><p>.9707</p></td><td class="cell"><p>.9570</p></td><td class="cell"><p>.9638</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p></p></td><td class="cell"><p>.9465</p></td><td class="cell"><p>.000927</p></td><td class="cell"><p>.9443</p></td><td class="cell"><p>.000945</p></td><td class="cell"><p>.9454</p></td><td class="cell"><p>.7721</p></td><td class="cell"><p>.7244</p></td><td class="cell"><p>.7475</p></td><td class="cell"><p>.9621</p></td><td class="cell"><p>.9653</p></td><td class="cell"><p>.9637</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9450</p></td><td class="cell"><p>.000939</p></td><td class="cell"><p>.9437</p></td><td class="cell"><p>.000949</p></td><td class="cell"><p>.9443</p></td><td class="cell"><p>.7716</p></td><td class="cell"><p>.7099</p></td><td class="cell"><p>.7395</p></td><td class="cell"><p>.9605</p></td><td class="cell"><p>.9666</p></td><td class="cell"><p>.9636</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>26</p></td><td class="cell"><p></p></td><td class="cell"><p>.9490</p></td><td class="cell"><p>.000906</p></td><td class="cell"><p>.9372</p></td><td class="cell"><p>.000999</p></td><td class="cell"><p>.9430</p></td><td class="cell"><p>.6780</p></td><td class="cell"><p>.7591</p></td><td class="cell"><p>.7163</p></td><td class="cell"><p>.9733</p></td><td class="cell"><p>.9511</p></td><td class="cell"><p>.9621</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9421</p></td><td class="cell"><p>.000962</p></td><td class="cell"><p>.9339</p></td><td class="cell"><p>.001023</p></td><td class="cell"><p>.9380</p></td><td class="cell"><p>.7074</p></td><td class="cell"><p>.7050</p></td><td class="cell"><p>.7062</p></td><td class="cell"><p>.9631</p></td><td class="cell"><p>.9543</p></td><td class="cell"><p>.9587</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9367</p></td><td class="cell"><p>.001003</p></td><td class="cell"><p>.9377</p></td><td class="cell"><p>.000996</p></td><td class="cell"><p>.9372</p></td><td class="cell"><p>.6295</p></td><td class="cell"><p>.7394</p></td><td class="cell"><p>.6800</p></td><td class="cell"><p>.9642</p></td><td class="cell"><p>.9526</p></td><td class="cell"><p>.9584</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>27</p></td><td class="cell"><p></p></td><td class="cell"><p>.9386</p></td><td class="cell"><p>.000988</p></td><td class="cell"><p>.9325</p></td><td class="cell"><p>.001033</p></td><td class="cell"><p>.9355</p></td><td class="cell"><p>.6708</p></td><td class="cell"><p>.6840</p></td><td class="cell"><p>.6773</p></td><td class="cell"><p>.9626</p></td><td class="cell"><p>.9541</p></td><td class="cell"><p>.9584</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9296</p></td><td class="cell"><p>.001054</p></td><td class="cell"><p>.9290</p></td><td class="cell"><p>.001058</p></td><td class="cell"><p>.9293</p></td><td class="cell"><p>.6862</p></td><td class="cell"><p>.6541</p></td><td class="cell"><p>.6698</p></td><td class="cell"><p>.9514</p></td><td class="cell"><p>.9549</p></td><td class="cell"><p>.9532</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>33</p></td><td class="cell"><p></p></td><td class="cell"><p>.9285</p></td><td class="cell"><p>.001061</p></td><td class="cell"><p>.9261</p></td><td class="cell"><p>.001077</p></td><td class="cell"><p>.9273</p></td><td class="cell"><p>.6866</p></td><td class="cell"><p>.6326</p></td><td class="cell"><p>.6585</p></td><td class="cell"><p>.9502</p></td><td class="cell"><p>.9548</p></td><td class="cell"><p>.9525</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9237</p></td><td class="cell"><p>.001093</p></td><td class="cell"><p>.9234</p></td><td class="cell"><p>.001095</p></td><td class="cell"><p>.9236</p></td><td class="cell"><p>.6830</p></td><td class="cell"><p>.5934</p></td><td class="cell"><p>.6350</p></td><td class="cell"><p>.9453</p></td><td class="cell"><p>.9579</p></td><td class="cell"><p>.9516</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9237</p></td><td class="cell"><p>.001093</p></td><td class="cell"><p>.9234</p></td><td class="cell"><p>.001095</p></td><td class="cell"><p>.9236</p></td><td class="cell"><p>.6830</p></td><td class="cell"><p>.5934</p></td><td class="cell"><p>.6350</p></td><td class="cell"><p>.9453</p></td><td class="cell"><p>.9579</p></td><td class="cell"><p>.9516</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9238</p></td><td class="cell"><p>.001093</p></td><td class="cell"><p>.9234</p></td><td class="cell"><p>.001095</p></td><td class="cell"><p>.9236</p></td><td class="cell"><p>.6830</p></td><td class="cell"><p>.5934</p></td><td class="cell"><p>.6351</p></td><td class="cell"><p>.9453</p></td><td class="cell"><p>.9579</p></td><td class="cell"><p>.9516</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9197</p></td><td class="cell"><p>.001119</p></td><td class="cell"><p>.9169</p></td><td class="cell"><p>.001137</p></td><td class="cell"><p>.9183</p></td><td class="cell"><p>.6558</p></td><td class="cell"><p>.5690</p></td><td class="cell"><p>.6093</p></td><td class="cell"><p>.9434</p></td><td class="cell"><p>.9532</p></td><td class="cell"><p>.9483</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>15</p></td><td class="cell"><p></p></td><td class="cell"><p>.9191</p></td><td class="cell"><p>.001123</p></td><td class="cell"><p>.9014</p></td><td class="cell"><p>.001228</p></td><td class="cell"><p>.9102</p></td><td class="cell"><p>.5466</p></td><td class="cell"><p>.5588</p></td><td class="cell"><p>.5527</p></td><td class="cell"><p>.9525</p></td><td class="cell"><p>.9308</p></td><td class="cell"><p>.9415</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9219</p></td><td class="cell"><p>.001105</p></td><td class="cell"><p>.8951</p></td><td class="cell"><p>.001262</p></td><td class="cell"><p>.9083</p></td><td class="cell"><p>.4703</p></td><td class="cell"><p>.5899</p></td><td class="cell"><p>.5234</p></td><td class="cell"><p>.9624</p></td><td class="cell"><p>.9159</p></td><td class="cell"><p>.9386</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9221</p></td><td class="cell"><p>.001104</p></td><td class="cell"><p>.8947</p></td><td class="cell"><p>.001264</p></td><td class="cell"><p>.9082</p></td><td class="cell"><p>.4697</p></td><td class="cell"><p>.5891</p></td><td class="cell"><p>.5227</p></td><td class="cell"><p>.9627</p></td><td class="cell"><p>.9155</p></td><td class="cell"><p>.9385</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9120</p></td><td class="cell"><p>.001167</p></td><td class="cell"><p>.8974</p></td><td class="cell"><p>.001250</p></td><td class="cell"><p>.9047</p></td><td class="cell"><p>.5263</p></td><td class="cell"><p>.5333</p></td><td class="cell"><p>.5297</p></td><td class="cell"><p>.9466</p></td><td class="cell"><p>.9290</p></td><td class="cell"><p>.9377</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.8884</p></td><td class="cell"><p>.001296</p></td><td class="cell"><p>.8817</p></td><td class="cell"><p>.001330</p></td><td class="cell"><p>.8850</p></td><td class="cell"><p>.6114</p></td><td class="cell"><p>.6030</p></td><td class="cell"><p>.6072</p></td><td class="cell"><p>.9133</p></td><td class="cell"><p>.9069</p></td><td class="cell"><p>.9101</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.0155</p></td><td class="cell"><p>.000509</p></td><td class="cell"><p>.0155</p></td><td class="cell"><p>.000508</p></td><td class="cell"><p>.0155</p></td><td class="cell"><p>.0047</p></td><td class="cell"><p>.0049</p></td><td class="cell"><p>.0048</p></td><td class="cell"><p>.0165</p></td><td class="cell"><p>.0164</p></td><td class="cell"><p>.0165</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 7: CITYU: Word Segmentation: Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9670</p></td><td class="cell"><p>.000736</p></td><td class="cell"><p>.9725</p></td><td class="cell"><p>.000674</p></td><td class="cell"><p>.9697</p></td><td class="cell"><p>.8988</p></td><td class="cell"><p>.8525</p></td><td class="cell"><p>.8750</p></td><td class="cell"><p>.9731</p></td><td class="cell"><p>.9839</p></td><td class="cell"><p>.9785</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9657</p></td><td class="cell"><p>.000750</p></td><td class="cell"><p>.9715</p></td><td class="cell"><p>.000685</p></td><td class="cell"><p>.9686</p></td><td class="cell"><p>.8963</p></td><td class="cell"><p>.8411</p></td><td class="cell"><p>.8678</p></td><td class="cell"><p>.9719</p></td><td class="cell"><p>.9841</p></td><td class="cell"><p>.9780</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9181</p></td><td class="cell"><p>.001129</p></td><td class="cell"><p>.9024</p></td><td class="cell"><p>.001222</p></td><td class="cell"><p>.9102</p></td><td class="cell"><p>.6656</p></td><td class="cell"><p>.5843</p></td><td class="cell"><p>.6223</p></td><td class="cell"><p>.9407</p></td><td class="cell"><p>.9346</p></td><td class="cell"><p>.9377</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.8860</p></td><td class="cell"><p>.001309</p></td><td class="cell"><p>.9349</p></td><td class="cell"><p>.001016</p></td><td class="cell"><p>.9098</p></td><td class="cell"><p>.6595</p></td><td class="cell"><p>.5657</p></td><td class="cell"><p>.6090</p></td><td class="cell"><p>.9063</p></td><td class="cell"><p>.9764</p></td><td class="cell"><p>.9401</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p></p></td><td class="cell"><p>.0445</p></td><td class="cell"><p>.000862</p></td><td class="cell"><p>.0446</p></td><td class="cell"><p>.000863</p></td><td class="cell"><p>.0446</p></td><td class="cell"><p>.0226</p></td><td class="cell"><p>.0229</p></td><td class="cell"><p>.0227</p></td><td class="cell"><p>.0465</p></td><td class="cell"><p>.0466</p></td><td class="cell"><p>.0465</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 8: CKIP: Word Segmentation: Closed Track"></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9501</p></td><td class="cell"><p>.001445</p></td><td class="cell"><p>.9440</p></td><td class="cell"><p>.001527</p></td><td class="cell"><p>.9470</p></td><td class="cell"><p>.7404</p></td><td class="cell"><p>.7649</p></td><td class="cell"><p>.7524</p></td><td class="cell"><p>.9669</p></td><td class="cell"><p>.9577</p></td><td class="cell"><p>.9623</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>26</p></td><td class="cell"><p></p></td><td class="cell"><p>.9497</p></td><td class="cell"><p>.001451</p></td><td class="cell"><p>.9361</p></td><td class="cell"><p>.001624</p></td><td class="cell"><p>.9429</p></td><td class="cell"><p>.6556</p></td><td class="cell"><p>.7481</p></td><td class="cell"><p>.6988</p></td><td class="cell"><p>.9732</p></td><td class="cell"><p>.9490</p></td><td class="cell"><p>.9610</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9455</p></td><td class="cell"><p>.001507</p></td><td class="cell"><p>.9371</p></td><td class="cell"><p>.001612</p></td><td class="cell"><p>.9413</p></td><td class="cell"><p>.7004</p></td><td class="cell"><p>.7373</p></td><td class="cell"><p>.7184</p></td><td class="cell"><p>.9651</p></td><td class="cell"><p>.9521</p></td><td class="cell"><p>.9586</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9383</p></td><td class="cell"><p>.001597</p></td><td class="cell"><p>.9396</p></td><td class="cell"><p>.001582</p></td><td class="cell"><p>.9390</p></td><td class="cell"><p>.6962</p></td><td class="cell"><p>.6780</p></td><td class="cell"><p>.6870</p></td><td class="cell"><p>.9577</p></td><td class="cell"><p>.9612</p></td><td class="cell"><p>.9594</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.9432</p></td><td class="cell"><p>.001536</p></td><td class="cell"><p>.9333</p></td><td class="cell"><p>.001657</p></td><td class="cell"><p>.9383</p></td><td class="cell"><p>.6882</p></td><td class="cell"><p>.6885</p></td><td class="cell"><p>.6883</p></td><td class="cell"><p>.9637</p></td><td class="cell"><p>.9527</p></td><td class="cell"><p>.9581</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9412</p></td><td class="cell"><p>.001562</p></td><td class="cell"><p>.9345</p></td><td class="cell"><p>.001643</p></td><td class="cell"><p>.9378</p></td><td class="cell"><p>.7228</p></td><td class="cell"><p>.6688</p></td><td class="cell"><p>.6948</p></td><td class="cell"><p>.9586</p></td><td class="cell"><p>.9575</p></td><td class="cell"><p>.9580</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p></p></td><td class="cell"><p>.9369</p></td><td class="cell"><p>.001615</p></td><td class="cell"><p>.9270</p></td><td class="cell"><p>.001727</p></td><td class="cell"><p>.9319</p></td><td class="cell"><p>.6636</p></td><td class="cell"><p>.6624</p></td><td class="cell"><p>.6630</p></td><td class="cell"><p>.9587</p></td><td class="cell"><p>.9480</p></td><td class="cell"><p>.9533</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9345</p></td><td class="cell"><p>.001643</p></td><td class="cell"><p>.9289</p></td><td class="cell"><p>.001707</p></td><td class="cell"><p>.9317</p></td><td class="cell"><p>.7124</p></td><td class="cell"><p>.6602</p></td><td class="cell"><p>.6853</p></td><td class="cell"><p>.9522</p></td><td class="cell"><p>.9521</p></td><td class="cell"><p>.9522</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9336</p></td><td class="cell"><p>.001653</p></td><td class="cell"><p>.9277</p></td><td class="cell"><p>.001720</p></td><td class="cell"><p>.9306</p></td><td class="cell"><p>.7091</p></td><td class="cell"><p>.6589</p></td><td class="cell"><p>.6831</p></td><td class="cell"><p>.9515</p></td><td class="cell"><p>.9508</p></td><td class="cell"><p>.9512</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>27</p></td><td class="cell"><p></p></td><td class="cell"><p>.9354</p></td><td class="cell"><p>.001632</p></td><td class="cell"><p>.9173</p></td><td class="cell"><p>.001828</p></td><td class="cell"><p>.9263</p></td><td class="cell"><p>.5521</p></td><td class="cell"><p>.6877</p></td><td class="cell"><p>.6125</p></td><td class="cell"><p>.9661</p></td><td class="cell"><p>.9316</p></td><td class="cell"><p>.9485</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9247</p></td><td class="cell"><p>.001753</p></td><td class="cell"><p>.9162</p></td><td class="cell"><p>.001840</p></td><td class="cell"><p>.9204</p></td><td class="cell"><p>.6859</p></td><td class="cell"><p>.5896</p></td><td class="cell"><p>.6341</p></td><td class="cell"><p>.9438</p></td><td class="cell"><p>.9467</p></td><td class="cell"><p>.9452</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>33</p></td><td class="cell"><p></p></td><td class="cell"><p>.9241</p></td><td class="cell"><p>.001758</p></td><td class="cell"><p>.9165</p></td><td class="cell"><p>.001836</p></td><td class="cell"><p>.9203</p></td><td class="cell"><p>.6746</p></td><td class="cell"><p>.6195</p></td><td class="cell"><p>.6459</p></td><td class="cell"><p>.9441</p></td><td class="cell"><p>.9424</p></td><td class="cell"><p>.9432</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9233</p></td><td class="cell"><p>.001767</p></td><td class="cell"><p>.9161</p></td><td class="cell"><p>.001841</p></td><td class="cell"><p>.9197</p></td><td class="cell"><p>.6801</p></td><td class="cell"><p>.5846</p></td><td class="cell"><p>.6287</p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9471</p></td><td class="cell"><p>.9449</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9233</p></td><td class="cell"><p>.001767</p></td><td class="cell"><p>.9162</p></td><td class="cell"><p>.001840</p></td><td class="cell"><p>.9197</p></td><td class="cell"><p>.6801</p></td><td class="cell"><p>.5849</p></td><td class="cell"><p>.6289</p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9471</p></td><td class="cell"><p>.9450</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9224</p></td><td class="cell"><p>.001777</p></td><td class="cell"><p>.9153</p></td><td class="cell"><p>.001849</p></td><td class="cell"><p>.9188</p></td><td class="cell"><p>.6672</p></td><td class="cell"><p>.5732</p></td><td class="cell"><p>.6166</p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9473</p></td><td class="cell"><p>.9450</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>15</p></td><td class="cell"><p></p></td><td class="cell"><p>.9150</p></td><td class="cell"><p>.001852</p></td><td class="cell"><p>.9001</p></td><td class="cell"><p>.001991</p></td><td class="cell"><p>.9075</p></td><td class="cell"><p>.4751</p></td><td class="cell"><p>.5689</p></td><td class="cell"><p>.5178</p></td><td class="cell"><p>.9502</p></td><td class="cell"><p>.9216</p></td><td class="cell"><p>.9356</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9074</p></td><td class="cell"><p>.001925</p></td><td class="cell"><p>.8897</p></td><td class="cell"><p>.002080</p></td><td class="cell"><p>.8985</p></td><td class="cell"><p>.4405</p></td><td class="cell"><p>.5020</p></td><td class="cell"><p>.4692</p></td><td class="cell"><p>.9447</p></td><td class="cell"><p>.9161</p></td><td class="cell"><p>.9302</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9076</p></td><td class="cell"><p>.001923</p></td><td class="cell"><p>.8896</p></td><td class="cell"><p>.002081</p></td><td class="cell"><p>.8985</p></td><td class="cell"><p>.4406</p></td><td class="cell"><p>.5028</p></td><td class="cell"><p>.4697</p></td><td class="cell"><p>.9449</p></td><td class="cell"><p>.9159</p></td><td class="cell"><p>.9302</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8588</p></td><td class="cell"><p>.002312</p></td><td class="cell"><p>.8850</p></td><td class="cell"><p>.002118</p></td><td class="cell"><p>.8717</p></td><td class="cell"><p>.6204</p></td><td class="cell"><p>.4183</p></td><td class="cell"><p>.4997</p></td><td class="cell"><p>.8779</p></td><td class="cell"><p>.9447</p></td><td class="cell"><p>.9101</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 9: CKIP: Word Segmentation: Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9586</p></td><td class="cell"><p>.001323</p></td><td class="cell"><p>.9541</p></td><td class="cell"><p>.001389</p></td><td class="cell"><p>.9563</p></td><td class="cell"><p>.7804</p></td><td class="cell"><p>.8050</p></td><td class="cell"><p>.7925</p></td><td class="cell"><p>.9728</p></td><td class="cell"><p>.9656</p></td><td class="cell"><p>.9692</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9507</p></td><td class="cell"><p>.001438</p></td><td class="cell"><p>.9503</p></td><td class="cell"><p>.001443</p></td><td class="cell"><p>.9505</p></td><td class="cell"><p>.7391</p></td><td class="cell"><p>.7704</p></td><td class="cell"><p>.7544</p></td><td class="cell"><p>.9676</p></td><td class="cell"><p>.964</p></td><td class="cell"><p>.9658</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9367</p></td><td class="cell"><p>.001616</p></td><td class="cell"><p>.9360</p></td><td class="cell"><p>.001625</p></td><td class="cell"><p>.9364</p></td><td class="cell"><p>.7527</p></td><td class="cell"><p>.6911</p></td><td class="cell"><p>.7206</p></td><td class="cell"><p>.9515</p></td><td class="cell"><p>.9575</p></td><td class="cell"><p>.9545</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9324</p></td><td class="cell"><p>.001667</p></td><td class="cell"><p>.9326</p></td><td class="cell"><p>.001665</p></td><td class="cell"><p>.9325</p></td><td class="cell"><p>.7459</p></td><td class="cell"><p>.6631</p></td><td class="cell"><p>.7021</p></td><td class="cell"><p>.9473</p></td><td class="cell"><p>.9571</p></td><td class="cell"><p>.9522</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9218</p></td><td class="cell"><p>.001782</p></td><td class="cell"><p>.8960</p></td><td class="cell"><p>.002027</p></td><td class="cell"><p>.9087</p></td><td class="cell"><p>.6454</p></td><td class="cell"><p>.5901</p></td><td class="cell"><p>.6165</p></td><td class="cell"><p>.944</p></td><td class="cell"><p>.9221</p></td><td class="cell"><p>.9329</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p></p></td><td class="cell"><p>.3977</p></td><td class="cell"><p>.003245</p></td><td class="cell"><p>.3944</p></td><td class="cell"><p>.003240</p></td><td class="cell"><p>.3961</p></td><td class="cell"><p>.3405</p></td><td class="cell"><p>.3359</p></td><td class="cell"><p>.3382</p></td><td class="cell"><p>.4025</p></td><td class="cell"><p>.3994</p></td><td class="cell"><p>.4009</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 10: CTB: Word Segmentation: Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9583</p></td><td class="cell"><p>.001408</p></td><td class="cell"><p>.9596</p></td><td class="cell"><p>.001386</p></td><td class="cell"><p>.9589</p></td><td class="cell"><p>.7730</p></td><td class="cell"><p>.7761</p></td><td class="cell"><p>.7745</p></td><td class="cell"><p>.9691</p></td><td class="cell"><p>.9704</p></td><td class="cell"><p>.9697</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>26</p></td><td class="cell"><p></p></td><td class="cell"><p>.9538</p></td><td class="cell"><p>.001477</p></td><td class="cell"><p>.9527</p></td><td class="cell"><p>.001493</p></td><td class="cell"><p>.9533</p></td><td class="cell"><p>.7031</p></td><td class="cell"><p>.7491</p></td><td class="cell"><p>.7254</p></td><td class="cell"><p>.9685</p></td><td class="cell"><p>.9639</p></td><td class="cell"><p>.9662</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9505</p></td><td class="cell"><p>.001527</p></td><td class="cell"><p>.9528</p></td><td class="cell"><p>.001492</p></td><td class="cell"><p>.9517</p></td><td class="cell"><p>.7580</p></td><td class="cell"><p>.6886</p></td><td class="cell"><p>.7216</p></td><td class="cell"><p>.9618</p></td><td class="cell"><p>.9701</p></td><td class="cell"><p>.9659</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9503</p></td><td class="cell"><p>.001529</p></td><td class="cell"><p>.9520</p></td><td class="cell"><p>.001505</p></td><td class="cell"><p>.9512</p></td><td class="cell"><p>.7540</p></td><td class="cell"><p>.6845</p></td><td class="cell"><p>.7176</p></td><td class="cell"><p>.9619</p></td><td class="cell"><p>.9694</p></td><td class="cell"><p>.9656</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>27</p></td><td class="cell"><p></p></td><td class="cell"><p>.9494</p></td><td class="cell"><p>.001543</p></td><td class="cell"><p>.9508</p></td><td class="cell"><p>.001522</p></td><td class="cell"><p>.9501</p></td><td class="cell"><p>.7208</p></td><td class="cell"><p>.7012</p></td><td class="cell"><p>.7108</p></td><td class="cell"><p>.9628</p></td><td class="cell"><p>.9659</p></td><td class="cell"><p>.9644</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p></p></td><td class="cell"><p>.9487</p></td><td class="cell"><p>.001553</p></td><td class="cell"><p>.9514</p></td><td class="cell"><p>.001513</p></td><td class="cell"><p>.9500</p></td><td class="cell"><p>.7507</p></td><td class="cell"><p>.6753</p></td><td class="cell"><p>.7110</p></td><td class="cell"><p>.9603</p></td><td class="cell"><p>.9696</p></td><td class="cell"><p>.9650</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9482</p></td><td class="cell"><p>.001560</p></td><td class="cell"><p>.9516</p></td><td class="cell"><p>.001511</p></td><td class="cell"><p>.9499</p></td><td class="cell"><p>.7596</p></td><td class="cell"><p>.6740</p></td><td class="cell"><p>.7142</p></td><td class="cell"><p>.9592</p></td><td class="cell"><p>.9702</p></td><td class="cell"><p>.9647</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9481</p></td><td class="cell"><p>.001561</p></td><td class="cell"><p>.9514</p></td><td class="cell"><p>.001513</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.7614</p></td><td class="cell"><p>.6742</p></td><td class="cell"><p>.7152</p></td><td class="cell"><p>.9591</p></td><td class="cell"><p>.9700</p></td><td class="cell"><p>.9645</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9487</p></td><td class="cell"><p>.001552</p></td><td class="cell"><p>.9509</p></td><td class="cell"><p>.001520</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.7583</p></td><td class="cell"><p>.6812</p></td><td class="cell"><p>.7177</p></td><td class="cell"><p>.9599</p></td><td class="cell"><p>.9687</p></td><td class="cell"><p>.9643</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9471</p></td><td class="cell"><p>.001575</p></td><td class="cell"><p>.9500</p></td><td class="cell"><p>.001533</p></td><td class="cell"><p>.9486</p></td><td class="cell"><p>.7670</p></td><td class="cell"><p>.6736</p></td><td class="cell"><p>.7173</p></td><td class="cell"><p>.9577</p></td><td class="cell"><p>.9688</p></td><td class="cell"><p>.9632</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9451</p></td><td class="cell"><p>.001603</p></td><td class="cell"><p>.9521</p></td><td class="cell"><p>.001503</p></td><td class="cell"><p>.9486</p></td><td class="cell"><p>.7694</p></td><td class="cell"><p>.6714</p></td><td class="cell"><p>.7171</p></td><td class="cell"><p>.9555</p></td><td class="cell"><p>.9713</p></td><td class="cell"><p>.9633</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9495</p></td><td class="cell"><p>.001542</p></td><td class="cell"><p>.9474</p></td><td class="cell"><p>.001571</p></td><td class="cell"><p>.9485</p></td><td class="cell"><p>.6638</p></td><td class="cell"><p>.7456</p></td><td class="cell"><p>.7023</p></td><td class="cell"><p>.9663</p></td><td class="cell"><p>.9579</p></td><td class="cell"><p>.9621</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9429</p></td><td class="cell"><p>.001633</p></td><td class="cell"><p>.9535</p></td><td class="cell"><p>.001481</p></td><td class="cell"><p>.9482</p></td><td class="cell"><p>.7536</p></td><td class="cell"><p>.6661</p></td><td class="cell"><p>.7072</p></td><td class="cell"><p>.954</p></td><td class="cell"><p>.9730</p></td><td class="cell"><p>.9634</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9456</p></td><td class="cell"><p>.001596</p></td><td class="cell"><p>.9492</p></td><td class="cell"><p>.001545</p></td><td class="cell"><p>.9474</p></td><td class="cell"><p>.7565</p></td><td class="cell"><p>.6613</p></td><td class="cell"><p>.7057</p></td><td class="cell"><p>.9567</p></td><td class="cell"><p>.9688</p></td><td class="cell"><p>.9627</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9434</p></td><td class="cell"><p>.001626</p></td><td class="cell"><p>.9459</p></td><td class="cell"><p>.001592</p></td><td class="cell"><p>.9447</p></td><td class="cell"><p>.6911</p></td><td class="cell"><p>.6883</p></td><td class="cell"><p>.6897</p></td><td class="cell"><p>.9582</p></td><td class="cell"><p>.9612</p></td><td class="cell"><p>.9597</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>37</p></td><td class="cell"><p></p></td><td class="cell"><p>.9459</p></td><td class="cell"><p>.001592</p></td><td class="cell"><p>.9418</p></td><td class="cell"><p>.001648</p></td><td class="cell"><p>.9439</p></td><td class="cell"><p>.6589</p></td><td class="cell"><p>.6698</p></td><td class="cell"><p>.6643</p></td><td class="cell"><p>.9628</p></td><td class="cell"><p>.9574</p></td><td class="cell"><p>.9601</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>33</p></td><td class="cell"><p></p></td><td class="cell"><p>.9402</p></td><td class="cell"><p>.001669</p></td><td class="cell"><p>.9433</p></td><td class="cell"><p>.001628</p></td><td class="cell"><p>.9417</p></td><td class="cell"><p>.7317</p></td><td class="cell"><p>.6517</p></td><td class="cell"><p>.6894</p></td><td class="cell"><p>.9524</p></td><td class="cell"><p>.9628</p></td><td class="cell"><p>.9576</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9350</p></td><td class="cell"><p>.001736</p></td><td class="cell"><p>.9378</p></td><td class="cell"><p>.001700</p></td><td class="cell"><p>.9364</p></td><td class="cell"><p>.7132</p></td><td class="cell"><p>.5796</p></td><td class="cell"><p>.6395</p></td><td class="cell"><p>.9480</p></td><td class="cell"><p>.9641</p></td><td class="cell"><p>.9560</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9350</p></td><td class="cell"><p>.001735</p></td><td class="cell"><p>.9379</p></td><td class="cell"><p>.001699</p></td><td class="cell"><p>.9364</p></td><td class="cell"><p>.7132</p></td><td class="cell"><p>.5800</p></td><td class="cell"><p>.6397</p></td><td class="cell"><p>.9480</p></td><td class="cell"><p>.9642</p></td><td class="cell"><p>.9560</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9342</p></td><td class="cell"><p>.001745</p></td><td class="cell"><p>.9366</p></td><td class="cell"><p>.001715</p></td><td class="cell"><p>.9354</p></td><td class="cell"><p>.6998</p></td><td class="cell"><p>.5706</p></td><td class="cell"><p>.6286</p></td><td class="cell"><p>.9480</p></td><td class="cell"><p>.9634</p></td><td class="cell"><p>.9556</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9099</p></td><td class="cell"><p>.002015</p></td><td class="cell"><p>.9250</p></td><td class="cell"><p>.001854</p></td><td class="cell"><p>.9174</p></td><td class="cell"><p>.6911</p></td><td class="cell"><p>.4834</p></td><td class="cell"><p>.5689</p></td><td class="cell"><p>.9227</p></td><td class="cell"><p>.9638</p></td><td class="cell"><p>.9428</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9077</p></td><td class="cell"><p>.002037</p></td><td class="cell"><p>.9078</p></td><td class="cell"><p>.002037</p></td><td class="cell"><p>.9077</p></td><td class="cell"><p>.4728</p></td><td class="cell"><p>.5603</p></td><td class="cell"><p>.5128</p></td><td class="cell"><p>.9333</p></td><td class="cell"><p>.9248</p></td><td class="cell"><p>.9290</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9078</p></td><td class="cell"><p>.002037</p></td><td class="cell"><p>.9073</p></td><td class="cell"><p>.002041</p></td><td class="cell"><p>.9075</p></td><td class="cell"><p>.4703</p></td><td class="cell"><p>.5583</p></td><td class="cell"><p>.5105</p></td><td class="cell"><p>.9335</p></td><td class="cell"><p>.9244</p></td><td class="cell"><p>.9289</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.8992</p></td><td class="cell"><p>.002119</p></td><td class="cell"><p>.9063</p></td><td class="cell"><p>.002051</p></td><td class="cell"><p>.9027</p></td><td class="cell"><p>.5301</p></td><td class="cell"><p>.5029</p></td><td class="cell"><p>.5161</p></td><td class="cell"><p>.9209</p></td><td class="cell"><p>.9316</p></td><td class="cell"><p>.9262</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.8992</p></td><td class="cell"><p>.002119</p></td><td class="cell"><p>.9062</p></td><td class="cell"><p>.002052</p></td><td class="cell"><p>.9027</p></td><td class="cell"><p>.5299</p></td><td class="cell"><p>.5029</p></td><td class="cell"><p>.5160</p></td><td class="cell"><p>.9210</p></td><td class="cell"><p>.9315</p></td><td class="cell"><p>.9262</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.8773</p></td><td class="cell"><p>.002310</p></td><td class="cell"><p>.8788</p></td><td class="cell"><p>.002297</p></td><td class="cell"><p>.8780</p></td><td class="cell"><p>.6714</p></td><td class="cell"><p>.5886</p></td><td class="cell"><p>.6273</p></td><td class="cell"><p>.8894</p></td><td class="cell"><p>.8985</p></td><td class="cell"><p>.8939</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 11: CTB: Word Segmentation: Open Track"></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9914</p></td><td class="cell"><p>.000648</p></td><td class="cell"><p>.9926</p></td><td class="cell"><p>.000602</p></td><td class="cell"><p>.9920</p></td><td class="cell"><p>.9685</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.9654</p></td><td class="cell"><p>.9928</p></td><td class="cell"><p>.9944</p></td><td class="cell"><p>.9936</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9760</p></td><td class="cell"><p>.001077</p></td><td class="cell"><p>.9826</p></td><td class="cell"><p>.000920</p></td><td class="cell"><p>.9793</p></td><td class="cell"><p>.9420</p></td><td class="cell"><p>.8655</p></td><td class="cell"><p>.9021</p></td><td class="cell"><p>.9780</p></td><td class="cell"><p>.9902</p></td><td class="cell"><p>.9840</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9766</p></td><td class="cell"><p>.001065</p></td><td class="cell"><p>.9721</p></td><td class="cell"><p>.001158</p></td><td class="cell"><p>.9743</p></td><td class="cell"><p>.9089</p></td><td class="cell"><p>.8553</p></td><td class="cell"><p>.8813</p></td><td class="cell"><p>.9805</p></td><td class="cell"><p>.9794</p></td><td class="cell"><p>.9800</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9702</p></td><td class="cell"><p>.001196</p></td><td class="cell"><p>.9753</p></td><td class="cell"><p>.001092</p></td><td class="cell"><p>.9728</p></td><td class="cell"><p>.9145</p></td><td class="cell"><p>.8361</p></td><td class="cell"><p>.8736</p></td><td class="cell"><p>.9735</p></td><td class="cell"><p>.9844</p></td><td class="cell"><p>.9789</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9665</p></td><td class="cell"><p>.001266</p></td><td class="cell"><p>.9738</p></td><td class="cell"><p>.001123</p></td><td class="cell"><p>.9702</p></td><td class="cell"><p>.8821</p></td><td class="cell"><p>.8857</p></td><td class="cell"><p>.8839</p></td><td class="cell"><p>.9715</p></td><td class="cell"><p>.9790</p></td><td class="cell"><p>.9753</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9589</p></td><td class="cell"><p>.001397</p></td><td class="cell"><p>.9612</p></td><td class="cell"><p>.001359</p></td><td class="cell"><p>.9601</p></td><td class="cell"><p>.7922</p></td><td class="cell"><p>.7902</p></td><td class="cell"><p>.7912</p></td><td class="cell"><p>.9687</p></td><td class="cell"><p>.9713</p></td><td class="cell"><p>.9700</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p></p></td><td class="cell"><p>.9485</p></td><td class="cell"><p>.001556</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.001536</p></td><td class="cell"><p>.9491</p></td><td class="cell"><p>.7261</p></td><td class="cell"><p>.6769</p></td><td class="cell"><p>.7006</p></td><td class="cell"><p>.9615</p></td><td class="cell"><p>.9672</p></td><td class="cell"><p>.9643</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9461</p></td><td class="cell"><p>.001590</p></td><td class="cell"><p>.9372</p></td><td class="cell"><p>.001707</p></td><td class="cell"><p>.9416</p></td><td class="cell"><p>.7223</p></td><td class="cell"><p>.6764</p></td><td class="cell"><p>.6986</p></td><td class="cell"><p>.9592</p></td><td class="cell"><p>.9535</p></td><td class="cell"><p>.9563</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9370</p></td><td class="cell"><p>.001710</p></td><td class="cell"><p>.9321</p></td><td class="cell"><p>.001770</p></td><td class="cell"><p>.9346</p></td><td class="cell"><p>.6556</p></td><td class="cell"><p>.6139</p></td><td class="cell"><p>.6341</p></td><td class="cell"><p>.9535</p></td><td class="cell"><p>.9521</p></td><td class="cell"><p>.9528</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9270</p></td><td class="cell"><p>.001831</p></td><td class="cell"><p>.9319</p></td><td class="cell"><p>.001773</p></td><td class="cell"><p>.9294</p></td><td class="cell"><p>.6576</p></td><td class="cell"><p>.6099</p></td><td class="cell"><p>.6329</p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9525</p></td><td class="cell"><p>.9476</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.9251</p></td><td class="cell"><p>.001853</p></td><td class="cell"><p>.9261</p></td><td class="cell"><p>.001841</p></td><td class="cell"><p>.9256</p></td><td class="cell"><p>.5967</p></td><td class="cell"><p>.7337</p></td><td class="cell"><p>.6581</p></td><td class="cell"><p>.9444</p></td><td class="cell"><p>.9352</p></td><td class="cell"><p>.9398</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9089</p></td><td class="cell"><p>.002025</p></td><td class="cell"><p>.8346</p></td><td class="cell"><p>.002615</p></td><td class="cell"><p>.8702</p></td><td class="cell"><p>.2011</p></td><td class="cell"><p>.3336</p></td><td class="cell"><p>.2509</p></td><td class="cell"><p>.9505</p></td><td class="cell"><p>.8505</p></td><td class="cell"><p>.8977</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 12: NCC: Word Segmentation: Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9402</p></td><td class="cell"><p>.001214</p></td><td class="cell"><p>.9407</p></td><td class="cell"><p>.001210</p></td><td class="cell"><p>.9405</p></td><td class="cell"><p>.6179</p></td><td class="cell"><p>.5984</p></td><td class="cell"><p>.6080</p></td><td class="cell"><p>.9562</p></td><td class="cell"><p>.9583</p></td><td class="cell"><p>.9573</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>26</p></td><td class="cell"><p></p></td><td class="cell"><p>.9452</p></td><td class="cell"><p>.001166</p></td><td class="cell"><p>.9320</p></td><td class="cell"><p>.001289</p></td><td class="cell"><p>.9386</p></td><td class="cell"><p>.4502</p></td><td class="cell"><p>.6196</p></td><td class="cell"><p>.5215</p></td><td class="cell"><p>.9698</p></td><td class="cell"><p>.9430</p></td><td class="cell"><p>.9562</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9365</p></td><td class="cell"><p>.001249</p></td><td class="cell"><p>.9365</p></td><td class="cell"><p>.001249</p></td><td class="cell"><p>.9365</p></td><td class="cell"><p>.6158</p></td><td class="cell"><p>.5542</p></td><td class="cell"><p>.5834</p></td><td class="cell"><p>.9524</p></td><td class="cell"><p>.9577</p></td><td class="cell"><p>.9551</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>34</p></td><td class="cell"><p></p></td><td class="cell"><p>.9417</p></td><td class="cell"><p>.001200</p></td><td class="cell"><p>.9272</p></td><td class="cell"><p>.001331</p></td><td class="cell"><p>.9344</p></td><td class="cell"><p>.4001</p></td><td class="cell"><p>.6454</p></td><td class="cell"><p>.4940</p></td><td class="cell"><p>.9687</p></td><td class="cell"><p>.9356</p></td><td class="cell"><p>.9518</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9387</p></td><td class="cell"><p>.001229</p></td><td class="cell"><p>.9301</p></td><td class="cell"><p>.001306</p></td><td class="cell"><p>.9344</p></td><td class="cell"><p>.5561</p></td><td class="cell"><p>.5728</p></td><td class="cell"><p>.5643</p></td><td class="cell"><p>.9577</p></td><td class="cell"><p>.9472</p></td><td class="cell"><p>.9524</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9389</p></td><td class="cell"><p>.001226</p></td><td class="cell"><p>.9298</p></td><td class="cell"><p>.001309</p></td><td class="cell"><p>.9343</p></td><td class="cell"><p>.5556</p></td><td class="cell"><p>.5743</p></td><td class="cell"><p>.5648</p></td><td class="cell"><p>.9580</p></td><td class="cell"><p>.9467</p></td><td class="cell"><p>.9523</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>37</p></td><td class="cell"><p></p></td><td class="cell"><p>.9396</p></td><td class="cell"><p>.001220</p></td><td class="cell"><p>.9286</p></td><td class="cell"><p>.001319</p></td><td class="cell"><p>.9341</p></td><td class="cell"><p>.5007</p></td><td class="cell"><p>.5411</p></td><td class="cell"><p>.5201</p></td><td class="cell"><p>.9614</p></td><td class="cell"><p>.9462</p></td><td class="cell"><p>.9537</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.9328</p></td><td class="cell"><p>.001282</p></td><td class="cell"><p>.9353</p></td><td class="cell"><p>.001260</p></td><td class="cell"><p>.9340</p></td><td class="cell"><p>.5907</p></td><td class="cell"><p>.5218</p></td><td class="cell"><p>.5542</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.9588</p></td><td class="cell"><p>.9543</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9307</p></td><td class="cell"><p>.001301</p></td><td class="cell"><p>.9318</p></td><td class="cell"><p>.001292</p></td><td class="cell"><p>.9312</p></td><td class="cell"><p>.6309</p></td><td class="cell"><p>.5222</p></td><td class="cell"><p>.5715</p></td><td class="cell"><p>.9456</p></td><td class="cell"><p>.9566</p></td><td class="cell"><p>.9511</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9380</p></td><td class="cell"><p>.001235</p></td><td class="cell"><p>.9223</p></td><td class="cell"><p>.001371</p></td><td class="cell"><p>.9301</p></td><td class="cell"><p>.4709</p></td><td class="cell"><p>.6247</p></td><td class="cell"><p>.5370</p></td><td class="cell"><p>.9613</p></td><td class="cell"><p>.9331</p></td><td class="cell"><p>.947</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9251</p></td><td class="cell"><p>.001348</p></td><td class="cell"><p>.9347</p></td><td class="cell"><p>.001266</p></td><td class="cell"><p>.9299</p></td><td class="cell"><p>.6577</p></td><td class="cell"><p>.4968</p></td><td class="cell"><p>.5660</p></td><td class="cell"><p>.9384</p></td><td class="cell"><p>.9643</p></td><td class="cell"><p>.9512</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>27</p></td><td class="cell"><p></p></td><td class="cell"><p>.9300</p></td><td class="cell"><p>.001307</p></td><td class="cell"><p>.9291</p></td><td class="cell"><p>.001314</p></td><td class="cell"><p>.9296</p></td><td class="cell"><p>.5459</p></td><td class="cell"><p>.5138</p></td><td class="cell"><p>.5294</p></td><td class="cell"><p>.9491</p></td><td class="cell"><p>.9511</p></td><td class="cell"><p>.9501</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9246</p></td><td class="cell"><p>.001352</p></td><td class="cell"><p>.9332</p></td><td class="cell"><p>.001279</p></td><td class="cell"><p>.9289</p></td><td class="cell"><p>.6524</p></td><td class="cell"><p>.4932</p></td><td class="cell"><p>.5617</p></td><td class="cell"><p>.9381</p></td><td class="cell"><p>.9629</p></td><td class="cell"><p>.9503</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9193</p></td><td class="cell"><p>.001395</p></td><td class="cell"><p>.9378</p></td><td class="cell"><p>.001237</p></td><td class="cell"><p>.9285</p></td><td class="cell"><p>.6516</p></td><td class="cell"><p>.4833</p></td><td class="cell"><p>.5549</p></td><td class="cell"><p>.9326</p></td><td class="cell"><p>.9695</p></td><td class="cell"><p>.9507</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9278</p></td><td class="cell"><p>.001326</p></td><td class="cell"><p>.9250</p></td><td class="cell"><p>.001349</p></td><td class="cell"><p>.9264</p></td><td class="cell"><p>.5529</p></td><td class="cell"><p>.4966</p></td><td class="cell"><p>.5232</p></td><td class="cell"><p>.9464</p></td><td class="cell"><p>.9488</p></td><td class="cell"><p>.9476</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.9268</p></td><td class="cell"><p>.001334</p></td><td class="cell"><p>.9260</p></td><td class="cell"><p>.001341</p></td><td class="cell"><p>.9264</p></td><td class="cell"><p>.6094</p></td><td class="cell"><p>.4948</p></td><td class="cell"><p>.5462</p></td><td class="cell"><p>.9426</p></td><td class="cell"><p>.9527</p></td><td class="cell"><p>.9476</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9278</p></td><td class="cell"><p>.001326</p></td><td class="cell"><p>.9249</p></td><td class="cell"><p>.001350</p></td><td class="cell"><p>.9263</p></td><td class="cell"><p>.5486</p></td><td class="cell"><p>.4940</p></td><td class="cell"><p>.5199</p></td><td class="cell"><p>.9466</p></td><td class="cell"><p>.9488</p></td><td class="cell"><p>.9477</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9264</p></td><td class="cell"><p>.001338</p></td><td class="cell"><p>.9241</p></td><td class="cell"><p>.001356</p></td><td class="cell"><p>.9253</p></td><td class="cell"><p>.5707</p></td><td class="cell"><p>.4977</p></td><td class="cell"><p>.5317</p></td><td class="cell"><p>.9441</p></td><td class="cell"><p>.9486</p></td><td class="cell"><p>.9463</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9236</p></td><td class="cell"><p>.001361</p></td><td class="cell"><p>.9269</p></td><td class="cell"><p>.001333</p></td><td class="cell"><p>.9252</p></td><td class="cell"><p>.6474</p></td><td class="cell"><p>.4941</p></td><td class="cell"><p>.5604</p></td><td class="cell"><p>.9373</p></td><td class="cell"><p>.9556</p></td><td class="cell"><p>.9464</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9086</p></td><td class="cell"><p>.001476</p></td><td class="cell"><p>.9110</p></td><td class="cell"><p>.001459</p></td><td class="cell"><p>.9098</p></td><td class="cell"><p>.5957</p></td><td class="cell"><p>.4080</p></td><td class="cell"><p>.4843</p></td><td class="cell"><p>.9241</p></td><td class="cell"><p>.9485</p></td><td class="cell"><p>.9361</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9071</p></td><td class="cell"><p>.001487</p></td><td class="cell"><p>.9106</p></td><td class="cell"><p>.001461</p></td><td class="cell"><p>.9088</p></td><td class="cell"><p>.5907</p></td><td class="cell"><p>.3987</p></td><td class="cell"><p>.4761</p></td><td class="cell"><p>.9228</p></td><td class="cell"><p>.9494</p></td><td class="cell"><p>.9359</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.8997</p></td><td class="cell"><p>.001539</p></td><td class="cell"><p>.8992</p></td><td class="cell"><p>.001542</p></td><td class="cell"><p>.8995</p></td><td class="cell"><p>.4232</p></td><td class="cell"><p>.3710</p></td><td class="cell"><p>.3954</p></td><td class="cell"><p>.9234</p></td><td class="cell"><p>.9294</p></td><td class="cell"><p>.9264</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8995</p></td><td class="cell"><p>.001540</p></td><td class="cell"><p>.8992</p></td><td class="cell"><p>.001542</p></td><td class="cell"><p>.8994</p></td><td class="cell"><p>.4224</p></td><td class="cell"><p>.3702</p></td><td class="cell"><p>.3946</p></td><td class="cell"><p>.9233</p></td><td class="cell"><p>.9295</p></td><td class="cell"><p>.9264</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.7804</p></td><td class="cell"><p>.002121</p></td><td class="cell"><p>.8581</p></td><td class="cell"><p>.001788</p></td><td class="cell"><p>.8174</p></td><td class="cell"><p>.5409</p></td><td class="cell"><p>.2134</p></td><td class="cell"><p>.3060</p></td><td class="cell"><p>.7924</p></td><td class="cell"><p>.9561</p></td><td class="cell"><p>.8666</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.7747</p></td><td class="cell"><p>.002140</p></td><td class="cell"><p>.8513</p></td><td class="cell"><p>.001823</p></td><td class="cell"><p>.8112</p></td><td class="cell"><p>.5405</p></td><td class="cell"><p>.2014</p></td><td class="cell"><p>.2935</p></td><td class="cell"><p>.7864</p></td><td class="cell"><p>.9568</p></td><td class="cell"><p>.8633</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>33</p></td><td class="cell"><p></p></td><td class="cell"><p>.3082</p></td><td class="cell"><p>.002367</p></td><td class="cell"><p>.3073</p></td><td class="cell"><p>.002365</p></td><td class="cell"><p>.3078</p></td><td class="cell"><p>.2217</p></td><td class="cell"><p>.1678</p></td><td class="cell"><p>.1910</p></td><td class="cell"><p>.3125</p></td><td class="cell"><p>.3166</p></td><td class="cell"><p>.3145</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 13: NCC: Word Segmentation: Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>34</p></td><td class="cell"><p></p></td><td class="cell"><p>.9735</p></td><td class="cell"><p>.000823</p></td><td class="cell"><p>.9779</p></td><td class="cell"><p>.000753</p></td><td class="cell"><p>.9757</p></td><td class="cell"><p>.8893</p></td><td class="cell"><p>.8867</p></td><td class="cell"><p>.8880</p></td><td class="cell"><p>.9777</p></td><td class="cell"><p>.9824</p></td><td class="cell"><p>.9800</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.9568</p></td><td class="cell"><p>.001041</p></td><td class="cell"><p>.9616</p></td><td class="cell"><p>.000984</p></td><td class="cell"><p>.9592</p></td><td class="cell"><p>.8264</p></td><td class="cell"><p>.8144</p></td><td class="cell"><p>.8204</p></td><td class="cell"><p>.9633</p></td><td class="cell"><p>.9691</p></td><td class="cell"><p>.9662</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9620</p></td><td class="cell"><p>.000980</p></td><td class="cell"><p>.9496</p></td><td class="cell"><p>.001120</p></td><td class="cell"><p>.9557</p></td><td class="cell"><p>.6337</p></td><td class="cell"><p>.7673</p></td><td class="cell"><p>.6941</p></td><td class="cell"><p>.9783</p></td><td class="cell"><p>.9569</p></td><td class="cell"><p>.9675</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9528</p></td><td class="cell"><p>.001086</p></td><td class="cell"><p>.9478</p></td><td class="cell"><p>.001139</p></td><td class="cell"><p>.9503</p></td><td class="cell"><p>.7109</p></td><td class="cell"><p>.7619</p></td><td class="cell"><p>.7355</p></td><td class="cell"><p>.9648</p></td><td class="cell"><p>.9563</p></td><td class="cell"><p>.9606</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9440</p></td><td class="cell"><p>.001177</p></td><td class="cell"><p>.9517</p></td><td class="cell"><p>.001098</p></td><td class="cell"><p>.9478</p></td><td class="cell"><p>.7305</p></td><td class="cell"><p>.6381</p></td><td class="cell"><p>.6812</p></td><td class="cell"><p>.9547</p></td><td class="cell"><p>.9698</p></td><td class="cell"><p>.9622</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9376</p></td><td class="cell"><p>.001239</p></td><td class="cell"><p>.9521</p></td><td class="cell"><p>.001093</p></td><td class="cell"><p>.9448</p></td><td class="cell"><p>.7826</p></td><td class="cell"><p>.6110</p></td><td class="cell"><p>.6862</p></td><td class="cell"><p>.9453</p></td><td class="cell"><p>.9745</p></td><td class="cell"><p>.9597</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>14</p></td><td class="cell"><p></p></td><td class="cell"><p>.9446</p></td><td class="cell"><p>.001171</p></td><td class="cell"><p>.9263</p></td><td class="cell"><p>.001339</p></td><td class="cell"><p>.9354</p></td><td class="cell"><p>.4643</p></td><td class="cell"><p>.7160</p></td><td class="cell"><p>.5633</p></td><td class="cell"><p>.9685</p></td><td class="cell"><p>.9328</p></td><td class="cell"><p>.9503</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p></p></td><td class="cell"><p>.9324</p></td><td class="cell"><p>.001286</p></td><td class="cell"><p>.9349</p></td><td class="cell"><p>.001263</p></td><td class="cell"><p>.9337</p></td><td class="cell"><p>.6070</p></td><td class="cell"><p>.5296</p></td><td class="cell"><p>.5657</p></td><td class="cell"><p>.9486</p></td><td class="cell"><p>.9583</p></td><td class="cell"><p>.9534</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9191</p></td><td class="cell"><p>.001396</p></td><td class="cell"><p>.9380</p></td><td class="cell"><p>.001235</p></td><td class="cell"><p>.9285</p></td><td class="cell"><p>.6543</p></td><td class="cell"><p>.4840</p></td><td class="cell"><p>.5564</p></td><td class="cell"><p>.9323</p></td><td class="cell"><p>.9697</p></td><td class="cell"><p>.9506</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.9268</p></td><td class="cell"><p>.001334</p></td><td class="cell"><p>.9279</p></td><td class="cell"><p>.001325</p></td><td class="cell"><p>.9273</p></td><td class="cell"><p>.6265</p></td><td class="cell"><p>.5032</p></td><td class="cell"><p>.5581</p></td><td class="cell"><p>.9417</p></td><td class="cell"><p>.9546</p></td><td class="cell"><p>.9481</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9323</p></td><td class="cell"><p>.001287</p></td><td class="cell"><p>.9134</p></td><td class="cell"><p>.001440</p></td><td class="cell"><p>.9228</p></td><td class="cell"><p>.6075</p></td><td class="cell"><p>.5820</p></td><td class="cell"><p>.5945</p></td><td class="cell"><p>.9485</p></td><td class="cell"><p>.9303</p></td><td class="cell"><p>.9393</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 14: SXU: Word Segmentation: Closed Track"></table><p>word has only one POS-tag in the training cor­pus, but has other POS-tags in the testing cor­pus, it is a multi-tag word.</p><p>• OOV tag: If a tag of a word is found in the test corpus, but not in the training corpus, or the word itself is an OOV word, the corresponding word-tag pair is called OOV tag.</p><p>• IV tag: if the pair of word and tag does occur in the training corpus, the pair is called IV tag.</p><p>• IV multi-tag words: the multi-tag words that occurred in training data.</p><p>For each submission, we compute total accuracy <i>(Arotai</i>),IV recall (R/y), OOV recall (Roov), and IV Multi-tag word recall <i>(Rmtiv </i>) for evaluation. The formula for total accuracy is: ATotal = NN°rrfe°t, where Ncorrect denotes the number of words that are correctly tagged, and <i>Ntruth </i>denotes the number of words in the truth corpus.</p><p>The recall for IV, OOV and IV Multi-tag words are supposed to indicate participating system's per­formance on these three categories.</p><p>As Chinese word segmentation task, a baseline and a topline for each corpus are computed to reflect<page local="9" global="77"/></p><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9622</p></td><td class="cell"><p>.001132</p></td><td class="cell"><p>.9625</p></td><td class="cell"><p>.001127</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.7429</p></td><td class="cell"><p>.7159</p></td><td class="cell"><p>.7292</p></td><td class="cell"><p>.974</p></td><td class="cell"><p>.9764</p></td><td class="cell"><p>.9752</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>26</p></td><td class="cell"><p></p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.001131</p></td><td class="cell"><p>.9554</p></td><td class="cell"><p>.001225</p></td><td class="cell"><p>.9588</p></td><td class="cell"><p>.6454</p></td><td class="cell"><p>.7022</p></td><td class="cell"><p>.6726</p></td><td class="cell"><p>.9794</p></td><td class="cell"><p>.9678</p></td><td class="cell"><p>.9736</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9549</p></td><td class="cell"><p>.001231</p></td><td class="cell"><p>.9611</p></td><td class="cell"><p>.001148</p></td><td class="cell"><p>.9580</p></td><td class="cell"><p>.6626</p></td><td class="cell"><p>.6639</p></td><td class="cell"><p>.6632</p></td><td class="cell"><p>.9707</p></td><td class="cell"><p>.9772</p></td><td class="cell"><p>.9739</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9543</p></td><td class="cell"><p>.001239</p></td><td class="cell"><p>.9568</p></td><td class="cell"><p>.001206</p></td><td class="cell"><p>.9556</p></td><td class="cell"><p>.7273</p></td><td class="cell"><p>.6232</p></td><td class="cell"><p>.6712</p></td><td class="cell"><p>.9666</p></td><td class="cell"><p>.9781</p></td><td class="cell"><p>.9723</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p></p></td><td class="cell"><p>.9558</p></td><td class="cell"><p>.001219</p></td><td class="cell"><p>.9552</p></td><td class="cell"><p>.001228</p></td><td class="cell"><p>.9555</p></td><td class="cell"><p>.6922</p></td><td class="cell"><p>.6638</p></td><td class="cell"><p>.6777</p></td><td class="cell"><p>.9701</p></td><td class="cell"><p>.9716</p></td><td class="cell"><p>.9708</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9523</p></td><td class="cell"><p>.001264</p></td><td class="cell"><p>.9569</p></td><td class="cell"><p>.001205</p></td><td class="cell"><p>.9546</p></td><td class="cell"><p>.7506</p></td><td class="cell"><p>.6129</p></td><td class="cell"><p>.6748</p></td><td class="cell"><p>.9632</p></td><td class="cell"><p>.9801</p></td><td class="cell"><p>.9716</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9528</p></td><td class="cell"><p>.001258</p></td><td class="cell"><p>.9560</p></td><td class="cell"><p>.001217</p></td><td class="cell"><p>.9544</p></td><td class="cell"><p>.7369</p></td><td class="cell"><p>.6164</p></td><td class="cell"><p>.6713</p></td><td class="cell"><p>.9645</p></td><td class="cell"><p>.9782</p></td><td class="cell"><p>.9713</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9594</p></td><td class="cell"><p>.001171</p></td><td class="cell"><p>.9493</p></td><td class="cell"><p>.001302</p></td><td class="cell"><p>.9543</p></td><td class="cell"><p>.6653</p></td><td class="cell"><p>.6694</p></td><td class="cell"><p>.6674</p></td><td class="cell"><p>.9753</p></td><td class="cell"><p>.9642</p></td><td class="cell"><p>.9697</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9534</p></td><td class="cell"><p>.001250</p></td><td class="cell"><p>.9544</p></td><td class="cell"><p>.001238</p></td><td class="cell"><p>.9539</p></td><td class="cell"><p>.7395</p></td><td class="cell"><p>.6275</p></td><td class="cell"><p>.6789</p></td><td class="cell"><p>.9650</p></td><td class="cell"><p>.9754</p></td><td class="cell"><p>.9702</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9536</p></td><td class="cell"><p>.001248</p></td><td class="cell"><p>.9541</p></td><td class="cell"><p>.001242</p></td><td class="cell"><p>.9538</p></td><td class="cell"><p>.7352</p></td><td class="cell"><p>.6287</p></td><td class="cell"><p>.6778</p></td><td class="cell"><p>.9654</p></td><td class="cell"><p>.9748</p></td><td class="cell"><p>.9701</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9535</p></td><td class="cell"><p>.001249</p></td><td class="cell"><p>.9532</p></td><td class="cell"><p>.001253</p></td><td class="cell"><p>.9533</p></td><td class="cell"><p>.7305</p></td><td class="cell"><p>.6257</p></td><td class="cell"><p>.6741</p></td><td class="cell"><p>.9656</p></td><td class="cell"><p>.9740</p></td><td class="cell"><p>.9698</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9593</p></td><td class="cell"><p>.001173</p></td><td class="cell"><p>.9474</p></td><td class="cell"><p>.001324</p></td><td class="cell"><p>.9533</p></td><td class="cell"><p>.6463</p></td><td class="cell"><p>.6749</p></td><td class="cell"><p>.6603</p></td><td class="cell"><p>.9762</p></td><td class="cell"><p>.9613</p></td><td class="cell"><p>.9687</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9518</p></td><td class="cell"><p>.001270</p></td><td class="cell"><p>.9547</p></td><td class="cell"><p>.001234</p></td><td class="cell"><p>.9533</p></td><td class="cell"><p>.7020</p></td><td class="cell"><p>.6020</p></td><td class="cell"><p>.6481</p></td><td class="cell"><p>.9653</p></td><td class="cell"><p>.9772</p></td><td class="cell"><p>.9712</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9512</p></td><td class="cell"><p>.001278</p></td><td class="cell"><p>.9553</p></td><td class="cell"><p>.001226</p></td><td class="cell"><p>.9532</p></td><td class="cell"><p>.7462</p></td><td class="cell"><p>.6275</p></td><td class="cell"><p>.6817</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.9767</p></td><td class="cell"><p>.9694</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9509</p></td><td class="cell"><p>.001282</p></td><td class="cell"><p>.9544</p></td><td class="cell"><p>.001238</p></td><td class="cell"><p>.9526</p></td><td class="cell"><p>.7396</p></td><td class="cell"><p>.6281</p></td><td class="cell"><p>.6793</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.9754</p></td><td class="cell"><p>.9688</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9499</p></td><td class="cell"><p>.001295</p></td><td class="cell"><p>.9536</p></td><td class="cell"><p>.001249</p></td><td class="cell"><p>.9517</p></td><td class="cell"><p>.7271</p></td><td class="cell"><p>.5966</p></td><td class="cell"><p>.6554</p></td><td class="cell"><p>.9619</p></td><td class="cell"><p>.9774</p></td><td class="cell"><p>.9696</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>27</p></td><td class="cell"><p></p></td><td class="cell"><p>.9514</p></td><td class="cell"><p>.001276</p></td><td class="cell"><p>.9511</p></td><td class="cell"><p>.001279</p></td><td class="cell"><p>.9512</p></td><td class="cell"><p>.6834</p></td><td class="cell"><p>.6202</p></td><td class="cell"><p>.6502</p></td><td class="cell"><p>.9658</p></td><td class="cell"><p>.9709</p></td><td class="cell"><p>.9684</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9505</p></td><td class="cell"><p>.001287</p></td><td class="cell"><p>.9515</p></td><td class="cell"><p>.001275</p></td><td class="cell"><p>.9510</p></td><td class="cell"><p>.7326</p></td><td class="cell"><p>.6106</p></td><td class="cell"><p>.6660</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.9738</p></td><td class="cell"><p>.9680</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>37</p></td><td class="cell"><p></p></td><td class="cell"><p>.9554</p></td><td class="cell"><p>.001224</p></td><td class="cell"><p>.9459</p></td><td class="cell"><p>.001342</p></td><td class="cell"><p>.9507</p></td><td class="cell"><p>.6206</p></td><td class="cell"><p>.6113</p></td><td class="cell"><p>.6159</p></td><td class="cell"><p>.9735</p></td><td class="cell"><p>.9641</p></td><td class="cell"><p>.9688</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>34</p></td><td class="cell"><p></p></td><td class="cell"><p>.9558</p></td><td class="cell"><p>.001220</p></td><td class="cell"><p>.9442</p></td><td class="cell"><p>.001362</p></td><td class="cell"><p>.9500</p></td><td class="cell"><p>.5176</p></td><td class="cell"><p>.6966</p></td><td class="cell"><p>.5939</p></td><td class="cell"><p>.9794</p></td><td class="cell"><p>.9539</p></td><td class="cell"><p>.9665</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9558</p></td><td class="cell"><p>.001219</p></td><td class="cell"><p>.9441</p></td><td class="cell"><p>.001363</p></td><td class="cell"><p>.9499</p></td><td class="cell"><p>.5788</p></td><td class="cell"><p>.7154</p></td><td class="cell"><p>.6399</p></td><td class="cell"><p>.9762</p></td><td class="cell"><p>.9539</p></td><td class="cell"><p>.9649</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>33</p></td><td class="cell"><p></p></td><td class="cell"><p>.9387</p></td><td class="cell"><p>.001423</p></td><td class="cell"><p>.9392</p></td><td class="cell"><p>.001418</p></td><td class="cell"><p>.9390</p></td><td class="cell"><p>.6741</p></td><td class="cell"><p>.5627</p></td><td class="cell"><p>.6134</p></td><td class="cell"><p>.9530</p></td><td class="cell"><p>.9638</p></td><td class="cell"><p>.9584</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9378</p></td><td class="cell"><p>.001434</p></td><td class="cell"><p>.9390</p></td><td class="cell"><p>.001420</p></td><td class="cell"><p>.9384</p></td><td class="cell"><p>.6731</p></td><td class="cell"><p>.5110</p></td><td class="cell"><p>.5810</p></td><td class="cell"><p>.9520</p></td><td class="cell"><p>.9701</p></td><td class="cell"><p>.9610</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9376</p></td><td class="cell"><p>.001435</p></td><td class="cell"><p>.9391</p></td><td class="cell"><p>.001419</p></td><td class="cell"><p>.9383</p></td><td class="cell"><p>.6729</p></td><td class="cell"><p>.5107</p></td><td class="cell"><p>.5807</p></td><td class="cell"><p>.9519</p></td><td class="cell"><p>.9701</p></td><td class="cell"><p>.9609</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9377</p></td><td class="cell"><p>.001434</p></td><td class="cell"><p>.9389</p></td><td class="cell"><p>.001421</p></td><td class="cell"><p>.9383</p></td><td class="cell"><p>.6731</p></td><td class="cell"><p>.5110</p></td><td class="cell"><p>.5810</p></td><td class="cell"><p>.9520</p></td><td class="cell"><p>.9699</p></td><td class="cell"><p>.9609</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>d</p></td><td class="cell"><p>.9360</p></td><td class="cell"><p>.001452</p></td><td class="cell"><p>.9369</p></td><td class="cell"><p>.001443</p></td><td class="cell"><p>.9365</p></td><td class="cell"><p>.6550</p></td><td class="cell"><p>.4949</p></td><td class="cell"><p>.5638</p></td><td class="cell"><p>.9512</p></td><td class="cell"><p>.9691</p></td><td class="cell"><p>.9600</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9185</p></td><td class="cell"><p>.001624</p></td><td class="cell"><p>.9107</p></td><td class="cell"><p>.001692</p></td><td class="cell"><p>.9146</p></td><td class="cell"><p>.4898</p></td><td class="cell"><p>.4423</p></td><td class="cell"><p>.4648</p></td><td class="cell"><p>.9416</p></td><td class="cell"><p>.9386</p></td><td class="cell"><p>.9401</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9185</p></td><td class="cell"><p>.001624</p></td><td class="cell"><p>.9106</p></td><td class="cell"><p>.001693</p></td><td class="cell"><p>.9145</p></td><td class="cell"><p>.4886</p></td><td class="cell"><p>.4414</p></td><td class="cell"><p>.4638</p></td><td class="cell"><p>.9417</p></td><td class="cell"><p>.9386</p></td><td class="cell"><p>.9401</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.7820</p></td><td class="cell"><p>.002450</p></td><td class="cell"><p>.7793</p></td><td class="cell"><p>.002460</p></td><td class="cell"><p>.7807</p></td><td class="cell"><p>.4969</p></td><td class="cell"><p>.3538</p></td><td class="cell"><p>.4133</p></td><td class="cell"><p>.7976</p></td><td class="cell"><p>.8125</p></td><td class="cell"><p>.8050</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 15: SXU: Word Segmentation: Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>Cr</p></td><td class="cell"><p>P</p></td><td class="cell"><p>Cp</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Poov</p></td><td class="cell"><p>Foov</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Pjv</p></td><td class="cell"><p>Fjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9768</p></td><td class="cell"><p>.000894</p></td><td class="cell"><p>.9703</p></td><td class="cell"><p>.001007</p></td><td class="cell"><p>.9735</p></td><td class="cell"><p>.7825</p></td><td class="cell"><p>.8415</p></td><td class="cell"><p>.8109</p></td><td class="cell"><p>.9872</p></td><td class="cell"><p>.9767</p></td><td class="cell"><p>.9820</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9738</p></td><td class="cell"><p>.000948</p></td><td class="cell"><p>.9620</p></td><td class="cell"><p>.001134</p></td><td class="cell"><p>.9679</p></td><td class="cell"><p>.7089</p></td><td class="cell"><p>.8040</p></td><td class="cell"><p>.7534</p></td><td class="cell"><p>.9881</p></td><td class="cell"><p>.9694</p></td><td class="cell"><p>.9786</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9547</p></td><td class="cell"><p>.001233</p></td><td class="cell"><p>.9622</p></td><td class="cell"><p>.001132</p></td><td class="cell"><p>.9584</p></td><td class="cell"><p>.6705</p></td><td class="cell"><p>.6628</p></td><td class="cell"><p>.6666</p></td><td class="cell"><p>.9701</p></td><td class="cell"><p>.9787</p></td><td class="cell"><p>.9744</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9545</p></td><td class="cell"><p>.001236</p></td><td class="cell"><p>.9572</p></td><td class="cell"><p>.001201</p></td><td class="cell"><p>.9559</p></td><td class="cell"><p>.7543</p></td><td class="cell"><p>.6400</p></td><td class="cell"><p>.6925</p></td><td class="cell"><p>.9654</p></td><td class="cell"><p>.9776</p></td><td class="cell"><p>.9714</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9639</p></td><td class="cell"><p>.001108</p></td><td class="cell"><p>.9479</p></td><td class="cell"><p>.001319</p></td><td class="cell"><p>.9558</p></td><td class="cell"><p>.6103</p></td><td class="cell"><p>.7089</p></td><td class="cell"><p>.6559</p></td><td class="cell"><p>.9829</p></td><td class="cell"><p>.9587</p></td><td class="cell"><p>.9707</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>8</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.9586</p></td><td class="cell"><p>.001182</p></td><td class="cell"><p>.9467</p></td><td class="cell"><p>.001333</p></td><td class="cell"><p>.9526</p></td><td class="cell"><p>.6126</p></td><td class="cell"><p>.6967</p></td><td class="cell"><p>.6519</p></td><td class="cell"><p>.9773</p></td><td class="cell"><p>.9583</p></td><td class="cell"><p>.9677</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9575</p></td><td class="cell"><p>.001197</p></td><td class="cell"><p>.9461</p></td><td class="cell"><p>.001339</p></td><td class="cell"><p>.9518</p></td><td class="cell"><p>.7274</p></td><td class="cell"><p>.6920</p></td><td class="cell"><p>.7093</p></td><td class="cell"><p>.9699</p></td><td class="cell"><p>.9604</p></td><td class="cell"><p>.9652</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p></p></td><td class="cell"><p>.9516</p></td><td class="cell"><p>.001273</p></td><td class="cell"><p>.9515</p></td><td class="cell"><p>.001275</p></td><td class="cell"><p>.9516</p></td><td class="cell"><p>.6843</p></td><td class="cell"><p>.6174</p></td><td class="cell"><p>.6491</p></td><td class="cell"><p>.9661</p></td><td class="cell"><p>.9716</p></td><td class="cell"><p>.9688</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.8777</p></td><td class="cell"><p>.001945</p></td><td class="cell"><p>.8705</p></td><td class="cell"><p>.001993</p></td><td class="cell"><p>.8741</p></td><td class="cell"><p>.5621</p></td><td class="cell"><p>.6371</p></td><td class="cell"><p>.5972</p></td><td class="cell"><p>.8947</p></td><td class="cell"><p>.8815</p></td><td class="cell"><p>.8880</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 16: Named Entity Recognition Training and Truth data statistics"></table><p>"NE: Number of Named Entities. bPER: Number of Person names. cLOC: Number of Location names. dORG: Number of Organization names eRoov :OOVrate<page local="10" global="78"/></p><table caption="Table 17: Named Entity Recognition Truth data OOV statistics" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>Training</p></td><td class="cell"><p>ORGd</p></td><td class="cell"><p></p></td><td class="cell"><p>Truth</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>NE"</p></td><td class="cell"><p>PER* LOCc</p></td><td class="cell"><p>NE</p></td><td class="cell"><p>PER LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>66255</p></td><td class="cell"><p>16552 36213</p></td><td class="cell"><p>13490</p></td><td class="cell"><p>13014</p></td><td class="cell"><p>4940 4847</p></td><td class="cell"><p>3227</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSRA</p></td><td class="cell"><p>37811</p></td><td class="cell"><p>9028 18522</p></td><td class="cell"><p>10261</p></td><td class="cell"><p>7707</p></td><td class="cell"><p>1864 3658</p></td><td class="cell"><p>2185</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 18: Named Entity Recognition Baseline" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>NE</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>OOV</p></td><td class="cell"><p>Roove</p></td><td class="cell"><p>OOV</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>OOV Roov</p></td><td class="cell"><p>OOV Roov</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>6354</p></td><td class="cell"><p>0.4882</p></td><td class="cell"><p>3878</p></td><td class="cell"><p>0.7850</p></td><td class="cell"><p>900 0.1857</p></td><td class="cell"><p>1576 0.4884</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSRA</p></td><td class="cell"><p>1651</p></td><td class="cell"><p>0.2142</p></td><td class="cell"><p>564</p></td><td class="cell"><p>0.3026</p></td><td class="cell"><p>315 0.0861</p></td><td class="cell"><p>772 0.3533</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 19: CITYU: Named Entity Recognition: Closed Track" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p></p></td><td class="cell"><p>PpER</p></td><td class="cell"><p>FpER</p></td><td class="cell"><p>Rloc</p></td><td class="cell"><p>Ploc</p></td><td class="cell"><p>Floc</p></td><td class="cell"><p>Rorg</p></td><td class="cell"><p>Porg</p></td><td class="cell"><p>Forg</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>.4912</p></td><td class="cell"><p>.7562</p></td><td class="cell"><p>.5955</p></td><td class="cell"><p>.2130</p></td><td class="cell"><p>.7056</p></td><td class="cell"><p>.3272</p></td><td class="cell"><p>7681</p></td><td class="cell"><p>.8438</p></td><td class="cell"><p>.8042</p></td><td class="cell"><p>.5011</p></td><td class="cell"><p>.6341</p></td><td class="cell"><p>.5598</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MSRA</p></td><td class="cell"><p>.5451</p></td><td class="cell"><p>.6937</p></td><td class="cell"><p>.6105</p></td><td class="cell"><p>.6459</p></td><td class="cell"><p>.9205</p></td><td class="cell"><p>.7591</p></td><td class="cell"><p>.4513</p></td><td class="cell"><p>.7847</p></td><td class="cell"><p>.5731</p></td><td class="cell"><p>.6160</p></td><td class="cell"><p>.5091</p></td><td class="cell"><p>.5575</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>RpER</p></td><td class="cell"><p>PpER</p></td><td class="cell"><p></p></td><td class="cell"><p>Rloc</p></td><td class="cell"><p>Ploc</p></td><td class="cell"><p>Floc</p></td><td class="cell"><p>Rorg</p></td><td class="cell"><p>Porg</p></td><td class="cell"><p>Forg</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.8247</p></td><td class="cell"><p>.8768</p></td><td class="cell"><p>.8499</p></td><td class="cell"><p>.8615</p></td><td class="cell"><p>.9240</p></td><td class="cell"><p>.8917</p></td><td class="cell"><p>.9098</p></td><td class="cell"><p>.8612</p></td><td class="cell"><p>.8848</p></td><td class="cell"><p>.6402</p></td><td class="cell"><p>.8221</p></td><td class="cell"><p>.7199</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.7556</p></td><td class="cell"><p>.8850</p></td><td class="cell"><p>.8152</p></td><td class="cell"><p>.7688</p></td><td class="cell"><p>.9165</p></td><td class="cell"><p>.8362</p></td><td class="cell"><p>.8659</p></td><td class="cell"><p>.8695</p></td><td class="cell"><p>.8677</p></td><td class="cell"><p>.5699</p></td><td class="cell"><p>.8589</p></td><td class="cell"><p>.6852</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.7541</p></td><td class="cell"><p>.8846</p></td><td class="cell"><p>.8142</p></td><td class="cell"><p>.7638</p></td><td class="cell"><p>.9167</p></td><td class="cell"><p>.8333</p></td><td class="cell"><p>.8675</p></td><td class="cell"><p>.8684</p></td><td class="cell"><p>.8680</p></td><td class="cell"><p>.5689</p></td><td class="cell"><p>.8596</p></td><td class="cell"><p>.6847</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.7608</p></td><td class="cell"><p>.8751</p></td><td class="cell"><p>.8140</p></td><td class="cell"><p>.7771</p></td><td class="cell"><p>.9143</p></td><td class="cell"><p>.8401</p></td><td class="cell"><p>.8692</p></td><td class="cell"><p>.8551</p></td><td class="cell"><p>.8621</p></td><td class="cell"><p>.5730</p></td><td class="cell"><p>.8451</p></td><td class="cell"><p>.6829</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.7570</p></td><td class="cell"><p>.8585</p></td><td class="cell"><p>.8046</p></td><td class="cell"><p>.7682</p></td><td class="cell"><p>.8976</p></td><td class="cell"><p>.8279</p></td><td class="cell"><p>.8750</p></td><td class="cell"><p>.8314</p></td><td class="cell"><p>.8526</p></td><td class="cell"><p>.5624</p></td><td class="cell"><p>.8462</p></td><td class="cell"><p>.6757</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.7286</p></td><td class="cell"><p>.8933</p></td><td class="cell"><p>.8026</p></td><td class="cell"><p>.7306</p></td><td class="cell"><p>.9254</p></td><td class="cell"><p>.8165</p></td><td class="cell"><p>.8535</p></td><td class="cell"><p>.8789</p></td><td class="cell"><p>.8660</p></td><td class="cell"><p>.5380</p></td><td class="cell"><p>.8650</p></td><td class="cell"><p>.6634</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.7277</p></td><td class="cell"><p>.8926</p></td><td class="cell"><p>.8017</p></td><td class="cell"><p>.7287</p></td><td class="cell"><p>.9252</p></td><td class="cell"><p>.8153</p></td><td class="cell"><p>.8529</p></td><td class="cell"><p>.8781</p></td><td class="cell"><p>.8653</p></td><td class="cell"><p>.5380</p></td><td class="cell"><p>.8633</p></td><td class="cell"><p>.6628</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.0874</p></td><td class="cell"><p>.1058</p></td><td class="cell"><p>.0957</p></td><td class="cell"><p>.0656</p></td><td class="cell"><p>.0962</p></td><td class="cell"><p>.0780</p></td><td class="cell"><p>.1388</p></td><td class="cell"><p>.1200</p></td><td class="cell"><p>.1288</p></td><td class="cell"><p>.0437</p></td><td class="cell"><p>.0789</p></td><td class="cell"><p>.0562</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.0211</p></td><td class="cell"><p>.0326</p></td><td class="cell"><p>.0256</p></td><td class="cell"><p>.0128</p></td><td class="cell"><p>.0218</p></td><td class="cell"><p>.0161</p></td><td class="cell"><p>.0390</p></td><td class="cell"><p>.0433</p></td><td class="cell"><p>.0410</p></td><td class="cell"><p>.0068</p></td><td class="cell"><p>.0192</p></td><td class="cell"><p>.0101</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 20: CITYU: Named Entity Recognition: Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>RpER</p></td><td class="cell"><p>PpER</p></td><td class="cell"><p>Fper</p></td><td class="cell"><p>Rloc</p></td><td class="cell"><p>Ploc</p></td><td class="cell"><p>Floc</p></td><td class="cell"><p>Rorg</p></td><td class="cell"><p>Porg</p></td><td class="cell"><p>Forg</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>23</p></td><td class="cell"><p></p></td><td class="cell"><p>.8743</p></td><td class="cell"><p>.9342</p></td><td class="cell"><p>.9033</p></td><td class="cell"><p>.9526</p></td><td class="cell"><p>.9721</p></td><td class="cell"><p>.9623</p></td><td class="cell"><p>.9342</p></td><td class="cell"><p>.9235</p></td><td class="cell"><p>.9288</p></td><td class="cell"><p>.6644</p></td><td class="cell"><p>.8805</p></td><td class="cell"><p>.7573</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.8579</p></td><td class="cell"><p>.9179</p></td><td class="cell"><p>.8869</p></td><td class="cell"><p>.8822</p></td><td class="cell"><p>.9449</p></td><td class="cell"><p>.9125</p></td><td class="cell"><p>.9336</p></td><td class="cell"><p>.9099</p></td><td class="cell"><p>.9216</p></td><td class="cell"><p>.7072</p></td><td class="cell"><p>.8852</p></td><td class="cell"><p>.7862</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.8826</p></td><td class="cell"><p>.8826</p></td><td class="cell"><p>.8826</p></td><td class="cell"><p>.9168</p></td><td class="cell"><p>.8947</p></td><td class="cell"><p>.9056</p></td><td class="cell"><p>.9329</p></td><td class="cell"><p>.8942</p></td><td class="cell"><p>.9132</p></td><td class="cell"><p>.7546</p></td><td class="cell"><p>.8411</p></td><td class="cell"><p>.7955</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.8975</p></td><td class="cell"><p>.8616</p></td><td class="cell"><p>.8792</p></td><td class="cell"><p>.9474</p></td><td class="cell"><p>.9153</p></td><td class="cell"><p>.9311</p></td><td class="cell"><p>.9389</p></td><td class="cell"><p>.8966</p></td><td class="cell"><p>.9173</p></td><td class="cell"><p>.7589</p></td><td class="cell"><p>.7274</p></td><td class="cell"><p>.7428</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.7163</p></td><td class="cell"><p>.8000</p></td><td class="cell"><p>.7559</p></td><td class="cell"><p>.7180</p></td><td class="cell"><p>.8194</p></td><td class="cell"><p>.7653</p></td><td class="cell"><p>.8389</p></td><td class="cell"><p>.7845</p></td><td class="cell"><p>.8108</p></td><td class="cell"><p>.5296</p></td><td class="cell"><p>.7986</p></td><td class="cell"><p>.6369</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 21: MSRA: Named Entity Recognition: Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>RpER</p></td><td class="cell"><p>PpER</p></td><td class="cell"><p></p></td><td class="cell"><p>Rloc</p></td><td class="cell"><p>Ploc</p></td><td class="cell"><p>Floc</p></td><td class="cell"><p>Rorg</p></td><td class="cell"><p>Porg</p></td><td class="cell"><p>Forg</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.9186</p></td><td class="cell"><p>.9377</p></td><td class="cell"><p>.9281</p></td><td class="cell"><p>.9437</p></td><td class="cell"><p>.9665</p></td><td class="cell"><p>.9549</p></td><td class="cell"><p>.9423</p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9426</p></td><td class="cell"><p>.8577</p></td><td class="cell"><p>.9036</p></td><td class="cell"><p>.8800</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8862</p></td><td class="cell"><p>.9304</p></td><td class="cell"><p>.9078</p></td><td class="cell"><p>.9195</p></td><td class="cell"><p>.9651</p></td><td class="cell"><p>.9418</p></td><td class="cell"><p>.9043</p></td><td class="cell"><p>.9379</p></td><td class="cell"><p>.9208</p></td><td class="cell"><p>.8275</p></td><td class="cell"><p>.8871</p></td><td class="cell"><p>.8563</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.8779</p></td><td class="cell"><p>.9274</p></td><td class="cell"><p>.9020</p></td><td class="cell"><p>.9029</p></td><td class="cell"><p>.9628</p></td><td class="cell"><p>.9319</p></td><td class="cell"><p>.9101</p></td><td class="cell"><p>.9341</p></td><td class="cell"><p>.9219</p></td><td class="cell"><p>.8027</p></td><td class="cell"><p>.8841</p></td><td class="cell"><p>.8414</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.8752</p></td><td class="cell"><p>.9255</p></td><td class="cell"><p>.8996</p></td><td class="cell"><p>.9040</p></td><td class="cell"><p>.9618</p></td><td class="cell"><p>.9320</p></td><td class="cell"><p>.8991</p></td><td class="cell"><p>.9346</p></td><td class="cell"><p>.9165</p></td><td class="cell"><p>.8105</p></td><td class="cell"><p>.8780</p></td><td class="cell"><p>.8429</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.8822</p></td><td class="cell"><p>.9156</p></td><td class="cell"><p>.8986</p></td><td class="cell"><p>.9126</p></td><td class="cell"><p>.9461</p></td><td class="cell"><p>.9290</p></td><td class="cell"><p>.9079</p></td><td class="cell"><p>.9248</p></td><td class="cell"><p>.9163</p></td><td class="cell"><p>.8133</p></td><td class="cell"><p>.8724</p></td><td class="cell"><p>.8418</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p></p></td><td class="cell"><p>.8058</p></td><td class="cell"><p>.9107</p></td><td class="cell"><p>.8550</p></td><td class="cell"><p>.9029</p></td><td class="cell"><p>.9519</p></td><td class="cell"><p>.9268</p></td><td class="cell"><p>.8185</p></td><td class="cell"><p>.9278</p></td><td class="cell"><p>.8697</p></td><td class="cell"><p>.7016</p></td><td class="cell"><p>.8405</p></td><td class="cell"><p>.7648</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>37</p></td><td class="cell"><p></p></td><td class="cell"><p>.8331</p></td><td class="cell"><p>.8730</p></td><td class="cell"><p>.8526</p></td><td class="cell"><p>.8557</p></td><td class="cell"><p>.8084</p></td><td class="cell"><p>.8314</p></td><td class="cell"><p>.8576</p></td><td class="cell"><p>.9138</p></td><td class="cell"><p>.8848</p></td><td class="cell"><p>.7730</p></td><td class="cell"><p>.8666</p></td><td class="cell"><p>.8171</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 22: MSRA: Named Entity Recognition: Open Track"></table><p>the different degree of difficulty of tagging individ­ual corpora. The algorithm of baseline and topline is briefly described as follows: Baseline indicates the different degree of difficulty of tagging individual corpus.</p><p>The baseline of each corpus is calculated by gen­erating a list of words and POS tags from the train­ing corpus, then: 1. tagging those IV words in the testing corpus which have only one POS tag in the list. 2. for those IV words that have not only one tag in training corpus, the unique most frequent tag in training corpus will be assigned to them. 3. for each IV word that does not have a unique most fre­quent tag in training corpus, one of its tag which is most frequent in the overall phase is assigned to it; 4. for those words that do not fall into any of the former three categories are assigned with a overall most frequent tag.</p><p>The topline algorithm is similar to baseline, in­stead the list of words and POS tags is generated from testing corpus.</p><p>Chinese POS tagging results for all runs grouped by corpus and track appear in Tables 27-36; all ta­bles are sorted by ATotal.</p><p>The baseline and topline has shown that, with pre­liminary knowledge and mechanical algorithm, it is easy to achieve an accuracy over approximately 0.85. When excluding the effect caused by OOV tags, the accuracy can even be over 0.93.</p><p>There are two kind of problem in POS tagging task we should cope with: multi tag disambiguation and unknown words guessing. We could consider that the value of (topline - baseline) is the accuracy drop caused by unknown words guessing, and the value of (1 - topline) is the accuracy drop caused by multi tag disambiguation. The average of these two value is 0.0628 and 0.0600, therefore these two kind of problem can equally affect the performance of POS tagging system.</p><p>For this reason, unlike the topline of Chinese word segmentation, the topline of Chinese POS tag­ging could be easily exceeded by tagging systems, because the algorithm of this topline just excludes the effect of OOV tags, which is not a dominant de­terminant in this task.</p><p>In closed track, the highest total accuracy is achieved in the NCC corpus which has the low­est OOV tag rate, and the lowest total accuracy is achieved in the CITYU corpus which has the high­est OOV tag rate.</p><p>Most of the participants outperformed baseline, some have exceeded topline. When comparing the OOV recall and IV multi tag word recall with topline, participant's system can easily approaching or surpass the IV multi tag word recall, but none system could successfully approach the OOV recall. This might because participant's systems do better in solving the multi tag disambiguation problem than in coping with the unknown words guessing prob­lem.</p></subsection></section><section number="5" title="Conclusions &amp; Future Directions"><p>The Fourth SIGHAN Chinese Language Processing Bakeoff successfully brought together a collection of 28 strong research groups to assess the progress of research in three important tasks, Chinese word segmentation, named entity recognition and Chi­nese POS tagging, that in turn enable other impor­tant language processing technologies.<page local="11" global="79"/> The individ­ual group presentations at the SIGHAN workshop will detail the approaches that yielded strong perfor­mance for both tasks. Issues of out-of-vocabulary word handling, annotation consistency and unknown guessing all continue to challenge system designers and bakeoff organizers alike.</p><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>R</p></td><td class="cell"><p>P</p></td><td class="cell"><p>F</p></td><td class="cell"><p>RPER</p></td><td class="cell"><p>PPER</p></td><td class="cell"><p>FpER</p></td><td class="cell"><p>Rloc</p></td><td class="cell"><p>Ploc</p></td><td class="cell"><p>Floc</p></td><td class="cell"><p>Rorg</p></td><td class="cell"><p>Porg</p></td><td class="cell"><p>Forg</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.9995</p></td><td class="cell"><p>.9982</p></td><td class="cell"><p>.9988</p></td><td class="cell"><p>1</p></td><td class="cell"><p>.9989</p></td><td class="cell"><p>.9995</p></td><td class="cell"><p>.9997</p></td><td class="cell"><p>.9975</p></td><td class="cell"><p>.9986</p></td><td class="cell"><p>.9986</p></td><td class="cell"><p>.9986</p></td><td class="cell"><p>.9986</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p></p></td><td class="cell"><p>.9961</p></td><td class="cell"><p>.9956</p></td><td class="cell"><p>.9958</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>.9992</p></td><td class="cell"><p>.9929</p></td><td class="cell"><p>.9960</p></td><td class="cell"><p>.9876</p></td><td class="cell"><p>.9963</p></td><td class="cell"><p>.9920</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p></p></td><td class="cell"><p>.9377</p></td><td class="cell"><p>.9603</p></td><td class="cell"><p>.9489</p></td><td class="cell"><p>.9657</p></td><td class="cell"><p>.9574</p></td><td class="cell"><p>.9615</p></td><td class="cell"><p>.9593</p></td><td class="cell"><p>.9769</p></td><td class="cell"><p>.9680</p></td><td class="cell"><p>.8778</p></td><td class="cell"><p>.9338</p></td><td class="cell"><p>.9049</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>23</p></td><td class="cell"><p></p></td><td class="cell"><p>.9111</p></td><td class="cell"><p>.9471</p></td><td class="cell"><p>.9288</p></td><td class="cell"><p>.9458</p></td><td class="cell"><p>.9833</p></td><td class="cell"><p>.9642</p></td><td class="cell"><p>.9336</p></td><td class="cell"><p>.9397</p></td><td class="cell"><p>.9366</p></td><td class="cell"><p>.8439</p></td><td class="cell"><p>.9280</p></td><td class="cell"><p>.8840</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9135</p></td><td class="cell"><p>.9321</p></td><td class="cell"><p>.9227</p></td><td class="cell"><p>.9560</p></td><td class="cell"><p>.9601</p></td><td class="cell"><p>.9581</p></td><td class="cell"><p>.9221</p></td><td class="cell"><p>.9388</p></td><td class="cell"><p>.9304</p></td><td class="cell"><p>.8627</p></td><td class="cell"><p>.8959</p></td><td class="cell"><p>.8790</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>18</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9084</p></td><td class="cell"><p>.9278</p></td><td class="cell"><p>.9180</p></td><td class="cell"><p>.9544</p></td><td class="cell"><p>.9575</p></td><td class="cell"><p>.9559</p></td><td class="cell"><p>.9169</p></td><td class="cell"><p>.9322</p></td><td class="cell"><p>.9245</p></td><td class="cell"><p>.8549</p></td><td class="cell"><p>.8938</p></td><td class="cell"><p>.8739</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8675</p></td><td class="cell"><p>.9163</p></td><td class="cell"><p>.8912</p></td><td class="cell"><p>.9217</p></td><td class="cell"><p>.9630</p></td><td class="cell"><p>.9419</p></td><td class="cell"><p>.8445</p></td><td class="cell"><p>.9352</p></td><td class="cell"><p>.8875</p></td><td class="cell"><p>.8600</p></td><td class="cell"><p>.8502</p></td><td class="cell"><p>.8551</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.8791</p></td><td class="cell"><p>.9035</p></td><td class="cell"><p>.8911</p></td><td class="cell"><p>.9549</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.9524</p></td><td class="cell"><p>.9194</p></td><td class="cell"><p>.9129</p></td><td class="cell"><p>.9161</p></td><td class="cell"><p>.7469</p></td><td class="cell"><p>.8408</p></td><td class="cell"><p>.7911</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>11</p></td><td class="cell"><p></p></td><td class="cell"><p>.8674</p></td><td class="cell"><p>.9003</p></td><td class="cell"><p>.8836</p></td><td class="cell"><p>.9083</p></td><td class="cell"><p>.9216</p></td><td class="cell"><p>.9149</p></td><td class="cell"><p>.8989</p></td><td class="cell"><p>.9166</p></td><td class="cell"><p>.9077</p></td><td class="cell"><p>.7799</p></td><td class="cell"><p>.8516</p></td><td class="cell"><p>.8141</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p></p></td><td class="cell"><p>.8238</p></td><td class="cell"><p>.9038</p></td><td class="cell"><p>.8619</p></td><td class="cell"><p>.9206</p></td><td class="cell"><p>.9517</p></td><td class="cell"><p>.9359</p></td><td class="cell"><p>.8362</p></td><td class="cell"><p>.9424</p></td><td class="cell"><p>.8862</p></td><td class="cell"><p>.7204</p></td><td class="cell"><p>.7966</p></td><td class="cell"><p>.7565</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.8452</p></td><td class="cell"><p>.8720</p></td><td class="cell"><p>.8584</p></td><td class="cell"><p>.8734</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.9100</p></td><td class="cell"><p>.8710</p></td><td class="cell"><p>.8909</p></td><td class="cell"><p>.8808</p></td><td class="cell"><p>.7780</p></td><td class="cell"><p>.7798</p></td><td class="cell"><p>.7789</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.7890</p></td><td class="cell"><p>.8347</p></td><td class="cell"><p>.8112</p></td><td class="cell"><p>.8771</p></td><td class="cell"><p>.9196</p></td><td class="cell"><p>.8979</p></td><td class="cell"><p>.8365</p></td><td class="cell"><p>.8331</p></td><td class="cell"><p>.8348</p></td><td class="cell"><p>.6343</p></td><td class="cell"><p>.7557</p></td><td class="cell"><p>.6897</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>In future analysis, we hope to develop additional analysis tools to better assess progress in these fun­damental tasks, in a more corpus independent fash­ion. Such developments will guide the planning of future evaluations.</p><p>Finally, while Chinese word segmentation, named entity recognition and Chinese POS tagging are im­portant in themselves, these three enabling technolo­gies are also the foundation of those upper level ap­plications such as parsing, reference resolution or machine translation. To evaluate the impact of im­provement in these three technologies on the sub­sequent applications is still the future work for this evaluation.</p></section><section number="6" title="Acknowledgements"><p>We gratefully acknowledge the generous assistance of the organizations listed below who provided the data for this bakeoff; without their support, it could not have taken place:</p><p>• City University of Hong Kong, Hong Kong;</p><p>• Chinese Knowledge Information Processing Group, Academia Sinica, Taiwan;</p><p>• Institute of Applied linguistics, M.O.E., China;</p><p>• Microsoft Research Asia, China;</p><p>• Peking University, China;</p><p>• Shanxi University, China;</p><p>• University of Colorado, USA;</p><p>We also thank Olivia Oi Yee Kwong, the co-organizers of the sixth SIGHAN workshop, in con­junction with which this bakeoff takes place, and</p><p>Yongsheng Guo from Institute of Applied Linguis­tics, M.O.E., P.R.C. who has made great effort for this bakeoff.</p><p>Professor Changning Huang merits special thanks for his help in this bakeoff. Finally, we thank all the participating sites who enabled the success of this bakeoff.</p><table caption="Table 25: Chinese POS tagging Baseline"></table><p>aATotai: total accuracy *Rjv : IV recall cRoov : OOV recall dR m Tj V : MTjv recall "TT:<page local="12" global="80"/> number of tag type. bATN: Average Tag Number per word. cMTiv : number of IV Multi-Tag word jRmtiv : coverage rate of IV Multi-Tag words eRoov: OOV tag rate<page local="13" global="81"/></p><table caption="Table 27: CITYU:POS tagging Closed Track" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>ATotai"    Rjv *    Roov<i>c </i>Rmtjv d</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p><p>CKIP</p><p>CTB</p><p>NCC</p><p>PKU</p></td><td class="cell"><p>.8425      .9021      .2543 .8083</p><p>.8861    .9451    .2814 .8740 .8609    .8967   .3313 .8057 .9159    .9543    .2242 .8636 .8809    .9237   .2038 .8296</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 26: Chinese POS tagging Topline</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>ATotai     Rjv     Roov Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p><p>CKIP</p><p>CTB</p><p>NCC</p><p>PKU</p></td><td class="cell"><p>.9310     .9330     .9107 .8727 .9606     .9597     .9699 .9103 .9147     .9120     .9555 .8369 .9588     .9593     .9507 .8822 .9351     .9354     .9305 .8600</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8951</p></td><td class="cell"><p>.9389</p></td><td class="cell"><p>.4637</p></td><td class="cell"><p>.8745</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.8929</p></td><td class="cell"><p>.9367</p></td><td class="cell"><p>.4608</p></td><td class="cell"><p>.8705</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.8905</p></td><td class="cell"><p>.9328</p></td><td class="cell"><p>.4733</p></td><td class="cell"><p>.8687</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.8865</p></td><td class="cell"><p>.9326</p></td><td class="cell"><p>.4322</p></td><td class="cell"><p>.8707</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.8693</p></td><td class="cell"><p>.9284</p></td><td class="cell"><p>.2868</p></td><td class="cell"><p>.8585</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.8564</p></td><td class="cell"><p>.9149</p></td><td class="cell"><p>.2805</p></td><td class="cell"><p>.8506</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.2793</p></td><td class="cell"><p>.2969</p></td><td class="cell"><p>.1051</p></td><td class="cell"><p>.2538</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>21</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.1890</p></td><td class="cell"><p>.2031</p></td><td class="cell"><p>.0550</p></td><td class="cell"><p>.1704</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 28: CITYU:POS tagging Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.8900</p></td><td class="cell"><p>.9329</p></td><td class="cell"><p>.4670</p></td><td class="cell"><p>.8695</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.8669</p></td><td class="cell"><p>.9089</p></td><td class="cell"><p>.4537</p></td><td class="cell"><p>.8495</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 29: CKIP:POS tagging Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9295</p></td><td class="cell"><p>.9629</p></td><td class="cell"><p>.5869</p></td><td class="cell"><p>.9123</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9286</p></td><td class="cell"><p>.9618</p></td><td class="cell"><p>.5875</p></td><td class="cell"><p>.9099</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9220</p></td><td class="cell"><p>.9556</p></td><td class="cell"><p>.5772</p></td><td class="cell"><p>.9088</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9160</p></td><td class="cell"><p>.9504</p></td><td class="cell"><p>.5631</p></td><td class="cell"><p>.9065</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>16</p></td><td class="cell"><p></p></td><td class="cell"><p>.9124</p></td><td class="cell"><p>.9549</p></td><td class="cell"><p>.4756</p></td><td class="cell"><p>.8953</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.8994</p></td><td class="cell"><p>.9561</p></td><td class="cell"><p>.3169</p></td><td class="cell"><p>.9001</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.8793</p></td><td class="cell"><p>.9334</p></td><td class="cell"><p>.3247</p></td><td class="cell"><p>.8943</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 23: Chinese POS tagging Training data statistics"></table><table caption="Table 30: CKIP:POS tagging Open Track"></table><table caption="Table 24: Chinese POS tagging Truth data statistics" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>Token</p></td><td class="cell"><p>WT</p></td><td class="cell"><p>TT"</p></td><td class="cell"><p>ATN*</p></td><td class="cell"><p>MTjv c</p></td><td class="cell"><p>Rmtjv d</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p>1092687</p></td><td class="cell"><p>43639</p></td><td class="cell"><p>44</p></td><td class="cell"><p>1.2588</p></td><td class="cell"><p>585056</p></td><td class="cell"><p>0.5354</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p>721551</p></td><td class="cell"><p>48045</p></td><td class="cell"><p>60</p></td><td class="cell"><p>1.0851</p></td><td class="cell"><p>335017</p></td><td class="cell"><p>0.4643</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p>642246</p></td><td class="cell"><p>42133</p></td><td class="cell"><p>37</p></td><td class="cell"><p>1.1690</p></td><td class="cell"><p>334317</p></td><td class="cell"><p>0.5205</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p>535023</p></td><td class="cell"><p>45108</p></td><td class="cell"><p>60</p></td><td class="cell"><p>1.0673</p></td><td class="cell"><p>178078</p></td><td class="cell"><p>0.3328</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU</p></td><td class="cell"><p>1116754</p></td><td class="cell"><p>55178</p></td><td class="cell"><p>103</p></td><td class="cell"><p>1.1194</p></td><td class="cell"><p>490243</p></td><td class="cell"><p>0.4390</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>Token</p></td><td class="cell"><p>WT</p></td><td class="cell"><p>TT</p></td><td class="cell"><p>ATN</p></td><td class="cell"><p>OOV</p></td><td class="cell"><p>Roove</p></td><td class="cell"><p>MTjv</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CITYU</p></td><td class="cell"><p></p></td><td class="cell"><p>184314</p></td><td class="cell"><p>17827</p></td><td class="cell"><p>43</p></td><td class="cell"><p>1.1446</p></td><td class="cell"><p>16977</p></td><td class="cell"><p>0.0921</p></td><td class="cell"><p>92934</p></td><td class="cell"><p>0.5042</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CKIP</p></td><td class="cell"><p></p></td><td class="cell"><p>91071</p></td><td class="cell"><p>15331</p></td><td class="cell"><p>63</p></td><td class="cell"><p>1.0530</p></td><td class="cell"><p>8085</p></td><td class="cell"><p>0.0888</p></td><td class="cell"><p>38640</p></td><td class="cell"><p>0.4243</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CTB</p></td><td class="cell"><p></p></td><td class="cell"><p>59955</p></td><td class="cell"><p>9797</p></td><td class="cell"><p>35</p></td><td class="cell"><p>1.1227</p></td><td class="cell"><p>3794</p></td><td class="cell"><p>0.0633</p></td><td class="cell"><p>30513</p></td><td class="cell"><p>0.5089</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NCC</p></td><td class="cell"><p></p></td><td class="cell"><p>102344</p></td><td class="cell"><p>17493</p></td><td class="cell"><p>55</p></td><td class="cell"><p>1.0675</p></td><td class="cell"><p>5392</p></td><td class="cell"><p>0.0527</p></td><td class="cell"><p>33853</p></td><td class="cell"><p>0.3308</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PKU</p></td><td class="cell"><p></p></td><td class="cell"><p>156407</p></td><td class="cell"><p>17643</p></td><td class="cell"><p>103</p></td><td class="cell"><p>1.1270</p></td><td class="cell"><p>9295</p></td><td class="cell"><p>0.0594</p></td><td class="cell"><p>68065</p></td><td class="cell"><p>0.4352</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 34: NCC:POS tagging Open Track" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9211</p></td><td class="cell"><p>.9542</p></td><td class="cell"><p>.5813</p></td><td class="cell"><p>.9082</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9004</p></td><td class="cell"><p>.9327</p></td><td class="cell"><p>.5686</p></td><td class="cell"><p>.8936</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 31: CTB:POS tagging Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9428</p></td><td class="cell"><p>.9557</p></td><td class="cell"><p>.7522</p></td><td class="cell"><p>.9197</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9401</p></td><td class="cell"><p>.9554</p></td><td class="cell"><p>.7135</p></td><td class="cell"><p>.9183</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>16</p></td><td class="cell"><p></p></td><td class="cell"><p>.9234</p></td><td class="cell"><p>.9507</p></td><td class="cell"><p>.5200</p></td><td class="cell"><p>.9051</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.9203</p></td><td class="cell"><p>.9460</p></td><td class="cell"><p>.5390</p></td><td class="cell"><p>.9055</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.9133</p></td><td class="cell"><p>.9438</p></td><td class="cell"><p>.4620</p></td><td class="cell"><p>.8983</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9088</p></td><td class="cell"><p>.9374</p></td><td class="cell"><p>.4866</p></td><td class="cell"><p>.8805</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8065</p></td><td class="cell"><p>.8608</p></td><td class="cell"><p>.0040</p></td><td class="cell"><p>.7395</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 32: CTB:POS tagging Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.9689</p></td><td class="cell"><p>.9767</p></td><td class="cell"><p>.8537</p></td><td class="cell"><p>.9554</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9646</p></td><td class="cell"><p>.9714</p></td><td class="cell"><p>.8648</p></td><td class="cell"><p>.9495</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9271</p></td><td class="cell"><p>.9400</p></td><td class="cell"><p>.7354</p></td><td class="cell"><p>.9016</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9120</p></td><td class="cell"><p>.9374</p></td><td class="cell"><p>.5361</p></td><td class="cell"><p>.8805</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8076</p></td><td class="cell"><p>.8608</p></td><td class="cell"><p>.0206</p></td><td class="cell"><p>.7396</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 33: NCC:POS tagging Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9541</p></td><td class="cell"><p>.9738</p></td><td class="cell"><p>.5998</p></td><td class="cell"><p>.9195</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9525</p></td><td class="cell"><p>.9717</p></td><td class="cell"><p>.6059</p></td><td class="cell"><p>.9135</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9494</p></td><td class="cell"><p>.9690</p></td><td class="cell"><p>.5959</p></td><td class="cell"><p>.9129</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9456</p></td><td class="cell"><p>.9658</p></td><td class="cell"><p>.5822</p></td><td class="cell"><p>.9116</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>16</p></td><td class="cell"><p></p></td><td class="cell"><p>.9395</p></td><td class="cell"><p>.9690</p></td><td class="cell"><p>.4086</p></td><td class="cell"><p>.9059</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.9336</p></td><td class="cell"><p>.9687</p></td><td class="cell"><p>.3017</p></td><td class="cell"><p>.9050</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9313</p></td><td class="cell"><p>.9604</p></td><td class="cell"><p>.4080</p></td><td class="cell"><p>.8809</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.9277</p></td><td class="cell"><p>.9664</p></td><td class="cell"><p>.2329</p></td><td class="cell"><p>.9000</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>24</p></td><td class="cell"><p></p></td><td class="cell"><p>.9172</p></td><td class="cell"><p>.9498</p></td><td class="cell"><p>.3312</p></td><td class="cell"><p>.8963</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8940</p></td><td class="cell"><p>.9303</p></td><td class="cell"><p>.2411</p></td><td class="cell"><p>.7948</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9496</p></td><td class="cell"><p>.9694</p></td><td class="cell"><p>.5938</p></td><td class="cell"><p>.9141</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9326</p></td><td class="cell"><p>.9604</p></td><td class="cell"><p>.4336</p></td><td class="cell"><p>.8809</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9280</p></td><td class="cell"><p>.9477</p></td><td class="cell"><p>.5749</p></td><td class="cell"><p>.8954</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.9096</p></td><td class="cell"><p>.9377</p></td><td class="cell"><p>.4045</p></td><td class="cell"><p>.8935</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8940</p></td><td class="cell"><p>.9303</p></td><td class="cell"><p>.2411</p></td><td class="cell"><p>.7948</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>25</p></td><td class="cell"><p></p></td><td class="cell"><p>.0836</p></td><td class="cell"><p>.0855</p></td><td class="cell"><p>.0488</p></td><td class="cell"><p>.0645</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 35: PKU:POS tagging Closed Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.9450</p></td><td class="cell"><p>.9679</p></td><td class="cell"><p>.5818</p></td><td class="cell"><p>.9252</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>30</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9420</p></td><td class="cell"><p>.9648</p></td><td class="cell"><p>.5813</p></td><td class="cell"><p>.9184</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9396</p></td><td class="cell"><p>.9608</p></td><td class="cell"><p>.6036</p></td><td class="cell"><p>.9173</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>9</p></td><td class="cell"><p></p></td><td class="cell"><p>.9368</p></td><td class="cell"><p>.9591</p></td><td class="cell"><p>.5832</p></td><td class="cell"><p>.9173</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>16</p></td><td class="cell"><p></p></td><td class="cell"><p>.9266</p></td><td class="cell"><p>.9574</p></td><td class="cell"><p>.4386</p></td><td class="cell"><p>.9079</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.9113</p></td><td class="cell"><p>.9518</p></td><td class="cell"><p>.2708</p></td><td class="cell"><p>.8958</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>37</p></td><td class="cell"><p></p></td><td class="cell"><p>.9065</p></td><td class="cell"><p>.9269</p></td><td class="cell"><p>.5836</p></td><td class="cell"><p>.8903</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9053</p></td><td class="cell"><p>.9451</p></td><td class="cell"><p>.2751</p></td><td class="cell"><p>.8758</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>19</p></td><td class="cell"><p></p></td><td class="cell"><p>.8815</p></td><td class="cell"><p>.9158</p></td><td class="cell"><p>.3386</p></td><td class="cell"><p>.8897</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8527</p></td><td class="cell"><p>.8936</p></td><td class="cell"><p>.2043</p></td><td class="cell"><p>.7646</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>c</p></td><td class="cell"><p>.8450</p></td><td class="cell"><p>.8855</p></td><td class="cell"><p>.2039</p></td><td class="cell"><p>.7471</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 36: PKU:POS tagging Open Track</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ID</p></td><td class="cell"><p>RunID</p></td><td class="cell"><p>ATotai</p></td><td class="cell"><p>Rjv</p></td><td class="cell"><p>Roov</p></td><td class="cell"><p>Rmtjv</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>28</p></td><td class="cell"><p></p></td><td class="cell"><p>.9411</p></td><td class="cell"><p>.9622</p></td><td class="cell"><p>.6057</p></td><td class="cell"><p>.9200</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>a</p></td><td class="cell"><p>.9329</p></td><td class="cell"><p>.9518</p></td><td class="cell"><p>.6332</p></td><td class="cell"><p>.8972</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>29</p></td><td class="cell"><p></p></td><td class="cell"><p>.9197</p></td><td class="cell"><p>.9512</p></td><td class="cell"><p>.4222</p></td><td class="cell"><p>.8990</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>39</p></td><td class="cell"><p></p></td><td class="cell"><p>.9134</p></td><td class="cell"><p>.9341</p></td><td class="cell"><p>.5862</p></td><td class="cell"><p>.8894</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>31</p></td><td class="cell"><p>b</p></td><td class="cell"><p>.8427</p></td><td class="cell"><p>.8935</p></td><td class="cell"><p>.0398</p></td><td class="cell"><p>.7643</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>22</p></td><td class="cell"><p></p></td><td class="cell"><p>.6649</p></td><td class="cell"><p>.6796</p></td><td class="cell"><p>.4308</p></td><td class="cell"><p>.6495</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Thomas Emerson. 2005. The second international Chi­nese word segmentation bakeoff. In <i>Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, </i>pages 123-133, Jeju Island, Korea.</p><p>Charles Grinstead and J. Laurie Snell. 1997. <i>Introduc­tion to Pro*a*ility. </i>American Mathematical Society, Providence, RI.</p><p>Gina-Anne Levow. 2006. The third international chinese language processing bakeoff: Word segmentation and named entity recognition. In <i>Proceedings ofthe Fifth SIGHAN Workshop on Chinese Language Processing, </i>pages 108-117, Sydney, Australia, July. Association for Computational Linguistics.</p><p>Richard Sproat and Thomas Emerson. 2003. The first international Chinese word segmentation bakeoff. In <i>The Second SIGHAN Workshop on Chinese Language Processing, </i>pages 133-143, Sapporo, Japan.</p></references></body></article>