<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="304"/><title>PKU_HIT: An Event Detection System Based on Instances Expansion and Rich Syntactic Features</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 304-307, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Li" givenname="Shiqi"><org  name="Hong Kong Polytechnic University" country="Hong Kong" city="Hung Hom"/></author><author surname="Liu" givenname="Pengyuan"><org  name="Harbin Institute of Technology" country="China" city="Harbin"/></author><author surname="Zhao" givenname="Tiejun"><org  name="Harbin Institute of Technology" country="China" city="Harbin"/></author><author surname="Lu" givenname="Qin"><org  name="Peking University" country="China" city="Beijing"/></author><author surname="Li" givenname="Hanjing"><org  name="Hong Kong Polytechnic University" country="Hong Kong" city="Hung Hom"/></author></firstpageheader><frontmatter><p><b>PKUHIT:</b><b> </b><b>An</b><b> Event Detection System Based on Instances Expansion</b></p><p><b>and Rich Syntactic Features</b></p><p><b>Shiqi Li<footnote anchor="1"/>, Pengyuan Liu<footnote anchor="2"/>, Tiejun Zhao<footnote anchor="1"/>, Qin Lu<footnote anchor="3"/> and Hanjing Li<footnote anchor="1"/></b></p><p><footnote anchor="1"/>School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China {sqli,tjzhao,hjlee}@mtlab.hit.edu.cn</p><p><footnote anchor="2"/>Institute of Computational Linguistics, Peking University, Beijing 100871, China</p><p>liupengyuan@pku.edu.cn</p><p><footnote anchor="3"/>Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China</p><p>csluqin@comp.polyu.edu.hk</p></frontmatter><abstract>This paper describes the PKUHIT system on event detection in the SemEval-2010 Task. We construct three modules for the three sub-tasks of this evaluation. For target verb WSD, we build a Naïve Bayesian classifier which uses additional training instances expanded from an untagged Chinese corpus automatically. For sentence SRL and event detection, we use a feature-based machine learning method which makes combined use of both constituent-based and dependency-based features. Experimental results show that the Macro Accuracy of the WSD module reaches 83.81% and F-Score of the SRL module is 55.71%. </abstract></header><body><section number="1" title="Introduction"><p>In this paper, we describe the system submitted to the SemEval-2010 Task 11 on event detection in Chinese news sentences (Zhou, 2010). The objective of the task is to detect and analyze basic event contents in Chinese news sentences, similar to the frame semantic structure extraction task in SemEval-2007. However, this task is a more complex as it involves three interrelated subtasks: (1) target verb word sense disambiguation (WSD), (2) sentence semantic role labeling (SRL) and (3) event detection (ED).</p><p>Therefore, the architecture of the system that we develop for the task consists of three modules:</p><p>WSD, SRL and ED. First, the WSD module is to recognize key verbs or verb phrases which describe the basic event in a sentence, and then select an appropriate situation description formula for the recognized key verbs (or verb phrases); Then, the SRL module anchors the arguments to suitable constituents in the sentence, and then label each argument with three functional tags, namely constituent type tag, semantic role tags and event role tag. Finally, in the ED module, complete situation description of the sentence can be achieved by combining the results of the WSD module and the SRL module.</p><p>For the WSD module, we consider the subtask as a general WSD problem. First of all, we automatically extract many instances from an untagged Chinese corpus using a heuristic rule inspired by Yarowsky (1993). Then we train a Naïve Bayesian (NB) classifier based on both the extracted instances and the official training data. We then use the NB classifier to predict situation the description formula and natural explanation of each target verb in testing data.</p><p>For the SRL module, we use a rich syntactic feature-based learning method. As the state-of-the-art method in the field of SRL, feature-based method represents a predicate-argument structure (PAS) by a flat vector using a set of linguistic features. Then PAS can be directly classified by machine learning algorithms based on the corresponding vectors. In feature-based SRL, the significance of syntactic information in SRL was proven by (Punyakanok et al., 2005).<page local="2" global="305"/> In our method, we exploit a rich set of syntactic features from two syntactic views: constituent and dependency. As the two syntactic views focus on different syntactic elements, constituent-based features and dependency-based features can complement each other in SRL to some extent. Finally, the ED module can be readily implemented by combining the SRL and the WSD result using some simply rules.</p></section><section number="2" title="System Description"><subsection number="2.1" title="Target Verb WSD"><p>The WSD module is based on a simple heuristic rule by which we can extract sense-labeled instances automatically. The heuristic rule assumes that one sense per 3-gram which is proposed by us initially through investigating a Chinese sense-tagged corpus STC (Wu et al., 2006). The assumption is similar to the celebrated one sense per collocation supposition (Yarowsky, 1993), whereas ours has more expansibility. STC is an ongoing project which is to build a sense-tagged corpus containing sense-tagged 1, 2 and 3 months of People's Daily 2000 now. According to our investigation, given a specific 3-gram (w_iWverbWi) to any target verb, on average, we expect to see the same label 95.4% of the time. Based on this observation, we consider one sense per 3-gram (w_iwverbwi) or at least we can extract instances with this pattern.</p><p>For all the 27 multiple-sense target verbs in the official training data, we found their 3-gram (w_iwverbwi) and extracted the instances with the same 3-gram from a Chinese monolingual corpus - the 2001 People's Daily (about 116M bytes). We consider the same 3-gram instances should have the same label. Then an additional sense­labeled training corpus is built automatically in expectation of having 95.4% precision at most. And this corpus has 2145 instances in total (official training data have 4608 instances).</p><p>We build four systems to investigate the effect of our instances expansion using the Naïve Bayesian classifier. System configuration is shown in Table 1. In column 1, BL means baseline, X means instance expansion, 3 and 15 means the window size. In column 2, <i>wi </i>is the <i>i­th </i>word relative to the target word, <i>wi-1wi </i>is the 2-gram of words, <i>w/j </i>is the word with position information (j<b>e</b>[-3,+3]). In the last column, 'O' means using only the original training data and 'O+A'  means using both the original and additional training data. Syntactic feature and parameter optimizing are not used in this module.</p></subsection><subsection number="2.2" title="Sentence SRL and Event Detection"><p>We use a feature-based machine learning method to implement the SRL module in which three tags are labeled, namely the semantic role tag, the event role tag and the phrase type tag. We consider the SRL task as a four-step pipeline: (1) <b>parsing </b>which generates a constituent parse tree for the input sentence; (2) <b>pruning </b>which filters out many apparently impossible constituents (Xue and Palmer, 2004); (3) <b>semantic role identification (SRI) </b>which identifies the constituent that will be the semantic role of a predicate in a sentence, and (4) <b>semantic role classification (SRC) </b>which determines the type of identified semantic role. The machine learning method takes PAS as the classification unit which consists of a target predicate and an argument candidate. The SRI step utilizes a binary classifier to determine whether the argument candidate in the PAS is a real argument. Finally, in the SRC step, the semantic role tag and the event role tag of each identified argument can be obtained by two multi-value classifications on the SRI results. The remaining phrase type tag can be directly extracted from the constituent parsing tree.</p><p>The selection of the feature set is the most important factor for the feature-based SRL method. In addition to constituent-based features and dependency-based features, we also consider WSD-based features. To our knowledge, the combined use of constituents-based syntactic features and dependency-based syntactic features is the first attempts to use them both on the feature level of SRL. As a prevalent kind of syntactic features for SRL, constituent-based features have been extensively studied by many researchers. In this module, we use 34 constituent-based features, 35 dependency-based features, and 2 WSD-based features. Among the constituent-based features, 26 features are manually selected from effective features proven by existing SRL studies and 8 new features are defined by us.<page local="3" global="306"/> Firstly, the 26 constituent-based features used by others are:</p><table caption="Table 1: The system configuration" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>System</b></p></td><td class="cell"><p><b>Features</b></p></td><td class="cell"><p><b>Window Size</b></p></td><td class="cell"><p><b>Training Data</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BL_3</p></td><td class="cell"><p></p></td><td class="cell"><p>±3</p></td><td class="cell"><p>O</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>X_3</p></td><td class="cell"><p></p></td><td class="cell"><p>±3</p></td><td class="cell"><p>O+A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BL_15</p></td><td class="cell"><p>±15</p></td><td class="cell"><p>O</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>X_15</p></td><td class="cell"><p></p></td><td class="cell"><p>±15</p></td><td class="cell"><p>O+A</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p><i>• predicate </i>(c1), <i>path </i>(c2), <i>phrase type </i>(c3), <i>position </i>(c4), <i>voice </i>(c5), <i>head word </i>(c6), <i>predicate subcategorization </i>(c7), <i>syntactic frame </i>(c8), head word POS (c9), <i>partial path </i>(c10), <i>first/last word </i>(c11/c12), <i>first/last POS </i>(c13/c14), <i>left/right sibling type </i>(c15/c16), <i>left/right sibling head </i>(c17/c18), <i>left/right sibling POS </i>(c19/c20), <i>constituent tree distance </i>(c21), <i>temporal cue words </i>(c22), <i>Predicate POS </i>(c23), <i>argument's parent </i>type(c24), <i>argument's parent head </i>(c25) and <i>argument's parent POS </i>(c26).</p><p>And the 8 new features we define are:</p><p><i>• Locational cue words </i>(c27): a binary feature indicating whether the constituent contains location cue word.</p><p><i>• POS pattern of argument </i>(c28): the left-to-right chain of POS tags of argument's children.</p><p><i>• Phrase type pattern of argument </i>(c29): the left-to-right chain of phrase type labels of argument's children.</p><p><i>• Type of LCA and left child </i>(c30): The phrase type of the Lowest Common Ancestor (LCA) combined with its left child.</p><p><i>• Type of LCA and right child </i>(c31): The phrase type of the LCA combined with its right child.</p><p>• Three features: <i>word bag of path </i>(c32), <i>word bag of POS pattern </i>(c33) and <i>word bag of type pattern </i>(c34), for generalizing three sparse features: <i>path </i>(c7), <i>POS pattern argument </i>(c28) and <i>phrase type pattern of argument </i>(c29) by the bag-of-words representation.</p><p>Secondly, the selection of dependency-based features is similar to that of constituent-based features. But dependency parsing lacks constituent information. If we want to use dependency-based features to label constituents, we should map a constituent to one or more appropriate words in dependency trees. Here we use head word of a constituent to represent it in dependency parses. The 35 dependency-based features we adopt are:</p><p><i>• Predicate/Argument relation </i>(d1/d2), <i>relation path </i>(d3), <i>POS pattern of predicate's children </i>(d4), <i>relation pattern of predicate's children </i>(d5) , <i>child relation set </i>(d6), <i>child POS set </i>(d7), <i>predicate/argument parent word </i>(d8/d9), <i>predicate/argument parent POS </i>(d10/d11), <i>left/right  word   </i>(d12/d13),   <i>left/right POS</i></p><doubt alpha="51.2" length="41" tooSmall="False" monospace="0.0">(d14/d15),left/right   relation(d16/d17),</doubt><p><i>left/right sibling word </i>(d18/d19), <i>left/right sibling POS </i>(d20/d21), <i>left/right sibling relation </i>(d22/d23), <i>dep-exists </i>(d24) and <i>dep-</i> (d27), <i>relation path length </i>(d28), <i>high/low support verb </i>(d29/d30), <i>high/low support noun </i>(d31/d32)   and   <i>LCA's word/POS/relation</i> (d33/d34/d35).</p><doubt alpha="66.7" length="39" tooSmall="False" monospace="0.0">type(d25),POS path(d26),POS path length</doubt><p>In this work, the dependency parse trees are generated from the constituent parse trees using a constituent-to-dependency converter (Marneffe et al., 2006). The converter is suitable for semantic analysis as it can retrieve the semantic head rather than the general syntactic head.</p><p>Lastly, the 2 WSD-based features are:</p><p><i>• Situation description formula </i>(s1): predicate's situation description formula generated by the</p><p>WSD module.</p><p><i>• Natural explanation </i>(s2): predicate's natural explanation generated by the WSD module.</p></subsection></section><section number="3" title="Experimental Results and Discussion"><subsection number="3.1" title="Target Verb WSD"><p>Table 2 shows the official result of the WSD system. BL_3 with window size three using the original training corpus achieves the best result in our submission. It indicates the local features are more effective in our systems. There are two possible reasons why the performances of the X system with instance expansion are lower than the BL system. First, the additional instances extracted based on 3-gram provide a few local features but many topical features. But, local features are more effective for our systems as mentioned above. The local feature related information that the classifier gets from the additional instances is not sufficient. Second, the granularity of the WSD module is too small to be distinguished by 3-grams. As a result, the additional corpus built upon 3-gram has more exceptional instances (noises), and therefore it impairs the performance of X_3 and X_15. Taking the verb ' <b>If</b><b> </b>' (belong to ) as an example, it has two senses in the task, but both senses have the same natural explanation: <b>'!Ê|— </b><i>Ä7:M^ÄÄ7:^W </i>(part of or belong to), which is always considered as the sense in general SRL. The difference between the two senses is in their situation description formulas: 'partof (x,y)+NULL' vs. 'belongto (x,y)+NULL'.</p><table caption="Table 2: Official results of the WSD systems" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>System</b></p></td><td class="cell"><p><b>Micro-A (%)</b></p></td><td class="cell"><p><b>Macro-A (%)</b></p></td><td class="cell"><p><b>Rank</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BL 3 X 3 BL 15 X 15</p></td><td class="cell"><p>81.30 79.82 79.23 77.74</p></td><td class="cell"><p>83.81 82.58 82.18 81.42</p></td><td class="cell"><p>3/7 4/7 5/7 6/7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="4" global="307"/></subsection><subsection number="3.2" title="Sentence SRL and Event Detection"><p>In the SRL module, we use the training data provided by SemEval-2010 to train the SVM classifiers without any external resources. The training data contain 4,608 sentences, 100 target predicates and 13,926 arguments. We use the SVM-Light Toolkit (Joachims, 1999) for the implementation of SVM, and use the Stanford Parser (Levy and Manning, 2003) as the parser and the constituent-to-dependency converter. We employ the linear kernel for SVM and set the regularization parameter to the default value which is the reciprocal of the average Euclidean norm of the training data. The evaluation results of our SRL module on the official test data are shown in Table 3, where 'AB', 'SR', 'PT' and 'ER' represent argument boundary, semantic role tag, phrase type tag, and event role tag.</p><p>It is clear that 'AB' plays an important role as the labeling of the other three tags is directly based on it. Through analyzing the results, we find that errors in the recognition of 'AB' are mainly caused by two factors: the automatic constituent parsing and the pruning algorithm. It is inevitable that some constituents and hierarchical relations are misidentified in automatic parsing of Chinese. These errors are further enlarged by the heuristic-based pruning algorithm because the algorithm is built upon the gold-standard paring trees, and therefore a lot of real arguments are pruned out when using the noisy automatic parses. So the pruning algorithm is the current bottleneck of SRL in the evaluation.</p><p>From the fact that the results of 'AB+SR' and 'AB+ER' are close to that of 'AB', it can be inferred that the SR and ER results should be satisfactory if the errors in 'AB' are not propagated. Furthermore, the result of 'AB+PT' is low as the phrase types here is inconsistent with those in Stanford Parser. The problem should be improved by a set of mapping rules.</p><p>Finally, in the ED module, we combine the results of WSD and SRL by filling variables of the situation description formula obtained by the WSD module with the arguments obtained by the SRL module according to their event role tags. Table 4 shows the final results which are generated by combining the results of WSD and SRL. Obviously the reduced overall ranking comparing to WSD is due to the SRL module.</p></subsection></section><section number="4" title="Conclusions"><p>In this paper, we propose a modular approach for the SemEval-2010 Task on Chinese event detection. Our system consists of three modules:</p><p>WSD, SRL and ED. The WSD module is based on instances expansion, and the SRL module is based on rich syntactic features. Evaluation results show that our system is good at WSD, semantic role tagging and event role tagging, but poor at pruning and boundary detection. In future studies, we will modify the pruning algorithm to reduce the bottleneck of the current system.</p></section><section title="Acknowledgments"><p>This work is partially supported by the Hong Kong Polytechnic University under Grant No. G-U297 and G-U596, and by the National Natural Science Foundation of China under Grant No. 60736014 and 60803094.</p><table caption="Table 3: Official results of the SRL system" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Precision(%)</b></p></td><td class="cell"><p><b>Recall(%)</b></p></td><td class="cell"><p><b>F-Score(%)</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AB</p></td><td class="cell"><p>73.10</p></td><td class="cell"><p>66.83</p></td><td class="cell"><p>69.82</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AB+SR</p></td><td class="cell"><p>67.44</p></td><td class="cell"><p>61.65</p></td><td class="cell"><p>64.42</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AB+PT</p></td><td class="cell"><p>61.78</p></td><td class="cell"><p>56.48</p></td><td class="cell"><p>59.01</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AB+ER</p></td><td class="cell"><p>69.05</p></td><td class="cell"><p>63.12</p></td><td class="cell"><p>65.95</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall</p></td><td class="cell"><p>58.33</p></td><td class="cell"><p>53.32</p></td><td class="cell"><p>55.71</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 4: Official results of the ED systems" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>System</b></p></td><td class="cell"><p><b>Micro-A (%)</b></p></td><td class="cell"><p><b>Macro-A (%)</b></p></td><td class="cell"><p><b>Rank</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BL 3 X 3 BL 15 X 15</p></td><td class="cell"><p>20.33 20.05 20.05 20.05</p></td><td class="cell"><p>20.19 20.23 20.22 20.14</p></td><td class="cell"><p>4/7 5/7 6/7 7/7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Thorsten Joachims. 1999. Making large-Scale SVM Learning Practical. Advances in Kernel Methods. <i>Support Vector Learning, </i>B. Schölkopf and C. Burges and A. Smola (ed), MIT Press.</p><p>Roger Levy and Christopher D. Manning. 2003. Is it harder to parse Chinese, or the Chinese Treebank.</p><p><i>Proceedings of ACL-2003.</i></p><p>Vasin Punyakanok, Dan Roth, and Wentau Yih. 2005. The necessity of syntactic parsing for semantic role labeling. <i>Proceedings of IJCAI-2005.</i></p><p>Yunfang Wu, Peng Jin, Yangsen Zhang, and Shiwen Yu. 2006. A Chinese corpus with word sense annotation. <i>Proceedings of ICCPOL-2006.</i></p><p>David Yarowsky. 1993. One sense per collocation.</p><p><i>Proceedings of the ARPA Workshop on Human Language Technology.</i></p><p>Qiang Zhou. 2010. SemEval-2010 task 11: Event detection in Chinese News Sentences. <i>Proceedings of SemEval-2010.</i></p></references></body></article>