<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="509"/><title>Barrier Features for Classification of Semantic Relations</title><pubinfo>Proceedings of Recent Advances in Natural Language Processing,pages 509-514, Hissar, Bulgaria, 12-14 September 2011.</pubinfo><author surname="Alicante" givenname="Anita"><org  name='University "Federico II' country="Italy"/></author><author surname="Corazza" givenname="Anna"><org  name='University "Federico II' country="Italy"/></author></firstpageheader><frontmatter><p><b>Barrier Features for Classification of Semantic Relations</b></p><p><b>Anita Alicante Anna Corazza</b></p><p>University "Federico II", Naples, Italy    University "Federico II", Naples, Italy anita.alicante@unina.it corazza@na.infn.it</p></frontmatter><abstract>Approaches based on machine learning, such as Support Vector Machines, are of­ten used to classify semantic relations be­tween entities. In such framework, classi­fication accuracy strongly depends on the set of features which are used to represent the input to the classifier. We are propos­ing here a new type of features, namely the <i>barrier features, </i>which can be used in ad­dition to more usual features, such as <i>n-</i>grams of PoS, word suffixes and prefixes, hypernyms from WordNet etc., and to the parse tree of the whole sentence. Barrier features aim at giving a compact repre­sentation of the context of each entity in­volved in the relation. The effectiveness of the new features is assessed on documents from the TREC data set annotated by Roth and Yih. The obtained results show not only that the performance of the proposed approach are state-of-the-art but also that such improvement is due to the introduc­tion of the barrier features. </abstract></header><body><section number="1" title="Introduction"><p>Different approaches have been proposed for the identification of entities and relations in text. In this paper we focus only on the task of classifica­tion of semantic relations, that is of assigning to each semantic relation a label taken from a finite set, and therefore we assume that pairs of related entities are given us together with their labels. In general, not all relation labels are compatible with every pair of entity types. Table 3 reports all such constraints in the considered data set. Thus not all relation labels can be compatible with all en­tity pairs and therefore we decompose the prob­lem in several binary classifications, one for each possible relation label and apply Support Vector Machines (SVMs) to each of these subtasks.</p><p>Moreover, by applying SVMs with a combi­nation of different kernel functions we can han­dle together different kinds of information, both structured and not. In fact, we applied tree ker­nels (Moschitti, 2006) to the whole sentence parse tree and a linear kernel to a vector of binary fea­tures extracted from the words surrounding each of the involved entities. Among the latter, we introduce a novel kind of binary feature, which we call <i>barrier features </i>and experimentally prove that they improve classification performance. Al­though inspired by the barrier rules of the con­straint grammar framework proposed in (Karlsson et al., 1995) for PoS tagging, barrier features have been completely redesigned for this task as binary values rather than rules.</p><p>The experimental assessment of the approach is performed on the newswire documents annotated by Roth and Yih (Roth and Yih, 2004). The best published results for relation classification on this data set have been obtained by the MO|K system, described in (Giuliano et al., 2007). The follow­ing Section 2 is devoted to the discussion of related works. Afterwards, the approach we are proposing is presented in Section 3, with a discussion of the adopted features, and in particular of barrier fea­tures. In Section 4 the experimental assessment is described. In the final section some planned de­velopment of the presented results are considered.</p></section><section number="2" title="Related work"><p>In the past few years a lot of works have been de­voted to relation extraction and classification. Be­cause of space limitation, we are giving a prefer­ence here to systems which have been assessed on the Roth and Yih data set. Systems assessed on other data sets include (Beamer et al., 2007; Davidov and Rappoport, 2008) for the Task 4 of SemEval07, (Rink and Harabagiu, 2010) for the Task08 of SemEval10 and (Kambhatla, 2004; Cu­lotta and Sorensen, 2004; Guodong et al., 2005;<page local="2" global="510"/></p><doubt alpha="52.1" length="48" tooSmall="False" monospace="0.0">GuoDong et al., 2006; Qian et al., 2008) for the</doubt><p>Automatic Content Extraction (ACE), which is not freely available.</p><p>Systems devoted to relation extraction and clas­sification usually first extract and label entities and afterwards relations. An important exception to this two pass approach is represented by (Roth and Yih, 2007), where entity and relation extrac­tion and labeling are integrated and (Kate and Mooney, 2010) where a new method for joint en­tity and relation extraction is presented using a "card-pyramid" graph.</p><p>Most relation labeling systems are based on some machine learning approach, and build a clas­sifier which associates a label to a representation of the input sentence. In such approaches the representation of the input is crucial, as only the information it contains can be used for labeling. Nearly all such systems consider some form of parsing: the complete parse tree of the input sen­tence is considered, among the others, by (Miller et al., 2000) and (Kambhatla, 2004), which con­siders both constituency and dependency parse trees. Another approach based on dependency parse trees is presented in (Reichartz et al., 2009). Systems that instead of the complete parse tree only consider some form of shallow parsing in­clude (Giuliano et al., 2007) and (Zhang et al., 2005).</p><p>We compare our performance with the MO|Kon the Roth and Yih data set (Giuliano et al., 2007). In addition to presenting a novel approach to relation extraction and labeling, they evaluate the effect of automatic named-entity recognition on its performance. Their approach is based on shallow linguistic features, which are combined with semantic information, such as WordNet hy-pernym relations of the candidate entities. Ker­nels are employed to combine two different infor­mation sources: the global context where the two entities appear and (independently) the two local contexts of the entities. A specific kernel function is associated to each of the different types of infor­mation.</p></section><section number="3" title="The proposed approach 3.1   Problem definition"><p>In this subsection, we briefly introduce a few def­initions together with some examples. A sentence of length n is a string of n tokens <i>S = tit2 </i>... <i>tn</i>and can include a number <i>N &gt; </i>0 of entities <i>{Ei,E2,.</i><i>.., EN</i>}, each corresponding to a se­quence of consecutive tokens, that is a substring of <i>S. </i>The entity indexes follow their order in the sen­tence, and each entity is labeled by an <i>entity-type </i>in a finite set E of labels. Although in the cor­pus we used for assessment entities are also rep­resented by noun phrases, this definition is more general.</p><p>A subset of all ordered entity pairs corresponds to relations: Rj;j = <i>(Ej,, Ej</i>); Ej is called <i>agent </i>and Ej <i>target, </i>where the entities Ej and Ej can be composed by one or more tokens of the sentence and Ej can either precede or follow Ej. This def­inition excludes cross-sentence relations. A label taken from a finite set R of possible labels is as­sociated to each relation. We are considering the task of associating the correct label to each rela­tion, that is the <i>classification </i>task of <i>sematic rela­tions.</i></p><p>For the sake of clarity, let us consider the ex­ample sentence si of Table 1. It contains four dif­ferent relations containing six entities, namely (ei, e2) with label "work for", (e2, <i>e3) </i>with label "org-based in", (e4, e5) labeled as "work for", and (e5, <i>e6) </i>for "orgbased in". Indeed, entities e2 and e5are involved in two different relations.</p><subsection number="3.2" title="The proposed solution"><p>For each possible relation label we build a bi­nary classifier based on SVMs which takes as in­put both a feature vector and the parse tree of the whole sentence. The former refers to the two input entities and its elements are therefore called <i>entity features, </i>while the latter refers to the whole sen­tence. Entity features include word and PoS un-igrams, PoS bigrams and trigrams, word prefixes and suffixes, word length, and a set of word fea­tures indicating whether the initial letter is upper case, whether all letters are upper or lower case and whether the token contains a period or num­ber or hyphen. Furthermore, we also included the most likely WordNet<footnote anchor="1"/> (Fellbaum, 1998) sense tag for each token involved in the entity, which always corresponds to the first one in the list of possible senses, together with all the hypernyms.</p><p>These two sets of features are combined in the SVM-based classifier by integrating two kernels, namely tree kernels (Moschitti, 2006) and a linear kernel. The former is applied to the parse tree and evaluate the similarity between two trees in terms<page local="3" global="511"/></p><footnote label="1"> http://wordnet.princeton.edu/</footnote><p>si Also being considered are (ei)<i>Judge Ralph K. Winter </i>(/ei)of the <i>{e</i><i>2)2nd U. S. Circuit Court of Appeals</i><i>(/e2) </i>in <i>{e</i><i>3)New York City </i><i>{/e3) </i>and {e4) <i>Judge Kenneth Starr </i><i>{/e4) </i>of the <i>{e</i><i>5)U. S. Circuit Court of Appeals</i><i>{/e5) </i>for the <i>{e</i><i>6)District of Columbia</i><i>{/e6), </i>said the source, who spoke on condition of anonymity.</p><p>S2 The/DT spy/NN ,/, high-ranking/JJ {e2)Korean/JJ CIA/NNP{/e2) official/JJ {e1)Sohn/NNP Ho/NNP Young/NNP{/e1) ,/, wanted/VBD to/TO defect/VB ....</p><p>Table 1: <i>Example sentences taken from the Roth and Yih data set used for assessment.</i></p><p>of the number of fragments they have in common; the latter has been chosen in the system tuning phase as described in Section 4. The same weight is associated to the two kernels.</p><subsubsection number="3.2.1" title="Barrier features"><p>In addition to these, we consider a novel kind of features, which we call <i>barrier features, </i>to model the context of each token in entities. Their defini­tion is based on the set of PoS tags in a window surrounding the token. The length of the window varies and is based on the PoS's of the correspond­ing tokens: for each token in the entity <i>(trigger), </i>an <i>endpoint </i>token is chosen on the basis of the PoS of the trigger. In fact, for each token in the en­tity the corresponding endpoint is defined as the closest preceding or following token having one of the PoS associated with the PoS of the consid­ered token. Such (trigger PoS, endpoint PoS) pairs are predefined and depend on the considered lan­guage: in the experiments we used the ones re­ported in Table 2.</p><p>In the task we are considering here, barrier fea­tures aim at describing the syntactic context of tokens in entities, which can only be nouns or adjectives. Therefore, we only considered pat­terns for this PoS, while completely disregarding other important PoS tags including verbs. On the other hand, endpoints try to bound the interest­ing context of the considered trigger. The choice of the pairs (trigger PoS, endpoint PoS) has been inspired by the corresponding barrier rules. We think that their favourable impact is connected to their complementarity to simpler features like bi-grams and trigrams on one side and the complete parse tree on the other. We plan to explore the pos­sibility of considering other pairs in the future.</p><p>In the experiments described in Section 4 bar­rier features only consider the case where the end­point token precedes the entity. An entity token can have several endpoints and a new barrier fea­ture corresponding to the set of PoS's between the endpoint and the token is introduced for every possible endpoint. If no endpoint is found before the entity token, the set of all the PoS tags from the beginning of the sentence to this entity are consid­ered. As the barrier features are based on <i>sets </i>of tags, order and possible repetitions of tags are not considered.</p></subsubsection></subsection><subsection number="3.3" title="Smoothing"><p>A very large number of different barrier features can occur in addition to all other usual features and therefore the choice of the smoothing strategy is crucial. First of all, if the feature we observe in the input to the classifier does not exist in the set of features collected on the training set, then we consider all barrier features having the same (trigger PoS, endpoint PoS) pair and a PoS's set including the considered one. A side effect of this strategy is that more than one barrier feature can be positive (equal to 1) at the same time.</p><p>Furthermore, we introduce for every kind of feature a new feature UNKNOWN which intuitively corresponds to the unseen (rare) case. We train this feature by cumulating all cases having less than 3 occurrences. Therefore, there is an unknown feature for barriers, one for word unigrams, one for PoS unigrams, and so on. Whenever an input would not activate any value for a given type of features, the corresponding unknown feature is set.</p><p>In the sentence s2 in Table 1 the relation cor­responding to the entity pair <i>(ei,e2) </i>is labeled as <i>work for. </i>As entity e2 is composed by two tokens ("Korean CIA") the corresponding feature vector results from the OR combination of the features corresponding to each token. If the entity were composed by only one token, the feature vector would only contain 1's in correspondence of the features computed for this token.</p><p>Thus, features based on words are extracted from the window "The/DT spy/NN ,/, high-ranking/JJ Korean/JJ CIA/NNP". Barrier features construction is based on a window whose length is not predetermined, but depends on the PoS's of the tokens preceding the one we are considering, in this case "CIA".<page local="4" global="512"/> Since "CIA" PoS is NNP, we apply the first rule reported in Table 2: the end­point is the closest determiner preceding the token <i>CIA, </i>namely <i>The. </i>In this case the endpoint does not belong to the entity, but this is not always so. The resulting barrier feature is then given by the set <i>{JJ,</i><i> NN, ,}</i>, and contains, as discussed, only one repetition of <i>JJ,</i><i> </i>corresponding to the tokens <i>high-ranking/JJ Korean/JJ</i>, <i>spy/NN </i>and <b>Endpoint Trigger</b></p><p>DT NN, NNP PRP NNS JJ JJR, RBR</p><p>Table 2: <i>Endpoints PoS and Trigger PoS </i><i>ofthe</i><i> barrier features employed in the assessment.</i></p><p>All features we consider are binary, taking val­ues 0 if the considered pattern does not occur, 1 otherwise. The entity feature vector is con­structed by merging a different feature vector for each of the tokens composing the considered enti­ties by a logical OR: the element of the final vector takes value 1 if the corresponding features takes value 1 in at least one of the all involved vec­tors. More precisely, let Ej = ij;ki and Ej = <i>tji,tj,kj </i>be the two entities we are con­sidering. To obtain the feature vector correspond­ing to this entity pair, we merge the feature vectors of <i>tj,i,</i>tj,ki and <i>tj,i, ....,tj&gt;kj </i>corresponding to both entities. Note that this representation is inde­pendent of the order of the two considered entities.</p></subsection></section><section number="4" title="Experimental evaluation"><p>The aim of the experimental assessment is twofold: to verify whether the system employing barrier features is competitive with state of the art systems, and to evaluate the role of barrier features in the results. Performance is evaluated by com­puting Precision (P), Recall (R) and <i>Fi, </i>as usual.</p><subsection number="4.1" title="Data set"><p>For experimental assessment we used the data set used by Roth and Yih (Roth and Yih, 2004), de­rived from TREC corpus<footnote anchor="2"/>, which is freely avail­able. It includes three types of entities, namely</p><footnote label="2">The annotated data are freely available at http: //l2r.cs.uiuc.edu/~ cogcomp/Data/ER/ conll04.corp</footnote><p>PER (person), LOC (location) and ORG (orga­nization) and the five types of binary relations re­ported in Table 3.</p><p>Table 3: List of relations with the type of the in­volved entities and the number of occurrences in the Roth Yih Data set.</p><p>The Roth and Yih data set is not divided in training and test set. Therefore assessment is per­formed by following the 5-fold cross validation protocol, as in (Giuliano et al., 2007; Roth and</p><p>Yih, 2007; Kate and Mooney, 2010).</p></subsection><subsection number="4.2" title="System tuning and kernel choice"><p>The Roth and Yih data set comes along with the correct PoS tagging and therefore we consider gold case PoS's while constructing the features, as in (Giuliano et al., 2007). Then, the syntactic parse tree is automatically associated to each input sentence by using the Stanford Parser (Klein and employing the grammar for English distributed to­gether with the parser. For the sake of precision, we mention that we do not give correct PoS's in input to the parser.</p><doubt alpha="56.5" length="46" tooSmall="False" monospace="0.0">Manning, 2003a; Klein and Manning, 2003b)3, by</doubt><p>The classification was performed by using the</p><p>SVM package SVMLight-TK<footnote anchor="4"/> (Moschitti, 2006), which is based on SVMLight (Joachims, 1999), but also includes tree kernels, offering the possi­bility of combining tree-structured features with vectors, which is what we need to combine the in­put syntactic tree with the entity feature vector. In such approach, a further kernel can be introduced in addition to the tree kernel to apply to the un­structured features. To choose this second kernel, we compared the performance of different combi­nations including the tree kernel alone and in con­junction with other kernels.</p><p>As the data set has not been split in training and test sets, performance has been evaluated by following a 5-fold cross-validation protocol, includ­ing 5 iterations with a different split in training and test sets at each step.<page local="5" global="513"/> The choice of the best kernel combination has been again based on a 5­fold cross-validation protocol, applied to the train­ing set defined at each iteration. Significantly, the linear kernel showed the best performance for all split.</p><footnote label="3">The parser can be freely downloaded from http: //nlp.stanford.edu/software/lex-parser. shtml.</footnote><footnote label="4">The package is available from http://dit.unitn . it/~moschitt/Tree-Kernel.htm.</footnote><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Relation</b></p></td><td class="cell"><p><b>Example</b></p></td><td class="cell"><p><b>agent</b></p></td><td class="cell"><p><b>target</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>work for</p></td><td class="cell"><p>employ-company</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>ORG</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>kill</p></td><td class="cell"><p>murderer-victim</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>PER</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>live in</p></td><td class="cell"><p>Clinton-USA</p></td><td class="cell"><p>PER</p></td><td class="cell"><p>LOC</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>located in</p></td><td class="cell"><p>Rome - Italy</p></td><td class="cell"><p>LOC</p></td><td class="cell"><p>LOC</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>orgbased in</p></td><td class="cell"><p>Harvad -U.S.</p></td><td class="cell"><p>ORG</p></td><td class="cell"><p>LOC</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></subsection><subsection number="4.3" title="System performance"><p>Assessment considers five classifiers, one for each relation. The data set is divided in subsets corre­sponding to the different relations. For each re­lation, training has been performed by consider­ing gold positive examples for the considered re­lation while negative examples are represented by all the other pairs of entities having labels com­patible with the relation. In this way, the num­ber of negative examples is much larger than for positive examples. The SVM implementation we used allows to balance the number of positive and negative examples by a cost factor. We set it to the rate between the number of negative and posi­tive examples. Table 4 reports the comparison be­tween the performance of our system and the re­sults presented in (Giuliano et al., 2007) for the MO|K system. With the only exception of the <i>lo­cated in </i>relation, our system has an Fi larger than MO|K both on single relations and on average. Al­though we are not able to estimate the statistical significance of such comparison because we do not have the output of that system on each sen­tence, we think that this consistency is quite con­vincing. Note however that in two cases their pre­cision is better than ours, and in three cases their recall is better. However, the average values are always better for our system. Although we are not reporting the exact results here, we noticed that the WordNet features (hypernyms of each entity tokens) do not give any significant improvement on performance.</p></subsection><subsection number="4.4" title="Barrier feature contribution"><p>Last but not least, we tried to understand the con­tribution of barrier feature to the global system performance. In order to obtain a numerical esti­mation, we run exactly the same experiment with and without barrier features. Results are reported in Table 4 and show that their contribution to per­formance is always relevant and on average can be evaluated in an improvement in Fi of nearly the 15%.</p></subsection></section><section number="5" title="Conclusions and future work"><p>In this work we proposed a new kind of features for classifying semantic relations and showed how they are indeed effective in improving classifica­tion performance. Experimental assessment on the Roth and Yih data set not only shows that they per­formance are state of the art, but also that their contribution is relevant.</p><p>In the future, we plan to assess the barrier fea­tures on new data sets and on different tasks, such as relation extraction and entity classification. As the number of possible barrier features is very large, we plan to invest on the search for an effec­tive smoothing strategy, in order to limit as much as possible the effect of data sparsity.</p></section><section number="6" title="Aknowledgments"><p>We are in debt with Giorgio Satta for indicating us the potentialities of barrier rules.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>OS No Barrier Features</b></p></td><td class="cell"><p><b>OS With Barrier Features</b></p></td><td class="cell"><p><b>Mo|k</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Relation</b></p></td><td class="cell"><p><b>P</b></p></td><td class="cell"><p><b>R</b></p></td><td class="cell"><p><b>F1</b></p></td><td class="cell"><p><b>P</b></p></td><td class="cell"><p><b>R</b></p></td><td class="cell"><p><b>F1</b></p></td><td class="cell"><p><b>P</b></p></td><td class="cell"><p><b>R</b></p></td><td class="cell"><p><b>F1</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>kill</p></td><td class="cell"><p>70.30</p></td><td class="cell"><p>71.29</p></td><td class="cell"><p>70.79</p></td><td class="cell"><p>92.39</p></td><td class="cell"><p><b>75.63</b></p></td><td class="cell"><p>83.17</p></td><td class="cell"><p>82.80</p></td><td class="cell"><p>81.00</p></td><td class="cell"><p>81.89</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>live in</p></td><td class="cell"><p>73.26</p></td><td class="cell"><p>63.65</p></td><td class="cell"><p>68.12</p></td><td class="cell"><p>74.69</p></td><td class="cell"><p><b>73.39</b></p></td><td class="cell"><p>74.33</p></td><td class="cell"><p>78.00</p></td><td class="cell"><p>65.80</p></td><td class="cell"><p>71.38</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>work for</p></td><td class="cell"><p>66.22</p></td><td class="cell"><p>63.12</p></td><td class="cell"><p>64.63</p></td><td class="cell"><p><b>76.38</b></p></td><td class="cell"><p>86.18</p></td><td class="cell"><p>80.99</p></td><td class="cell"><p>76.80</p></td><td class="cell"><p>80.00</p></td><td class="cell"><p>78.37</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>located in</p></td><td class="cell"><p>61.53</p></td><td class="cell"><p>72.23</p></td><td class="cell"><p>71.88</p></td><td class="cell"><p><b>70.00</b></p></td><td class="cell"><p><b>75.40</b></p></td><td class="cell"><p><b>72.60</b></p></td><td class="cell"><p>79.60</p></td><td class="cell"><p>76.00</p></td><td class="cell"><p>77.76</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>orgabased in</p></td><td class="cell"><p>68.13</p></td><td class="cell"><p>66.33</p></td><td class="cell"><p>67.22</p></td><td class="cell"><p>86.58</p></td><td class="cell"><p>77.70</p></td><td class="cell"><p>81.90</p></td><td class="cell"><p>74.30</p></td><td class="cell"><p>77.20</p></td><td class="cell"><p>75.72</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>average</p></td><td class="cell"><p>69.89</p></td><td class="cell"><p>67.32</p></td><td class="cell"><p>68.53</p></td><td class="cell"><p>80.01</p></td><td class="cell"><p>77.66</p></td><td class="cell"><p>78.54</p></td><td class="cell"><p>78.30</p></td><td class="cell"><p>76.00</p></td><td class="cell"><p>77.02</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Brandon Beamer, Suma Bhat, Brant Chee, Andrew Fister, Alla Rozovskaya, and Roxana Girju. 2007. UIUC: A Knowledge-rich Approach to Identifying Semantic Relations between Nominals. In <i>Proc. of SemEval07: the Fourth International Workshop on Semantic Evaluations, </i>pages 386-389, Prague, Czech Republic, June. Association for Computa­tional Linguistics.</p><p>Aron Culotta and Jeffrey Sorensen. 2004. Depen­dency tree kernels for relation extraction. In <i>Proc. of ACL04: 42nd Meeting of the Association for Com­putational Linguistics , Main Volume, </i>pages 423­429, Barcelona, Spain, July.</p><p>Dmitry Davidov and Ari Rappoport. 2008. Classifi­cation of Semantic Relationships between Nominals Using Pattern Clusters. In <i>Proc. of ACL-08: HLT, </i>pages 227-235, Columbus, Ohio, June. Association for Computational Linguistics.</p><p>Christiane Fellbaum, editor. 1998. <i>WordNet: An Elec­tronic Lexical Database. </i>MIT Press, Cambridge, MA.</p><p>Claudio Giuliano, Alberto Lavelli, and Lorenza Ro­mano. 2007. Relation extraction and the influence of automatic named-entity recognition. <i>ACMTrans.</i></p><p><i>Speech Lang. Process., </i>5(1):1-26.</p><p>Zhou Guodong, Su Jian, Zhang Jie, and Zhang Min. 2005. Exploring various knowledge in relation ex­traction. In <i>ACL '05: Proceedings of the 43rd An­nual Meeting on Association for Computational Lin­guistics, </i>pages 427-434, Morristown, NJ, USA. As­sociation for Computational Linguistics.</p><page local="6" global="514"/><p>Table 4: Comparison of performance of Our System (OS) with and without barrier features and the best performing one MO|K on the Roth and Yih data set.</p><p>Zhou GuoDong, Su Jian, and Zhang Min. 2006. Modeling commonality among related classes in re­lation extraction. In <i>Proceedings of the 21st In­ternational Conference on Computational Linguis­tics and the 44th annual meeting ofthe Association for Computational Linguistics, </i>ACL-44, pages 121­128, Stroudsburg, PA, USA. Association for Com­putational Linguistics.</p><p>Thorsten Joachims. 1999. Making large-scale sup­port vector machine learning practical. In <i>Advances in kernel methods: support vector learning, </i>pages 169-184. MIT Press, Cambridge, MA, USA.</p><p>Nanda Kambhatla. 2004. Combining lexical, syn­tactic, and semantic features with maximum en­tropy models for information extraction. In <i>Proc. of ACL04: Companion Volume to the Proceedings of42st Annual Meeting ofthe Association for Com­putational Linguistics, </i>pages 178-181, Barcelona, Spain, July. Association for Computational Linguis­tics.</p><p>Fred Karlsson, Atro Voutilainen, Juha Heikkila, and Arto Anttila, editors. 1995. <i>Constraint Grammar: A Language-Independent System for Parsing Unre­stricted Text. </i>Mouton de Gruyter.</p><p>Rohit J. Kate and Raymond J. Mooney. 2010. Joint entity and relation extraction using card-pyramid parsing. In <i>Proceedings ofthe Fourteenth Confer­ence on Computational Natural Language Learning, </i>CoNLL '10, pages 203-212, Stroudsburg, PA, USA. Association for Computational Linguistics.</p><p>Dan Klein and Christopher D. Manning. 2003a. Accu­rate unlexicalized parsing. In <i>Proc. ofACL 03 ofthe 41st Annual Meeting on Association for Computa­tional Linguistics, </i>pages 423-430, Morristown, NJ, USA. Association for Computational Linguistics.</p><p>Dan Klein and Christopher D. Manning. 2003b. Fast exact inference with a factored model for natural language parsing. In <i>Proc ofNIPS03: In Advances in Neural Information Processing Systems, </i>pages 3­10. MIT Press.</p><p>Scott Miller, Heidi Fox, Lance Ramshaw, and Ralph Weischedel. 2000. A novel use of statistical pars­ing to extract information from text.  In <i>Proc. of</i> <i>ANLP00:</i><i> In 6th Applied Natural Language Process­ing Conference, </i>pages 226-233.</p><p>Alessandro Moschitti. 2006. Making tree kernels prac­tical for natural language learning. In <i>EACL.</i></p><p>Longhua Qian, Guodong Zhou, Fang Kong, Qiaoming Zhu, and Peide Qian. 2008. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In <i>Proceedings ofthe 22nd International Conference on Computational Linguistics - Volume </i>1, COLING '08, pages 697-704, Stroudsburg, PA, USA. Association for Computational Linguistics.</p><p>Frank Reichartz, Hannes Korte, and Gerhard Paass. 2009. Dependency tree kernels for relation ex­traction from natural language text. In Wray Buntine, Marko Grobelnik, Dunja Mladenic, and John Shawe-Taylor, editors, <i>Machine Learning and Knowledge Discovery in Databases, </i>volume 5782, chapter 18, pages 270-285. Springer Berlin Heidel­berg, Berlin, Heidelberg.</p><p>Bryan Rink and Sanda Harabagiu. 2010. Utd: Clas­sifying semantic relations by combining lexical and semantic resources. In <i>Proceedings ofthe 5th Inter­national Workshop on Semantic Evaluation, </i>pages 256-259, Uppsala, Sweden, July. Association for Computational Linguistics.</p><p>D. Roth and W. Yih. 2004. A Linear Programming Formulation for Global Inference in Natural Lan­guage Tasks. In <i>Proc. ofCoNLL-2004, </i>pages 1-8.</p><p>Boston, MA, USA.</p><p>D. Roth and W. Yih. 2007. Global inference for entity and relation identification via a linear programming formulation. In Lise Getoor and Ben Taskar, editors, <i>Introduction to Statistical Relational Learning. </i>MIT Press.</p><p>Min Zhang, Jian Su, Danmei Wang, Guodong Zhou, and Chew Lim Tan. 2005. Discovering relations between named entities from a large raw corpus us­ing tree similarity-based clustering. In <i>Proc. ofIJC-NLP05: International Joint Conference on Natural Language Processing, </i>pages 378-389.</p></references></body></article>