<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="252"/><title>UTDMet: Combining WordNet and Corpus Data for Argument Coercion Detection</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 252-255, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Roberts" givenname="Kirk"><org  name="Duke University" country="USA" city="Durham"/></author><author surname="Harabagiu" givenname="Sanda"><org  name="University of Texas at Dallas" country="USA" city="Richardson"/></author></firstpageheader><frontmatter><p><b>UTDMet: Combining WordNet and Corpus Data for Argument Coercion Detection</b></p><p><b>Kirk Roberts and Sanda Harabagiu</b></p><p>Human Language Technology Research Institute University of Texas at Dallas Richardson, Texas, USA {kirk,sanda}@hlt.utdallas.edu</p></frontmatter><abstract>This paper describes our system for the classification of argument coercion for SemEval-2010 Task 7. We present two ap­proaches to classifying an argument's se­mantic class, which is then compared to the predicate's expected semantic class to detect coercions. The first approach is based on learning the members of an arbi­trary semantic class using WordNet's hy-pernymy structure. The second approach leverages automatically extracted seman­tic parse information from a large corpus to identify similar arguments by the pred­icates that select them. We show the re­sults these approaches obtain on the task as well as how they can improve a tradi­tional feature-based approach. </abstract></header><body><section number="1" title="Introduction"><p>Argument coercion (a type of <i>metonymy) </i>occurs when the expected semantic class (relative to the a predicate) is substituted for an object of a dif­ferent semantic class. Metonymy is a pervasive phenomenon in language and the interpretation of métonymie expressions can impact tasks from se­mantic parsing (Scheffczyk et al, 2006) to ques­tion answering (Harabagiu et al, 2005). A seminal example in metonymy from (Lakoff and Johnson, 1980) is:</p><p>(1) The ham sandwich is waiting for his check.</p><p>The Arg 1 for the predicate <i>wait </i>is typically an animate, but the <i>"ham sandwich" </i>is clearly not an animate. Rather, the argument is coerced to ful­fill the predicate's typing requirement. This coer­cion is allowed because an object that would nor­mally fulfill the typing requirement (the customer) can be uniquely identified by an attribute (the ham sandwich he ordered).</p><p>SemEval-2010 Task 7 ("Argument Selection and Coercion") (Pustejovsky and Rumshisky, 2009) was designed to evaluate systems that de­tect such coercions and provide a "compositional history" of argument selection relative to the pred­icate. In order to accomplish this, an argument is annotated with both the semantic class to which it belongs (the "source" type) as well as the class ex­pected by the predicate (the "target" type). How­ever, in the data provided, the target type was un­ambiguous given the lemmatized predicate, so the remainder of this paper discusses source type clas­sification. The detection of coercion is then sim­ply performed by checking if the classified source type and target type are different.</p><p>In our system, we explore two approaches with separate underlying assumptions about how arbi­trary semantic classes can be learned. In our first approach, we assume a semantic class can be de­fined a priori from a set of seed terms and that WordNet is capable of defining the membership ofthat semantic class. We apply the PageRank al­gorithm in order to weight WordNet synsets given a set of seed concepts. In our second approach, we assume that arguments in the same semantic class will be selected by similar verbs. We apply a statistical test to determine the most representative predicates for an argument. This approach benefits from a large corpus from which we automatically extracted 200 million predicate-argument pairs.</p><p>The remainder of this paper is organized as fol­lows. Section 2 discusses our WordNet-based ap­proach. Section 3 describes our corpus approach. Section 4 discusses our experiments and results. Section 5 provides a conclusion and direction for future work. Due to space limitations, previous work is discussed when relevant.</p></section><section number="2" title="PageRanking WordNet Hypernyms"><p>Our first approach assumes that semantic class members can be defined and acquired a priori.</p><page local="2" global="253"/><p>Given a set of seed concepts, we mine WordNet for other concepts that may be in the same seman­tic class. Clearly, this approach has both practical limitations (WordNet does not contain every pos­sible concept) and linguistic limitations (concepts may belong to different semantic classes based on their context). However, given the often vague na­ture of semantic classes (is a <i>building </i>an Arti­fact or a Location?), access to a weighted list of semantic class members can prove useful for ar­guments not seen in the train set.</p><p>Using (Esuli and Sebastiani, 2007) as inspira­tion, we have implemented our own naive ver­sion of WordNet PageRank. They use sense-disambiguated glosses provided by extended WordNet (Harabagiu et al, 1999) to link synsets by starting with positive (or negative) sentiment concepts in order to find other concepts with pos­itive (or negative) sentiment values. For our task, however, hypernymy relations are more ap­propriate for determining a given synset's mem­bership in a semantic class. Hypernymy de­fines an Is-A relationship between the parent class (the <i>hypernym) </i>and one of its child classes (the <i>hyponym). </i>Furthermore, while PageRank as­sumes directed edges (e.g., hyperlinks in a web page), we use undirected edges. In this way, if HypernymOf(A, <i>B),</i><i> </i>then <i>A's </i>membership in a semantic class strengthens <i>B's </i>and vice versa.</p><p>Briefly, the formula for PageRank is:</p><doubt alpha="30.8" length="26" tooSmall="False" monospace="0.0">a&lt;fc&gt; = aa^W +(1 - a)e (1)</doubt><p>where <i>a^ </i>is the weight vector containing weights for every synset in WordNet at time <i>k; </i><b>Wjj </b>is the inverse of the total number of hypernyms and hy-ponyms for synset <i>i </i>if synset <i>j </i>is a hypernym or hyponym of synset <i>i; </i>e is the initial score vector; and a is a tuning parameter. In our implementa­tion, a(°) is initialized to all zeros; <i>a </i>is fixed at 0.5; and     = 1 if synset <i>i </i>is in the seed set <i>S,</i> and zero otherwise. The process is then run until convergence, defined by <i>\a]</i><i> </i><i>'</i><i> </i><i>-</i><i> </i><i>a]</i><i> </i><i>'\</i><i> </i><i>&lt;</i><i> </i>0.0001 for all <i>i.</i></p><doubt alpha="50.0" length="10" tooSmall="False" monospace="0.0">(if)(if—i)</doubt><p>The result of this PageRank is a weighted list containing every synset reachable by a hyper-nym/hyponym relation from a seed concept. We ran the PageRank algorithm six times, once for each semantic class, using the arguments in the train set as seeds. For arguments that are polyse-mous, we make a first WordNet sense assumption. Representative examples of the concepts gener­ated from this approach are shown in Table 1.</p></section><section number="3" title="Leveraging a Large Corpus of Semantic Parse Annotations"><p>Our second approach assumes that semantic class members are arguments of similar predicates. As (Pustejovsky and Rumshisky, 2009) elaborate, predicates select an argument from a specific se­mantic class, therefore terms that belong in the same semantic class should be selected by simi­lar predicates. However, this assumption is often violated: type coercion allows predicates to have arguments outside their intended semantic class. Our solution to this problem, partially inspired by (Lapata and Lascarides, 2003), is to collect statis­tics from an enormous amount of data in order to statistically filter out these coercions.</p><p>The English Gigaword Forth Edition corpus<footnote anchor="1"/>contains over 8.5 million documents of newswire text collected over a 15 year period. We processed these documents with the SENNA<footnote anchor="2"/> (Collobert and Weston, 2009) suite of natural language tools, which includes a part-of-speech tagger, phrase chunker, named entity recognizer, and PropBank semantic role labeler. We chose SENNA due to its speed, yet it still performs comparably with many state-of-the-art systems. Of the 8.5 million doc­uments in English Gigaword, 8 million were suc­cessfully processed. For each predicate-argument pair in these documents, we gathered counts by argument type and argument head. The head was determined with simple heuristics from the chunk parse and parts-of-speech for each argument (ar­guments consisting of more than three phrase chunks were discarded). When available, named entity types (e.g., Person, Organization, Location) were substituted for heads.<page local="3" global="254"/> This resulted in over 511 million predicate-argument pairs for argument types ArgO, ArgI, and Arg2. For this task, however, we chose only to use ArgI argu­ments (direct objects), which resulted in 210 mil­lion pairs, 7.65 million of which were unique. The Arg 1 argument was chosen because most of the arguments in the data are direct objects <footnote anchor="3"/>.</p><doubt alpha="36.4" length="11" tooSmall="False" monospace="0.0">'ldc2009t13</doubt><footnote label="2">http://ml . nec-labs.com/senna/</footnote><table caption="Table 1: Some of the concepts (and scores) learned from applying PageRank to WordNet hypernyms." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Artifact</p></td><td class="cell"><p>Document</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>funny _wag on</p></td><td class="cell"><p>.377</p></td><td class="cell"><p>white_paper</p></td><td class="cell"><p>.342</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>liquor</p></td><td class="cell"><p>.353</p></td><td class="cell"><p>progress-report</p></td><td class="cell"><p>.342</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>iced_tea</p></td><td class="cell"><p>.338</p></td><td class="cell"><p>screenplay</p></td><td class="cell"><p>.324</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tartan</p></td><td class="cell"><p>.325</p></td><td class="cell"><p>papyrus</p></td><td class="cell"><p>.313</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>alpaca</p></td><td class="cell"><p>.325</p></td><td class="cell"><p>pie_chart</p></td><td class="cell"><p>.308</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Event</p></td><td class="cell"><p>Location</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>rock.concert</p></td><td class="cell"><p>.382</p></td><td class="cell"><p>heliport</p></td><td class="cell"><p>.381</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>rodeo</p></td><td class="cell"><p>.369</p></td><td class="cell"><p>mukataa</p></td><td class="cell"><p>.380</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>radium_therapy</p></td><td class="cell"><p>.357</p></td><td class="cell"><p>subway-Station</p></td><td class="cell"><p>.342</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>seminar</p></td><td class="cell"><p>.347</p></td><td class="cell"><p>dairy-farm</p></td><td class="cell"><p>.326</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>pub.crawl</p></td><td class="cell"><p>.346</p></td><td class="cell"><p>gateway</p></td><td class="cell"><p>.320</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Proposition</p></td><td class="cell"><p>Sound</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>dibs</p></td><td class="cell"><p>.363</p></td><td class="cell"><p>whoosh</p></td><td class="cell"><p>.353</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>white.paper</p></td><td class="cell"><p>.322</p></td><td class="cell"><p>squish</p></td><td class="cell"><p>.353</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tall-tale</p></td><td class="cell"><p>.319</p></td><td class="cell"><p>yodel</p></td><td class="cell"><p>.339</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>commendation</p></td><td class="cell"><p>.310</p></td><td class="cell"><p>theme_song</p></td><td class="cell"><p>.320</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>field-theory</p></td><td class="cell"><p>.309</p></td><td class="cell"><p>oldie</p></td><td class="cell"><p>.312</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>The "best" predicates for a given argument are defined by a ranking based on Fisher's exact test (Fisher, 1922):</p><doubt alpha="37.1" length="35" tooSmall="False" monospace="0.0">_(a+b)\(c+d)\(a+c)\(b+d)\n\a\b\c\d\</doubt><p>where <i>a </i>is the number of times the given argument was used with the given predicate, <i>b </i>is the number of times the argument was used with a different predicate, c is the number of times the predicate was used with a different argument, <i>d </i>is the num­ber of times neither the given argument or predi­cate was used, and <i>n = a+b+c+d. </i>The top ranked (lowest <i>p)</i><i> </i>predicates for the most common argu­ments in the training data are shown in Table 2.</p></section><section number="4" title="Experiments"><p>We have conducted several experiments to test the performance of the approaches outlined in Sections 2 and 3 along with additional features commonly found in information extraction liter­ature. All experiments were conducted using the <i>$-yy</i><b><i>[muiticiass </i></b>SUpp0rt vector machine library<footnote anchor="4"/>.</p><subsection number="4.1" title="WordNet PageRank"><p>We experimented with the output of our WordNet PageRank implementation along three separate di­mensions: (1) which sense to use (since we did not incorporate a word sense disambiguation sys­tem), (2) whether to use the highest scoring semantic class or every class an argument belonged to, and (3) how to use the weight output by the al­gorithm. The results of these experiments yielded a single feature for each class that returns true if the argument is in that class, regardless of weight. This resulted in a micro-precision score of 75.6%.</p><footnote label="3">The notable exception to this, however, is arrive, where the data uses the destination argument. In the PropBank scheme (Palmer et al., 2005), this would correspond to the Arg 4, which usually signifies an end state.</footnote><footnote label="4">http://svmlight.j  oachims.org/svm_mul ticlass.html</footnote></subsection><subsection number="4.2" title="Gigaword Predicates"><p>We experimented with both (i) the number of pred­icates to use for an argument and (ii) the score threshold to use. Ultimately, the Fisher score did not prove nearly as useful as a classifier as it did as a ranker. Since the distribution of predicates for each argument varied significantly, choosing a high number of predicates would yield good re­sults for some arguments but not others. However, because of size of the training data, we were able to choose the top 5 predicates for each argument as features and still achieve a reasonable micro-precision score of 89.6%.</p></subsection><subsection number="4.3" title="Other Features"><p>Many other features common in information ex­traction are well-suited for this task. Given that SVMs can support millions of features, we chose to add many features simpler than those previously described in order to improve the final perfor­mance of the classifier. These include the lemma of the argument (both the last word's lemma and every word's lemma), the lemma of the predicate, the number of words in the argument, the casing of the argument, the part-of-speech of the argument's last word, the WordNet synset and all (recursive) hypernyms of the argument. Additionally, since the Event class is both the most common and the most often confused, we introduced two fea­tures based on annotated resources. The first fea­ture indicates the most common part-of-speech for the un-lemmatized argument in the Treebank cor­pus. This helped classify examples such as <i>think­ing </i>which was confused with a Proposition for the predicate <i>deny. </i>Second, we introduced a fea­ture that indicated if the un-lemmatized argument was considered an event in the TimeBank cor­pus (Pustejovsky et al, 2003) at least five times. This helped to distinguish events such as <i>meet­ing, </i>which was confused with a Location for the predicate <i>arrive.</i></p></subsection><subsection number="4.4" title="Ablation Test"><p>We conducted an ablation test using combina­tions of five feature sets: (1) our WordNet PageR-<page local="4" global="255"/></p><table caption="Table 2: Top ten predicates for the most common word in the train set for the six semantic classes." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>coffee</i></p></td><td class="cell"><p><i>book</i></p></td><td class="cell"><p><i>meeting</i></p></td><td class="cell"><p><i>station</i></p></td><td class="cell"><p><i>report</i></p></td><td class="cell"><p><i>voice</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>drink</p></td><td class="cell"><p>write</p></td><td class="cell"><p>hold</p></td><td class="cell"><p>own</p></td><td class="cell"><p>release</p></td><td class="cell"><p>hear</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>sip</p></td><td class="cell"><p>read</p></td><td class="cell"><p>attend</p></td><td class="cell"><p>build</p></td><td class="cell"><p>publish</p></td><td class="cell"><p>raise</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>brew</p></td><td class="cell"><p>publish</p></td><td class="cell"><p>schedule</p></td><td class="cell"><p>open</p></td><td class="cell"><p>confirm</p></td><td class="cell"><p>give</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>serve</p></td><td class="cell"><p>title</p></td><td class="cell"><p>chair</p></td><td class="cell"><p>attack</p></td><td class="cell"><p>issue</p></td><td class="cell"><p>add</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>spill</p></td><td class="cell"><p>sell</p></td><td class="cell"><p>convene</p></td><td class="cell"><p>close</p></td><td class="cell"><p>comment</p></td><td class="cell"><p>have</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>smell</p></td><td class="cell"><p>buy</p></td><td class="cell"><p>arrange</p></td><td class="cell"><p>operate</p></td><td class="cell"><p>submit</p></td><td class="cell"><p>silence</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>sell</p></td><td class="cell"><p>balance</p></td><td class="cell"><p>call</p></td><td class="cell"><p>fill</p></td><td class="cell"><p>deny</p></td><td class="cell"><p>sound</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>pour</p></td><td class="cell"><p>illustrate</p></td><td class="cell"><p>host</p></td><td class="cell"><p>shut</p></td><td class="cell"><p>file</p></td><td class="cell"><p>lend</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>buy</p></td><td class="cell"><p>research</p></td><td class="cell"><p>plan</p></td><td class="cell"><p>storm</p></td><td class="cell"><p>prepare</p></td><td class="cell"><p>crack</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>rise</p></td><td class="cell"><p>review</p></td><td class="cell"><p>make</p></td><td class="cell"><p>set</p></td><td class="cell"><p>voice</p></td><td class="cell"><p>find</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="51.6" length="31" tooSmall="False" monospace="0.0">+WNSH     +WNPR     +GWPA +EVNT</doubt><p>WORD EVNT GWPA WNPR WNSH</p><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">31.1</doubt><doubt alpha="11.4" length="44" tooSmall="False" monospace="0.0">I94.2I     I95.0I     I95.6  |     |  96.1 |</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">89.7</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">89.9</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">90.8</doubt><table caption="Table 3: Ablation test of feature sets showing micro-precision scores."></table><table caption="Table 4: Results forUTDMeton SemEval-2010 Task 7."></table><p>ank feature (WNPR), (2) our Gigaword Predicates feature (GWPA), (3) word, lemma, and part-of-speech features (WORD), (4) WordNet synset and hypernym features (WNSH), and (5) Treebank and TimeBank features (EVNT). Of these 2<footnote anchor="5"/> - 1 = 31 tests, 15 are shown in Table 3. The Giga­word Predicates (GWPA) was the best overall fea­ture, but each feature set ended up helping the fi­nal score. WordNet PageRank (WNPR) even im­proved the score when combined WordNet hyper­nym features (WNSH) despite the fact that they are heavily related. Ultimately, WordNet PageR­ank had a greater precision, while the other Word-Net features had greater recall.</p></subsection><subsection number="4.5" title="Task 7 Results"><p>Table 4 shows the official results for UTDMet on the Task 7 data. The target type was unambigu­ous given the lemmatized predicate. For classify­ing selection vs. coercion, we simply checked to see if the classified source type was the same as the target type. If this was the case, we returned selection, otherwise a coercion existed.</p></subsection></section><section number="5" title="Conclusion"><p>We have presented two approaches for determin­ing the semantic class of a predicate's argument. The two approaches capture different information and combine well to classify the "source" type in SemEval-2010 Task 7. We showed how this can be incorporated into a system to detect coercions as well as the argument's compositional history rel­ative to its predicate. In future work we plan to extend this system to more complex tasks such as when the predicate may be polysemous or unseen predicates may be encountered.</p></section><section title="Acknowledgments"><p>The authors would like to thank Bryan Rink for several insights during the course of this work.</p><table caption="Table 4: Results for UTDMet on SemEval-2010 Task 7." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Selection vs. Coercion</p></td><td class="cell"><p>Macro</p></td><td class="cell"><p>95.4</p></td><td class="cell"><p>95.7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Micro</p></td><td class="cell"><p>96.3</p></td><td class="cell"><p>96.3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source Type</p></td><td class="cell"><p>Macro</p></td><td class="cell"><p>96.5</p></td><td class="cell"><p>95.7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Micro</p></td><td class="cell"><p>96.1</p></td><td class="cell"><p>96.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Target Type</p></td><td class="cell"><p>Macro</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Micro</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Joint Type</p></td><td class="cell"><p>Macro</p></td><td class="cell"><p>85.5</p></td><td class="cell"><p>95.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Micro</p></td><td class="cell"><p>96.1</p></td><td class="cell"><p>96.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Ronan Collobert and Jason Weston. 2009. Deep Learning in Natural Language Processing. Tutorial at NIPS.</p><p>Andrea Esuli and Fabrizio Sebastiani. 2007. PageR-anking WordNet Synsets: An Application to Opin­ion Mining. In <i>Proceedings of the 45th Annual Meeting of the Association for Computational Lin­guistics, </i>pages 424-431.</p><p>Ronald A. Fisher. 1922. On the interpretation of <i>x<footnote anchor="2"/></i><i></i>from contingency tables, and the calculation of p. 85(l):87-94.</p><p>Sanda Harabagiu, George Miller, and Dan Moldovan. 1999. WordNet 2 - A Morphologically and Se-mantically Enhanced Resource. In <i>Proceedings of the SLGLEX Workshop on Standardizing Lexical Re­sources, </i>pages 1-7.</p><p>Sanda Harabagiu, Andrew Hickl, John Lehmann, and Dan Moldovan. 2005. Experiments with Inter­active Question-Answering. In <i>Proceedings of the 43rd Annual Meeting of the Association for Compu­tational Linguistics, </i>pages 205-214.</p><p>George Lakoff and Mark Johnson. 1980. <i>Metaphors We Live By. </i>University of Chicago Press.</p><p>Maria Lapata and Alex Lascarides. 2003. A Proba­bilistic Account of Logical Metonymy. <i>Computa­tional Linguistics, </i>21 (2) :261-315.</p><p>Martha Palmer, Paul Kingsbury, and Daniel Gildea.</p><p>2005. The Proposition Bank: An Annotated Cor­pus of Semantic Roles. <i>Computational Linguistics, </i>31(1):71-106.</p><p>James Pustejovsky and Anna Rumshisky. 2009. SemEval-2010 Task 7: Argument Selection and Co­ercion. In <i>Proceedings of the NAACL HLT Work­shop on Semantic Evaluations: Recent Achieve­ments and Future Directions, </i>pages 88-93.</p><p>James Pustejovsky, Patrick Hanks, Roser Sauri, An­drew See, Robert Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Ferro, and Marcia Lazo. 2003. The TIMEBANK Corpus. In <i>Proceedings of Corpus Linguistics, </i>pages 647-656.</p><p>Jan Scheffczyk, Adam Pease, and Michael Ellsworth.</p><p>2006. Linking FrameNet to the Suggested Upper Merged Ontology. In <i>Proceedings of Formal Ontol­ogy in Information Systems, </i>pages 289-300.</p></references></body></article>