<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="183"/><title>ILK2: Semantic Role Labeling of Catalan and Spanish using TiMBL</title><pubinfo>Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007),pages 183-186, Prague, June 2007. ©2007 Association for Computational Linguistics</pubinfo><author surname="Morante" givenname="Roser"><org  name="Tilburg University" country="The Netherlands" city="Tilburg"/></author><author surname="Busser" givenname="Bertjan"><org  name="Tilburg University" country="The Netherlands" city="Tilburg"/></author></firstpageheader><frontmatter><p><b>ILK2: Semantic Role Labelling for Catalan and Spanish using TiMBL</b></p><p><b>Roser Morante, Bertjan Busser</b></p><p>ILK, Dept. of Language and Information Sciences Tilburg University, P.O.Box 90153 NL-5000 LE Tilburg, The Netherlands (R.Morante,G.J.Busser}@uvt.nl</p></frontmatter><abstract>In this paper we present a semantic role la­beling system submitted to the task <i>Multi­level Semantic Annotation of Catalan and Spanish </i>in the context of SemEval-2007. The core of the system is a memory-based classifier that makes use of full syntactic in­formation. Building on standard features, we train two classifiers to predict separately the semantic class of the verb and the seman­tic roles. </abstract></header><body><section number="1" title="Introduction"><p>Semantic role labelling (SRL) has been addressed in the CoNLL-2004 and CoNLL-2005 Shared Tasks (Carreras and Marquez, 2004; Carreras and Marquez, 2005) for English. In the task <i>Multilevel Semantic Annotation ofCatalan and Spanish </i>of the SemEval competition 2007, the target are two differ­ent languages. The general SRL task consists of two tasks: prediction of semantic roles (SR) and predic­tion of the semantic class of the verb (SC).</p><p>The data provided in the task (Marquez et al., 2007) are sentences annotated with lemma, POS tags, syntactic information, semantic roles, and the semantic classes of the verb. A training corpus for Catalan (ca.3LB) and another for Spanish (sp.3LB) are provided. Although the setting is similar to the CoNLL-Shared Task 2005, three relevant differ­ences are that the corpora are significantly smaller, that the syntactic information is based on a manu­ally corrected treebank, which contains also syntac­tic functions (i.e. direct object, indirect object, etc.), and that the set of semantic roles is larger, especially for core arguments.</p><p>Our goal is to check whether simple individual systems could produce competitive results in both subtasks, and whether they would be robust enough when applied to two languages and to the held-out test sets provided.</p></section><section number="2" title="System description"><p>We approach the SRL task as two classification problems: prediction of SR and prediction of SC. We hypothesize that the two problems can be solved in the same way for both languages. We build two very similar systems that differ only in some of the features used, as we explain below.</p><p>The task is solved in three phases: 1) A pre­processing phase that is very similar to the sequen-tialization in (Marquez et al., 2005). We call it <i>focus selection. </i>It consists of identifying the potential can­didates to be assigned a semantic role or a semantic verb class. 2) The classification. 3) Some limited postprocessing.</p><subsection number="2.1" title="Focus selection"><p>The system starts by finding the target verb (which is marked in the corpus as such). Then, it finds the complete form of the verb (that in the corpus is tagged as verb group, infinitive, gerund, etc.) and the clause boundaries in order to look for the siblings of the verb that are under the same clause. Our as­sumption is that all siblings of the verb are potential candidates for semantic roles. The focus selection process produces two groups of focus tokens: on the one hand, the verbs and, on the other, the siblings of the verbs.<page local="2" global="184"/> These tokens will be the instances in each training set. Table 1 shows the number of training and test instances for each subtask.</p><p>Table 1: Number of instances per corpus for each task ('Ca' stands for Catalan, 'Sp' stands for Spanish).</p></subsection><subsection number="2.2" title="Classification"><p>In both systems we approach the classification task in one step, predicting directly the SR and the SC class. This means that in the SR task we do not perform a previous classification to select the tokens that might be assigned a role. We assume that all verbs belong to a class. As for the SR, we assume that most siblings of the verb will have a class, ex­cept for those that have syntactic functions AO, ET, MOD, NEG, IMPERS, PASS, and VOC. The sib­lings that do not have a semantic role are assigned the NONE tag. Because the corpus is small and be­cause the amount of instances with a NONE class is proportionally low, we do not consider it necessary to filter these cases.</p><p>Regarding the <b>learning algorithm, </b>we use the IB1 classifier as implemented in TiMBL (version 5.1) (Daelemans et al., 2004), a supervised induc­tive algorithm for learning classification tasks based on the k nearest neighbor (k-nn) algorithm. In IB1, similarity is defined by a feature-level distance met­ric between a test instance and a memorized training instance. The metric combines a per-feature value-based distance metric with global feature weights that account for relative differences in importance of the features.</p><p>The TiMBL parameters used in the systems are the IB 1 algorithm, the Jeffrey Divergence as feature metric, MVDM threshold at level 1, weighting us­ing GainRatio, k=11, and weighting neighbors as function of their Inverse Linear Distance (for details we refer the reader to the TiMBL reference guide (Daelemans et al., 2004)).</p><p>As for the <b>features, </b>we started by using the same feature set for both classifiers and then, after some experimentation, we decided to use slightly different feature sets for the two sub-tasks. Most of the features we designed are features that have become standard for the SRL task (Gildea and Jurafsky, 2002; Xue and Palmer, 2004; Carreras and Marquez, 2004; Carreras and Marquez, 2005). In our system, the features relate to the verb, the verb siblings, what we take to be the content word of the siblings, the clause, and the relation verb-arguments. Addition­ally, we added lexical features extracted from the verb lexicon provided for the task, and from Word­Net.</p><p>After experimenting with 323 features, we se­lected 98 for the SR task and 77 for the SC subclass. In order to select the features, we started with a basic system, the results of which were used as a baseline. Every new feature that was added to the basic system was evaluated in terms of average accuracy in 10­fold cross-validation experiments; if it improved the performance on held-out data, it was added to the se­lection. One problem with this hill-climbing method is that the selection of features is determined by the order in which the features have been introduced. We also performed experiments applying the feature selection process reported in (Tjong Kim Sang et al., 2005), a bi-directional hill climbing process. How­ever, experiments with this advanced method did not produce a better selection of features.</p><p>The features for the SR prediction subtask are the following:</p><p>• Features on the verb (6). They are shared by all the instances that represent phrases belonging to the same clause:</p><p><b>VForm; VLemma; VCau: </b>binary feature that indicate if the verb is in a causative construction with <i>hacer, fer </i>or if the main verb is <i>causar; </i><b>VPron, VImp, VPass: </b>binary features that indi­cate if the verb is pronominal, impersonal, and in passive form respectively.</p><p>• Features on the sibling in focus (12):</p><p><b>SibSynCat: </b>syntactic category; <b>SibSynFunc: </b>syntactic function; <b>SibPrep: </b>preposition; <b>SibLemW1, SibPOSW1, SibLemW2, SibPOSW2, SibLemW3, SibPOSW3: </b>lemma and POS tag of the first, second and third words of the sibling; <b>SibRelPos: </b>position of the sibling in relation to the verb (PRE or POST); <b>Sib+1RelPos: </b>position of the sibling next to the cur­rent phrase in relation to the verb (PRE or POST); <b>SibAbsPos: </b>absolute position of the sibling in the clause.</p><p>• Features that describe the properties of the content word (CW) of the focus sibling (13): in the case of prepositional phrases the CW is the head of the first noun phrase; in cases of coordination, we only take the first element of the coordination.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Training 3LB</p></td><td class="cell"><p>Test 3LB</p></td><td class="cell"><p>Test CESS</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Ca.</p></td><td class="cell"><p>Sp.</p></td><td class="cell"><p>Ca.</p></td><td class="cell"><p>Sp.</p></td><td class="cell"><p>Ca.</p></td><td class="cell"><p>Sp.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SR</p><p>SC</p></td><td class="cell"><p>23202 8932</p></td><td class="cell"><p>24668 9707</p></td><td class="cell"><p>1335 510</p></td><td class="cell"><p>1451 615</p></td><td class="cell"><p>1241 463</p></td><td class="cell"><p>1186 465</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="3" global="185"/><p><b>CWord; CWLemma; CWPOS: </b>we take only the first char­acter of the POS tags provided; <b>CWPOSType: </b>the type of POS, second character of the POS tags provided; <b>CWGender; CWne: </b>binary feature that indicates if the CW is a named en­tity; <b>CWtmp, CWloc: </b>binary features that indicate if the CW is a temporal or a locative adverb respectively; <b>CW+2POS, CW+3POS: </b>POS of the second and third words after CW.</p><p><b>CWwnscl, CWwnsc2, CWwnsc3: </b>additionally, if the CW is a noun, we extract information from WordNet (Fellbaum, 1998) about the first, second, and third more frequent seman­tic classes of the CW in WordNet. We cannot decide on a sin­gle one because the corpus is not disambiguated. The seman­tic class corresponds to the lexicographer files in WN3.0. For nouns there are 25 file numbers.</p><doubt alpha="63.3" length="30" tooSmall="False" monospace="0.0">• Features on the clause (24):</doubt><p><b>CCtot: </b>total number of siblings with function CC (cir-cumstancial complement); <b>SUJRelPos, CAGRelPos, CDRel-Pos, CIRelPos, ATRRelPos, CPREDRelPos, CREGRelPos: </b>relative positions of siblings with functions SUJ, CAG, CD, CI,ATR, CPRED, and CREG in relation to verb (PRE or POST); <b>SEsib: </b>binary feature that indicates if the clause con­tains a verbal se; <b>SIBtot: </b>total number of verb siblings in the clause; <b>SynFuncSib8, SynCatSib8, PrepSib8,W1Sib8, W2Sib8, W3Sib8, W4Sib8, SynFuncSib9, SynCatSib9, PrepSib9, W1Sib9, W2Sib9, W3Sib9, W4Sib9: </b>syntactic function, syntactic category, preposition, and first to fourth word of siblings 8 and 9.</p><p>• Features extracted from the lexicon of verbal frames (43) that the task organizers provided. We access the lexicon to check if it is possible for a verb to have a certain semantic role. We check it for all semantic role classes, except for ArgX-Ag, ArgX-Cau, ArgX-Pat, ArgX-Tem because they proved not to be informative. The features are binary.</p><p>For the SC prediction task the features are similar, but not exactly the same. Both systems contain some features about all candidate arguments. We point out the differences:</p><p>• Features that are in the SR system and that are not in the SC system:</p><p>Verb form <b>(VForm), </b>verb lemma <b>(VLemma), </b>absolute po­sition of the sibling in the clause <b>(SibAbsPos), </b>function of the sibling <b>(SibSynFunc), </b>preposition of the sibling <b>(SibPrep), </b>POS tag of the second and third words after CW <b>(CW+2POS, CW+3POS), </b>information about the WN classes of the CW <b>(CWwnsc1, CWwnsc2, CWwnsc3), </b>feature about the CW be­ing a named entity <b>(CWne, SIBtot), </b>syntactic function, syn­tactic category, preposition and first to fourth word of sib­lings 8 and 9 <b>(SynFuncSib8, SynCatSib8, PrepSib8,W1Sib8, W2Sib8, W3Sib8, W4Sib8, SynFuncSib9, SynCatSib9, PrepSib9, W1Sib9, W2Sib9, W3Sib9, W4Sib9).</b></p><p>• Features that are only in the SC system:</p><p><b>AllCats: </b>vector of the syntactic categories of the siblings in the order that they appear in the clause; <b>AllFuncs: </b>vector of the functions of the siblings in the order that they appear; <b>AllFuncs-Bin </b>vector with eight binary values that represent if a sibling with that function is present or not; <b>Sib+1Prep, Sib+2Prep: </b>prepositions of the two siblings after the verb.</p></subsection><subsection number="2.3" title="Postprocessing"><p>As for the <b>postprocessing phase, </b>it consists of six simple rules to correct some basic errors in predict­ing some types of ArgM arguments. It only applies to the SR task. The rules are the following ones:</p><p>1. If prediction = ArgM-LOC, ArgM-MNR or ArgM-ADV, and either <b>{SibPrep </b>= 'durante' or 'durant'}, or <b>{SibSynCat </b>= sn and one of the WN semantic classes = 28}, then prediction =</p><p>ArgM-TMP.</p></subsection></section><section number="2." title="If prediction = ArgM-LOC, ArgM-MNR or ArgM-ADV,"><p>and <b>CWLemma </b>is a temporal adverb, then prediction = ArgM-</p><p>TMP.</p><p>3. If prediction = ArgM-TMP and one of the WN classes = 15, then prediction = ArgM-LOC.</p><p>4. If prediction = ArgM-TMP, ArgM-MNR or ArgM-ADV, and <b>CWLemma </b>= locative adverb, then prediction = ArgM-LOC.</p><p>5. If prediction = ArgM-TMP or ArgM-ADV, and <b>CWwnsc1 </b>= 15, and <b>SibPrep </b>= 'en' or 'desde' or 'hacia' or 'a' or 'des_de' or 'cap_a', then prediction = ArgM-LOC.</p><p>6. If prediction = ArgM-ADV and <b>CWLemma </b>= causal con­junction, then prediction = ArgM-CAU.</p><p>We are aware of the fact that these are very simple rules and that more elaborate postprocessing tech­niques can be applied, like the ones used in (Tjong Kim Sang et al., 2005) in order to make sure that the same role was not predicted more than once in the same clause.</p><p>SRL TASK <u>Perf.Props</u> <u>| Precision |</u> <u>Recall</u> <u>| F</u><u>ß=1</u></p><doubt alpha="23.7" length="38" tooSmall="False" monospace="0.0">Overall ca -      86.44% 84.08 % 85.24</doubt><doubt alpha="23.7" length="38" tooSmall="False" monospace="0.0">Overall sp -      84.30% 83.78 % 84.04</doubt><doubt alpha="25.6" length="39" tooSmall="False" monospace="0.0">Overall SRL -      85.32% 83.93 % 84.62</doubt><p>Table 2: Overall results in the SR (above), SC (middle), and general SRL tasks ('Perf.Props': perfect propositions; 'ca': Catalan; 'sp': Spanish).</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SR TASK</p></td><td class="cell"><p>Perf.Props</p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>Fß=i</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test ca.3LB</p></td><td class="cell"><p>73.35%</p></td><td class="cell"><p>86.59%</p></td><td class="cell"><p>85.91%</p></td><td class="cell"><p>86.25</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test ca.CESS</p></td><td class="cell"><p>60.55%</p></td><td class="cell"><p>82.60%</p></td><td class="cell"><p>78.03%</p></td><td class="cell"><p>80.25</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall ca</p></td><td class="cell"><p>67.24%</p></td><td class="cell"><p>84.72%</p></td><td class="cell"><p>82.12%</p></td><td class="cell"><p>83.40</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test sp.3LB</p></td><td class="cell"><p>68.07%</p></td><td class="cell"><p>83.05%</p></td><td class="cell"><p>82.54%</p></td><td class="cell"><p>82.80</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test sp.CESS</p></td><td class="cell"><p>73.76%</p></td><td class="cell"><p>85.88%</p></td><td class="cell"><p>85.80%</p></td><td class="cell"><p>85.84</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall sp</p></td><td class="cell"><p>70.52%</p></td><td class="cell"><p>84.30%</p></td><td class="cell"><p>83.98%</p></td><td class="cell"><p>84.14</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall SR</p></td><td class="cell"><p>68.96%</p></td><td class="cell"><p>84.50%</p></td><td class="cell"><p>83.07%</p></td><td class="cell"><p>83.78</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SC TASK</p></td><td class="cell"><p>Perf.Props</p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>Fß=i</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test ca.3LB</p></td><td class="cell"><p>90.86%</p></td><td class="cell"><p>90.30%</p></td><td class="cell"><p>88.72%</p></td><td class="cell"><p>89.50</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test ca.CESS</p></td><td class="cell"><p>90.41%</p></td><td class="cell"><p>90.20%</p></td><td class="cell"><p>88.27%</p></td><td class="cell"><p>89.22</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall ca</p></td><td class="cell"><p>90.64%</p></td><td class="cell"><p>90.25%</p></td><td class="cell"><p>88.50%</p></td><td class="cell"><p>89.37</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test sp.3LB</p></td><td class="cell"><p>84.12%</p></td><td class="cell"><p>80.00%</p></td><td class="cell"><p>78.44%</p></td><td class="cell"><p>79.21</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Test sp.CESS</p></td><td class="cell"><p>90.54%</p></td><td class="cell"><p>89.89%</p></td><td class="cell"><p>89.89%</p></td><td class="cell"><p>89.89</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall sp</p></td><td class="cell"><p>86.88%</p></td><td class="cell"><p>84.30%</p></td><td class="cell"><p>83.36%</p></td><td class="cell"><p>83.83</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall SC</p></td><td class="cell"><p>88.67%</p></td><td class="cell"><p>87.12%</p></td><td class="cell"><p>85.81%</p></td><td class="cell"><p>86.46</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="4" global="186"/></section><section number="3" title="Results"></section><section number="4" title="Conclusions"><p>The overall official results of the system are shown in Table 2. The SC system performs better (over­all Fi = 86.46) than the SR system (overall Fi = 83.78). In global, the systems perform better for Catalan (overall F1 = 85.24) than for Spanish (over­all F1 = 84.04), although the SC system performs better for Catalan (89.37 vs. 86.46), and the SR sys­tem performs better for Spanish (84.14 vs 83.40).</p><p>Striking results are that the SR system gets signif­icantly better results with the held-out test for Span­ish, and that both of the complete SRL systems get significantly better results with the held-out test for Spanish. This might be due to differences in the pro­cess of gathering and annotation of the corpus.</p><p>Table 3: Detailed results on the Spanish CESS-ECE test cor­pus for the SR subtask. F: frequency of the semantic roles in the training corpus, without counting V.</p><p>Table 3 shows detailed results on the Spanish CESS-ECE corpus for the SR task. Low scores are generally related to low frequency of the SR in the training corpus, and high scores are related to high frequency or to overt marking of the SR.</p><p>We have presented two memory-based SRL systems that make use of full syntactic information and ap­proach the tasks in three steps. Results show that rather simple individual systems can produce com­petitive results in both tasks, and that they are ro­bust enough to be applied to two languages and to the held-out test sets provided. Improvements of the systems would consist in improving the focus selec­tion step, and applying more elaborate techniques for feature selection and postprocessing.</p></section><section title="Acknowledgements"><p>This research has been funded by the postdoctoral grant EX2005-1145 awarded by the Ministerio de Educacion y Cien-cia of Spain to the project <i>Técnicas semiautomaticaspara el eti-quetado de roles semanticos en corpus del espaflol. </i>We would like to thank Martin Reynaert, Caroline Sporleder, Antal van den Bosch, and the anonymous reviewers for their comments and suggestions.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SP-CESS</p></td><td class="cell"><p>F</p></td><td class="cell"><p>Precision</p></td><td class="cell"><p>Recall</p></td><td class="cell"><p>Fß=i</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overall</p></td><td class="cell"><p></p></td><td class="cell"><p>85.88%</p></td><td class="cell"><p>85.80%</p></td><td class="cell"><p>85.84</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgO-AGT</p></td><td class="cell"><p>16.19%</p></td><td class="cell"><p>92.83%</p></td><td class="cell"><p>92.41%</p></td><td class="cell"><p>92.62</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgO-CAU</p></td><td class="cell"><p>1.23%</p></td><td class="cell"><p>100%</p></td><td class="cell"><p>50%</p></td><td class="cell"><p>66.67</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Argl</p></td><td class="cell"><p>1.79%</p></td><td class="cell"><p>88.46%</p></td><td class="cell"><p>82.14%</p></td><td class="cell"><p>85.19</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Argl-LOC</p></td><td class="cell"><p>0.11%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Argl-PAT</p></td><td class="cell"><p>20.09%</p></td><td class="cell"><p>93.82%</p></td><td class="cell"><p>94.19%</p></td><td class="cell"><p>94.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Argl-TEM</p></td><td class="cell"><p>14.08%</p></td><td class="cell"><p>86.54%</p></td><td class="cell"><p>91.84%</p></td><td class="cell"><p>89.11</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2</p></td><td class="cell"><p>2.05%</p></td><td class="cell"><p>68.00%</p></td><td class="cell"><p>77.27%</p></td><td class="cell"><p>72.34</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-ATR</p></td><td class="cell"><p>9.88%</p></td><td class="cell"><p>91.67%</p></td><td class="cell"><p>90.41%</p></td><td class="cell"><p>91.03</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-BEN</p></td><td class="cell"><p>2.40%</p></td><td class="cell"><p>96.30%</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>98.11</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-EFI</p></td><td class="cell"><p>0.19%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-EXT</p></td><td class="cell"><p>0.19%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-LOC</p></td><td class="cell"><p>1.13%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg2-PAT</p></td><td class="cell"><p>0.01%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg3-ATR</p></td><td class="cell"><p>0.05%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg3-BEN</p></td><td class="cell"><p>0.16%</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>100.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg3-EIN</p></td><td class="cell"><p>0.08%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg3-FIN</p></td><td class="cell"><p>0.04%</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>33.33%</p></td><td class="cell"><p>50.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg3-ORI</p></td><td class="cell"><p>0.29%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00%</p></td><td class="cell"><p>0.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Arg4-DES</p></td><td class="cell"><p>0.60%</p></td><td class="cell"><p>83.33%</p></td><td class="cell"><p>83.33%</p></td><td class="cell"><p>83.33</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgL</p></td><td class="cell"><p>0.71%</p></td><td class="cell"><p>16.67%</p></td><td class="cell"><p>20.00%</p></td><td class="cell"><p>18.18</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-ADV</p></td><td class="cell"><p>10.67%</p></td><td class="cell"><p>68.12%</p></td><td class="cell"><p>68.12%</p></td><td class="cell"><p>68.12</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-CAU</p></td><td class="cell"><p>1.50%</p></td><td class="cell"><p>55.56%</p></td><td class="cell"><p>45.45%</p></td><td class="cell"><p>50.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-FIN</p></td><td class="cell"><p>1.30%</p></td><td class="cell"><p>64.71%</p></td><td class="cell"><p>84.62%</p></td><td class="cell"><p>73.33</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-LOC</p></td><td class="cell"><p>4.94%</p></td><td class="cell"><p>78.21%</p></td><td class="cell"><p>77.22%</p></td><td class="cell"><p>77.71</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-MNR</p></td><td class="cell"><p>2.28%</p></td><td class="cell"><p>36.36%</p></td><td class="cell"><p>57.14%</p></td><td class="cell"><p>44.44</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ArgM-TMP</p></td><td class="cell"><p>7.19%</p></td><td class="cell"><p>88.75%</p></td><td class="cell"><p>81.61%</p></td><td class="cell"><p>85.03</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>V</p></td><td class="cell"><p>-</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>100.00%</p></td><td class="cell"><p>100.00</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>X. Carreras and Ll. Màrquez. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In <i>Pro­ceedings ofCoNLL-2004, </i>Boston MA, USA.</p><p>X. Carreras and Ll. Maàrquez. 2005. Introduction to the CoNLL-2005 shared task: Semantic role labeling. In <i>Pro­ceedings of CoNLL-2005, </i>Ann Arbor, Michigan, June.</p><p>W. Daelemans, J. Zavrel, K. Van der Sloot, and A. Van den Bosch. 2004. TiMBL: Tilburg memory based learner, ver­sion 5.1, reference guide. Technical Report Series 04-02, ILK, Tilburg, The Netherlands.</p><p>Christiane Fellbaum, editor. 1998. <i>WordNet: An Electronic Database. </i>MIT Press, Cambridge, MA.</p><p>D. Gildea and D. Jurafsky. 2002. Automatic labeling of seman­tic roles. <i>Computational Linguistics, </i>28(3):245-288.</p><p>LL. Màrquez, P. Comas, J. Gimenez, and N. Català. 2005. Se­mantic role labeling as sequential tagging. In <i>Proceedings of CoNLL-2005, </i>Ann Arbor, Michigan.</p><p>Ll. Màrquez, M.A. Marti, M. Taule, and L. Villarejo. 2007. SemEval-2007 Task 09: Multilevel semantic annotation of catalan and spanish. In <i>Proceedings of SemEval-2007, the 4th Workshop on Semantic Evaluations, </i>Prague, Czech Re­public.</p><p>E. Tjong Kim Sang, S. Canisius, A. van den Bosch, and T. Bogers. 2005. Applying spelling error correction tech­niques for improving semantic role labelling. In <i>Proceed­ings ofCoNLL-2005, </i>pages 229-232, Ann Arbor, Michigan.</p><p>N. Xue and M. Palmer. 2004. Calibrating features for semantic role labeling. In <i>Proceedings of2004 Conference on Empir­ical Methods in Natural Language Processing, </i>Barcelona, Spain.</p></references></body></article>