In this paper we present a system for the labeling of semantic roles that produces VerbNet (Kipper et al., 2000) like annotations of free text sentences us­ing only full syntactic parses of the input sentences. The labeling process is modeled as a cascade of two distinct classification steps: (1) boundary detection (BD), in which the word sequences that encode a thematic role for a given predicate are recognized, and (2) role classification (RC), in which the type of thematic role with respect to the predicate is as­signed. After role classification, a set of simple heuristics are applied in order to ensure that only well formed annotations are output.

We designed our system on a per-predicate basis, training one boundary classifier and a battery of role classifiers for each predicate word. We clustered all the senses of the same verb together and ended up with 50 distinct boundary classifiers (one for each target predicate word) and 619 role classifiers to rec­ognize the 47 distinct role labels that appear in the training set.

The remainder of this paper is structured as fol­lows: Section 2 describes in some detail the architecture of our labeling system; Section 3 describes the features that we use to represent the classifier examples; Section 4 describes the experimental set­ting and reports the accuracy of the system on the SemEval2007 semantic role labeling closed task; fi­nally, Section 5 discusses the results and presents our conclusions.

Given a target predicate word in a natural language sentence, a SRL system is meant to correctly iden­tify all the arguments of the predicate. This problem is usually divided in two sub-tasks: (a) the detection of the boundaries (i. e. the word span) of each argu­ment and (b) the classification of the argument type, e.g. ArgO or ArgM in PropBank or Agent and Goal in FrameNet or VerbNet.

The standard approach to learn both the detection and the classification of predicate arguments is sum­marized by the following steps:

1 Given a sentence from the training-set, gener­ate a full syntactic parse-tree;

2 let P and A be the set of predicates and the set of parse-tree nodes (i.e. the potential argu­ments), respectively;

words of one argument of p, put Fp a in T+ (positive examples), otherwise put it in T-(negative examples).

For instance, in Figure 1.a, for each combination of the predicate approve with any other tree node a that does not overlap with the predicate, a classifier example Fapprove>a is generated. If a exactly covers one of the predicate arguments (in this case: "The charter", "by the EC Commission" or "on Sept. 21") it is regarded as a positive instance, otherwise it will be a negative one, e. g. FappTOve ,(NN charter).

The T + and T- sets are used to train the bound­ary classifier. To train the role multi-class classifier, T + can be reorganized as positive T+g. and nega­tive Ta"rg. examples for each argument i. In this way, an individual ONE-vs-ALL classifier for each argu­ment i can be trained. We adopted this solution, ac­cording to (Pradhan et al., 2005), since it is simple and effective. In the classification phase, given an unseen sentence, all its Fp>a are generated and clas­sified by each individual role classifier. The role la­bel associated with the maximum among the scores provided by the individual classifiers is eventually selected.

To make the annotations consistent with the un­derlying linguistic model, we employ a few simple heuristics to resolve the overlap situations that may occur, e. g. both "charter" and "the charter" in Figure 1 may be assigned a role:

• if more than two nodes are involved, i. e. a node d and two or more of its descendants n are classified as arguments, then assume that d is not an argument. This choice is justified by pre­vious studies (Moschitti et al., 2006b) showing that the accuracy of classification is higher for lower nodes;

• if only two nodes are involved, i. e. they dom­inate each other, then keep the one with the highest classification score.

We explicitly represent as attribute-value pairs the following features of each Fp a pair:

• Phrase Type, Predicate Word, Head Word, Po­sition and Voice as defined in (Gildea and Juras-

• Partial Path, No Direction Path, Head Word POS, First and Last Word/POS in Constituent and SubCategorization as proposed in (Pradhan

^ ^ approved IN The charter VBN-P approved the by DT NNP EC Commission

Fi gure 1: A sentence parse tree (a) and two example AST5™ structures relative to the predicate approve (b).

Table 1: Composition of the dataset in terms of: number of annotations (Props); number of candidate argument nodes (T); positive (T + ) and negative (T-) boundary classifier examples.

Syntactic Frame as designed in (Xue and

Palmer, 2004).

We also employ structured features derived by the full parses in an attempt to capture relevant aspects that may not be emphasized by the explicit feature representation. (Moschitti et al., 2006a) and (Mos­chitti et al., 2006b) defined several classes of struc­tured features that were successfully employed with tree kernels for the different stages of an SRL pro­cess. Figure 1 shows an example of the ASTm struc­tures that we used for both the boundary detection and the role classification stages.

In this section we discuss the setup and the results of the experiments carried out on the dataset of the SemEval2007 closed task on SRL.

In this paper we presented a system that employs tree kernels and a basic set of flat features for the classification of thematic roles.

We adopted a very simple approach that is meant to be as general and fast as possible. The issue of generality is addressed by training the bound­ary and role classifiers on a per-predicate basis and by employing tree kernel and structured features in the learning algorithm. The resulting architecture can indeed be used to learn the classification of roles of non-verbal predicates as well, and the au­tomatic feature selection triggered by the tree kernel should compensate for the lack of ad-hoc, well es­tablished explicit features for some classes of non­verbal predicates, e. g. adverbs or prepositions.

Splitting the learning problem also has the clear advantage of noticeably improving the efficiency of the classifiers, thus reducing training and classifica­tion time. On the other hand, this split results in some classifiers having too few training instances and therefore being very inaccurate. This is espe­cially true for the boundary classifiers, which con­versely need to be very accurate in order to posi­tively support the following stages of the SRL pro­cess. The solution of a monolithic boundary classi­fier that we previously employed (Moschitti et al., 2006b) is noticeably more accurate though much less efficient, especially for training. Indeed, after the SemEval2007 evaluation period was over, we ran another experiment using a monolithic boundary classifier. On the test set, we measured F1 values of 82.09 vs 79.40 and 77.17 vs 75.44 for the boundary detection and the complete SRL tasks, respectively.

Although it was provided as part of both the train­ing and test data, we chose not to use the verb sense information. This choice is motivated by our in­tention to depend on as less external resources as possible in order to be able to port our SRL system to other linguistic models and languages, for which such resources may not exist. Still, identifying the predicate sense is a key issue especially for role clas­sification, as the argument structure of a predicate is largely determined by its sense. In the near feature we plan to use larger structured features, i. e. span­ning all the potential arguments of a predicate, to improve the accuracy of our role classifiers.

The development of the SRL system was carried out at the University of Rome Tor Vergata and financed by the EU project PrestoSpace (FP6-507336).