The last years have seen a large body of work on modelling the semantic properties of individual words, both in the form of hand-built resources like WordNet and data-driven methods like seman­tic space models. It is still much less clear how the combined meaning of phrases can be described. Semantic roles describe an important aspect of phrasal meaning by characterising the relationship between predicates and their arguments on a seman­tic level (e.g., agent, patient). They generalise over surface categories (such as subject, object) and vari­ations (such as diathesis alternations). Two frame­works for semantic roles have found wide use in the community, PropBank (Palmer et al., 2005) and FrameNet (Fillmore et al., 2003). Their corpora are used to train supervised models for semantic role labelling (SRL) of new text (Gildea and Jurafsky, 2002; Carreras and Marquez, 2005). The resulting analysis can benefit a number of applications, such

© 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.Org/licenses/by-nc-sa/3.0/). Some rights reserved.

as Information Extraction (Moschitti et al., 2003) or Question Answering (Frank et al., 2007).

A commonly encountered criticism of seman­tic roles, and arguably a major obstacle to their adoption in NLP, is their limited coverage. Since manual semantic role tagging is costly, it is hardly conceivable that gold standard annotation will ulti­mately be available for every predicate of English. In addition, the lexically specific nature of the map­ping between surface syntax and semantic roles makes it difficult to generalise from seen predicates to unseen predicates for which no training data is available. Techniques for extending the coverage of SRL therefore address an important need.

Unfortunately, pioneering work in unsupervised SRL (Swier and Stevenson, 2004; Grenager and Manning, 2006) currently either relies on a small number of semantic roles, or cannot identify equiva­lent roles across predicates. A promising alternative direction is automatic data expansion, i.e., lever­aging existing annotations to classify unseen, but similar, predicates. Lhe feasibility of this approach was demonstrated by Gordon and Swanson (2007) for syntactically similar verbs. However, their ap­proach requires at least one annotated instance of each new predicate, limiting its practicability.

In this paper, we present a pilot study on the application of automatic data expansion to event nominalisations of verbs, such as agreement for agree or destruction for destroy. While event nom­inalisations often afford the same semantic roles as verbs, and often replace them in written lan­guage (Gurevich et al., 2006), they have played a largely marginal role in annotation. PropBank has only annotated verbs. FrameNet annotates nouns, but covers far fewer nouns than verbs. The same situation holds in other languages (Erk et al., 2003).

Our fundamental intuition is that it is possible to increase the annotation coverage of event nominal-isations by data expansion from verbal instances, since the verbal and nominal predicates share a large part of the underlying argument structure. We assume that annotation is available for verbal in­stances. Then, for a given instance of a nominal-isation and its arguments, the aim is to assign se­mantic role labels to these arguments. We solve this task by constructing mappings between the argu­ments of the noun and the semantic roles realised by the verb's arguments. Crucially, unlike previous work (Liu and Ng, 2007), we do not employ a clas­sical supervised approach, and thus do not require any nominal annotations.

Structure of the paper. Sec. 2 provides back­ground on nominalisations and SRL. Sec. 3 pro­vides concrete details on our expansion-based ap­proach to SRL for nominalisations. The second part of the paper (Sec. 4-6) provides a first evaluation of different mapping strategies based on syntactic, semantic, and hybrid information. Sec. 8 concludes.

Nominalisations (or deverbal nouns) are commonly defined as nouns morphologically derived from verbs, usually by suffixation (Quirk et al., 1985). They have been classified into at least three cate­gories in the linguistic literature, event, result, and agent/patient nominalisations (Grimshaw, 1990).

Event and result nominalisations account for the bulk of deverbal nouns. The first class refers to an event/activity/process, with the nominal expressing this action (e.g. killing, destruction). Nouns in the second class describe the result or goal of an ac­tion (e.g. agreement). Many nominals have both an event and a result reading (e.g., selection can mean the process of selecting or the selected ob­ject). Choosing a single reading for an instance is often difficult; see Nunes (1993); Grimshaw (1990).

A smaller class is agent/patient nominalisations. Agent nominals are usually identified by suffixes such as -er, -or, -ant (e.g. speaker, applicant), while patient nominalisations end with -ee, -ed (e.g. em­ployee). While these nominalisations can be anal­ysed as events {the baker's bread implies that bak­ing has taken place), they more naturally refer to participants. In consequence, agent/patient nomi­nals tend to realise fewer arguments - the average in FrameNet is 1.46 arguments, compared to 1.74 for events/results. As our goal is nominal SRL, we concentrate on the event/results class. SRL for nominalisations. Compared to the wealth of studies on verbal SRL (e.g., Gildea and Juraf-sky (2002); Fleischman and Hovy (2003)), there is relatively little work that specifically addresses nominal SRL. Nouns are generally treated like verbs: the task is split into two classification steps, argument recognition (telling arguments from non-arguments) and argument labelling (labelling recog­nised arguments with a role). Nominal SRL also typically draws on feature sets that are similar to those for verbs, i.e., comprising mainly syntac­tic and lexical-semantic information (Liu and Ng, 2007; Jiang and Ng, 2006).

On the other hand, there is converging evidence that nominal SRL is somewhat more difficult than verbal SRL. Table 1 shows some results for both verbal and nominal SRL from the literature. For both PropBank and for FrameNet, we find a differ­ence of 7-8% F-Score. Note, however, that these studies use different datasets and are thus not di­rectly comparable.

In order to confirm the difference between nouns and verbs, we modelled a controlled dataset (de­scribed in detail in Sec. 4) of verbs and corre­sponding event nominalisations. We used Shal-maneser (Erk and Pado, 2006), to our knowledge the only freely available SRL system that handles nouns. SRL models were trained on verbs and nouns separately, using the same settings and fea­tures. Table 2 shows the results, averaged over 10 cross-validation (CV) folds. Accuracy was about equal in the recognition step, and 5% higher for verbs in the labelling step. We analysed these re­sults by fitting a logit mixed model. These models determine which fixed factors are responsible for differences in a response variable (here: SRL per­formance) while correcting for imbalances intro­duced by random factors (see Jaeger (2008)). We modelled the training and test set sizes and the pred­icates' parts of speech as fixed effects, and frames and CV folds as random factors.

For both argument recognition and labelling, the amount of training data turned out to be a signifi­cant factor, i.e., more data leads to higher results. While the part of speech was not systematically linked to performance for argument recognition, it was a highly significant predictor of accuracy in the labelling step: Even when training set size was taken into account, verbal arguments were still significantly easier to label (z=4.5, p<0.001).

In sum, these results lend empirical support to claims that nominal arguments are less tightly cou­pled to syntactic realisation than verbal ones (Carl­son, 1984); their interpretation is harder to capture with shallow cues.

The previous section has established two observa­tions. First, the argument structures of verbs and their event nominalisations correspond largely. Sec­ond, nominal SRL is a difficult task, even given nominal training data, which is hard to obtain.

Our proposal in this paper is to take advan­tage of the first observation to address the sec­ond. We do so by modelling SRL for event nom­inalisations as a data expansion task — i.e., us­ing existing verbal annotations to carry out SRL for novel nominal instances. In this manner, we do away completely with the need for manual annotation of nominal instances that is required for previous supervised approaches (cf. Sec. 2). Consider the following examples, given in format [constituent]grammatiCal function/SEMANTic role:

(1) a. [PeterJsubj/coGNizER laughs

[about the joke]pP.about/STiMULus b. [PeterJsubj/coGNizER laughs [at him]pp.at/STiMULus

(2) [Peter's]Pre„om-Gen/? [hearty]pre„om-Mod/? laughter [about the event]pp-about/?

The sentences with the verbal predicate laugh in (1) are labelled with semantic roles, while the NP containing the event nominalisation laughter in (2) is not. The question we face are what information from (1) can be re-used to perform argument recog­nition and labelling on (2), and how.

In this respect, there is a fundamental difference between lexical-semantic and syntactic information.

Lexical-semantic features, such as the head word, are basically independent of the predicate's part of speech. Thus, the information from (1) that Peter is a Cognizer can be used directly for the analysis of the occurrence of Peter in (2). Unfortunately, pure lexical features tend to be sparse: the head word of the last role, event, is unseen in (1), and due to its abstract nature, also difficult to classify through semantic similarity. Therefore, it is necessary to consider syntactic features as well. However, these vary substantially between verbs and nouns. When they are applicable to both parts of speech, some mileage can be apparently gained: the phrase in (2) headed by event can be classified as Stimulus because it is an about-W like (la). In contrast, no direct inferences can be drawn about prenominal genitives or modifiers which do not exist for verbs.

In the remainder of this paper, we will present experiments on different ways of combining syn­tactic and lexical-semantic information to balance precision and recall in data expansion. We address argument recognition and labelling separately, since the two tasks require different kinds of information. We assume that the frame has been determined be­forehand with word sense disambiguation methods.

The dataset for our study consists of the annotated FrameNet 1.3 examples. We obtained pairs of verbs and corresponding event/result nominalisations by intersecting the FrameNet predicate list with a list of nominalisations obtained from Celex (Baayen et al., 1995) and Nomlex (Macleod et al., 1998). We found 306 nominalisations with correspond­ing verbs in the same frame, but excluded some pairs where either the nominalisation was not of the event/result type, or no annotated FrameNet exam­ples were available for either verb or noun. The final dataset, consisting of 265 pairs exemplifying 117 frames, served for both the analysis in Section 2 and the evaluations in subsequent sections. For the eval­uations, we used the 26,479 verbal role instances (2,066 distinct role types) as training data and the 6,502 nominal role instances (993 distinct role types) as test data. The specification of the dataset can be downloaded from http : //www. coli . uni- sb. de/~pado/nom_data. html.

Argument recognition is usually modelled as a su­pervised machine learning task. Unfortunately, argument recognition - at least within predicates - re­lies heavily on syntactic features, with the grammat­ical function (or alternatively syntactic path) feature as the single most important predictor (Gildea and Jurafsky, 2002). Since we are bootstrapping from verbal instances to nominal ones, and since there is typically considerable variation between nominal and verbal subcategorisation patterns, we cannot model argument recognition as a supervised task. Instead, we follow up on an idea developed by Xue and Palmer (2004) for verbal SRL, who charac­terise the set of grammatical functions that could fill a semantic role in the first place. In our apppli-cation, we simply extract all syntactic arguments of the nominalisation, including any premodifiers. We make no attempt to distinguish between adjuncts and compulsory arguments. Fig. 1 shows an exam­ple: in the NP Peter's laughter about the joke, the noun laughter has two syntactic arguments: the PP about the joke and the premodifying NP Peter's. Both are extracted as (potential) arguments.

This method cannot identify roles that are syntac­tically non-local, i.e., those that are not in the max­imal projection of the frame evoking noun. Such roles are more common for nouns than for verbs. Example 3 shows that an "external" NP like Bill can be analysed as filling the helper role of the noun help. However, the overall proportion of non-local roles is still fairly small in our data (around 10%).

(3) [Bill] helper offered help in case of need.

Table 3 gives the argument recognition results for our rule-based system on all roles in the gold stan­dard and on the local roles alone. This simple ap­proach is surprisingly effective, achieving an over­all F-Measure of 76.89% on all roles, while on local roles the F-Measure increases to 82.83% due to the higher recall. Precision is 82.01%, as not all syn­tactic arguments do fill a role. For example, modal modifiers such as hearty in (2) rarely fill a (core) role in FrameNet. False-negative errors, which af­fect recall, are partly due to parser errors and partly to role fillers that do not correspond to constituents or that are embedded in syntactic arguments. For in­stance, in (4) the PP in this country, which fills the Place role of cause, is embedded in the PP of suf­fering in this country, which fills the role Effect. We extract only the larger PP.

[in this country] pp.in ]PP.of

Argument labelling presents a different picture from argument recognition. Here, both syntactic and lexical-semantic information contribute to suc­cess in the task. We present three model families for nominal argument labelling that take different stances with respect to this observation.

The first (naive-semantic) and the second (naive-syntactic) model families represent extreme posi­tions that attempt to re-use verbal information as directly as possible. Models from the third fam­ily, distributional models infer the role of a noun's arguments by computing the semantic similarity between nominal arguments and semantic represen­tations of the verb roles given by the role fillers' semantic heads. In the lexical-level instantiation, the mapping is established between individual noun arguments and roles. In the function-level instantia­tion, complete nominal grammatical functions are mapped onto roles.

The results of our experiments are reported in Ta­ble 4. The models are compared against two base­lines: A random baseline which randomly chooses one of the verb roles for each of the arguments of the corresponding noun; a most common baseline which assigns to each nominal argument the most frequent role of the corresponding verb - i.e. the role which has most fillers. All models with the exception of naive syn significantly outperform the random baseline, but only dist-fun and all hybrid models outperform the most common baseline.

In general, the best performing methods are the hybrid ones, with best accuracy achieved by naive syn-gen + dist-fun. Non-hybrid approaches always have lower accuracy. This validates our main hy­pothesis in this paper, namely that the combination of syntactic information with distributional seman­tics is a promising strategy.

Matching our predictions, the low accuracy of the naive syntactic model is mainly due to a lack of coverage. In fact, the model leaves 5,010 of the 6,502 gold standard noun fillers unassigned since they realise syntactic roles that are unseen for the verbs in question. A large part of these are Pre-Modifier and PP-of functions, which are central for nouns, but mostly ungrammatical for verbs. On the 1,492 fillers for which a role was assigned, the model obtains an accuracy of 67%, indicating a rea­sonably high, but not perfect, accuracy for shared grammatical functions. The remaining errors stem from two sources. First, many grammatical func­tions are ambiguous, causing wrong assignments by a "syntax-only" model. For example, PP-in can indicate both time and place for many nominal-isations.Second, a certain number of grammatical functions do not preserve their role between verb to noun (Hull and Gomez, 1996). For example, PP-to realises the Message role of the verb require but the Addressee role of the noun request.

Distributional models show in general better per­formance than the naive syntactic approach (ap-prox. +25% accuracy). They do not suffer from the coverage problem, since they assign a role to each filler. Yet, the accuracy over assigned roles is lower than for the syntactic approach (52% for dist-fun).

We conclude that in the limited cases where a pure syntactic mapping is applicable, it is far more reliable than methods which are mainly based on lexical-semantic information. The major limitation of the latter is that lexical-semantics tend to fail when roles are semantically very similar. For ex­ample, for the noun announcement, the syntactic-level distributional model wrongly builds the map­ping (Addressee, PP-by) instead of (Speaker, PP-by), because the two roles are very similar se­mantically (the computed similarities of the PP-by arguments to addressee and speaker in the semantic space are 0.94 and 0.92, respectively).

The syntactic-level distributional model outper­forms the lexical-level, suggesting that generalising the mapping at the argument level offers more sta­ble statistical evidence to find the correct role, i.e. a set of noun arguments better defines the seman­tics of the mapping than a single argument. This is mostly the case when the context vector of the argument is not a good representation because the semantic head is ambiguous, infrequent or atypical. Consider, for example, the following sentence for the noun violation:

(5) Sterne's Tristram Shandy consists of a series of violations [of literary conventions] pp.of/Norm

The syntactic-level model builds the correct map­ping (Norm, PP-of), as the role fillers of the verb violate (e.g. principle, right, treaty, law) are very similar to the noun's category fillers (e.g. conven- tion, rule, agreement, treaty, norm), causing the cen-troids of Norm and PP-of to be close in the space. The lexical-level model, however, builds the incor­rect mapping (Protagonist, convention). This happens because convention is ambiguous, and one of its senses ("a large formal assembly") is compat­ible with the Protagonist role, and happens to have a large influence on the position of the vector for convention. Unfortunately, this is not the sense in which the word is used in this sentence.

We have presented a data expansion approach to SRL for event nominalisations. Instead of relying on manually annotated nominal training data, we harness annotated data for verbs to bootstrap a se­mantic role labeller for nouns. For argument recog­nition, we use a simple rule-based approach. For argument labelling, we profit from the fact that the argument structures of event nominalisations and the corresponding verbs are typically similar. This allows us to learn a mapping between verbal roles and nominal arguments, using syntactic features, lexical-semantic similarity, or both.

We found that our rule-based approach for argu­ment recognition works fairly well. For argument labelling, our approach does not yet attain the per­formance of supervised models, but has the crucial advantage of not requiring any labelled data for nominal predicates.

We achieved the highest accuracy with a hybrid syntactic-semantic model, which indicates that both types of information need to be combined for op­timal results. A purely syntactic approach results in a high precision, but low coverage because fre­quent grammatical functions in particular cannot be trivially mapped. Backing off to semantic similarity provides additional coverage. However, semantic similarity has to be considered on the level of com­plete functions rather than individual instances to promote "uniformity" in the mappings.

In this paper, we have only considered nominal SRL by data expansion, i.e. we only applied our approach to those nominalisations for which we have annotated data for the corresponding verbs. However, even if no data is available for the corre­sponding verb, it might still be possible to bootstrap from other verbs in the same frame (assuming that the frame is known for the nominalisation) and we plan to pursue this idea in furture research. We also intend to investigate whether a joint optimisation of

the mapping constrained by additional syntactic in­formation such as subcategorisation frames leads to better results. Finally, we will verify that our meth­ods, which we have evaluated on English FrameNet data, carry over to other corpora and languages.

Acknowledgments. Our work was partly funded by the German Research Foundation DFG (grant PI 154/9-3).