Metonymy is typically defined as a figure of speech in which a speaker uses one entity to refer to an­other that is related to it (Lakoff and Johnson, 1980). The identification of metonymy becomes important for NLP tasks such as question answering (Stallard, 1993) or geographic information retrieval (Leveling and Hartrumpf, 2006).

For regular cases of metonymy for locations and organizations, Markert and Nissim have proposed a set of metonymy classes. Annotating a subset of the BNC (British National Corpus), they extracted a set of metonymic proper nouns from two categories: country names (Markert and Nissim, 2002) and or­ganization names (Nissim and Markert, 2003).

In the metonymy resolution task at SemEval-2007, the goal was to identify metonymic names in a subset of the BNC. The task consists of two subtasks for company and country names, which are further divided into classification on a coarse level (recog­nizing literal and non-literal readings), on a medium level (differentiating non-literal readings into mixed and metonymic readings), and on a fine level (iden­tifying classes of regular metonymy, such as a name referring to the population, place-for-people). The task is described in more detail by Markert and Nissim (2007).

The following tools and resources are used for the metonymy classification:

Table 3 shows results for the official submission. Compared to results from the training phase on the Mascara data (tested with the leave-one-out method), performance is considerably lower. For this data, the combined classifier achieved a consid­erably higher precision (63.9% for non-literal read­ings; 57.3% for the fine class place-for-people and even 83.3% for the rare class place-for-event).

Performance may be affected by several reasons: A number of problems were encountered while pro­cessing the data. The TreeTagger automatically to-kenizes its input and applies sentence boundary de­tection. In some cases, the sentence boundary detec­tion did not work well, returning sentences of more than 170 words. Furthermore, the tagger output had to be aligned with the test data again, as multi-word

Table 2: Excerpt from results for training the clas­sifiers on the fine location name classes (2797 in­stances, leave-one-out) for the Mascara data.

names (e.g. New York) were split into different to­kens. In addition, the tag set of the tagger differs somewhat from the official PENN tag set and in­cludes additional tags for verbs.

In earlier experiments on metonymy classifica­tion on a German corpus (Leveling and Hartrumpf, 2006), the data was nearly evenly distributed be­tween literal and metonymic readings. This seems to make a classification task easier because there is no hidden bias in the classifier (i.e. the baseline of always selecting the literal readings is about 50%).

Features are obtained by shallow NLP methods only, not making use of a parser or chunker. Thus, important syntactic or semantic information to de­cide on metonymy might be missing in the features. However, semantic features are more difficult to de­termine, because reliable automatic tools for seman­tic annotation are still missing. This is also indi­cated by the fact that the grammatical roles (com­prising syntactic features) in Mascara data are hand­annotated.

However, some linguistic phenomena are already implicitly represented by shallower features from

Table 3: Results for the coarse (908 samples: 721 literal, 187 non-literal), medium (721 literal, 167 metonymic, 20 mixed), and fine classification (721 literal, 141 place-for-people, 10 place-for-event, 1 place-for-product, 4 object-for-name, 11 othermet, 20 mixed) of location names. class P R F

FUH.location.coarse (0.798 accuracy)

FUH.location.medium (0.795 accuracy)

FUH.location.fine (0.785 accuracy)

the surface level (given enough training instances). For instance, active/passive voice may be encoded by a combination of features for main verb/modal verbs. If only a small training corpus is available, overall performance will be higher when utilizing explicit syntactic or semantic features.

Finally, the data may be too sparse for a super­vised memory-based learning approach. The iden­tification of rare classes of metonymy (e.g. place-for-event) would greatly benefit from a larger corpus covering these classes.

Evaluation results on the training data were very promising, indicating a boost of precision by com­bining classification results. In the training phase, an accuracy of 83.7% was achieved on the coarse level, compared to the majority baseline accuracy of 81.8%. For the submission for the metonymy res­olution task at SemEval-2007, accuracy is close to the majority baseline (79.4%) on the coarse (79.8%), medium (79.5%), and fine (78.5%) level.

In summary, using different context sizes for dif­ferent kinds of context and combining results of dif­ferent classifiers for metonymy resolution increases performance. The general approach would profit from combining results of more diverse classifiers, i.e. classifiers employing features extracted from the surface, syntactic, and semantic context of a location name.

The research described was in part funded by the DFG (Deutsche Forschungsgemeinschaft) in the project IRSAW (Intelligent Information Retrieval on the Basis of a Semantically Annotated Web).