Prepositional phrases (PPs) are both common and semantically varied in open English text. While the conventional view on prepositions from the com­putational linguistics community has been that they are semantically transient at best, and semantically-vacuous at worst, a robust account of the semantics of prepositions and disambiguation method can be helpful in a range of NLP tasks including machine translation, parsing (prepositional phrase attach­ment) and semantic role labelling (Durand, 1993; O'Hara and Wiebe, 2003; Ye and Baldwin, 2006a).

The SemEval 2007 preposition sense disambigua­tion task provides a common test bed for the evalua­tion of preposition sense disambiguation systems.

Our proposed method is maximum entropy based, and combines features developed in the context of preposition sense disambiguation for semantic role labelling (Ye and Baldwin, 2006a), and verb sense disambiguation (Ye and Baldwin, 2006b).

The remainder of this paper is structured as fol­lows. We first discuss the pre-processing steps used in our system (Section 2), and outline the fea­tures our preposition disambiguation method uses (Section 3) and our parameter tuning method (Sec­tion 4). We then discuss and analyse the results of our method (Section 5) and conclude the paper (Sec­tion 6).

The following list shows the pre-processing steps that our system goes through and the tools used:

Part of speech tagging SVMTool version 1.2 (Gimenez and Marquez, 2004).

Chunking An in-house chunker implemented with fnTBL, a transformation based learner (Ngai and Florian, 2001), and trained on the British Na­tional Corpus (BNC). Parsing Charniak's re-ranking parser, version Au­gust, 2006 (Charniak and Johnson, 2005).

Named entity extraction A statistical NER sys­tem described in Cohn et al. (2005).

Supersense tagging A WordNet-based super­sense tagger (Ciaramita and Altun, 2006).

Semantic role labeling ASSERT version 1.4 (Pradhanetal.,2004).

The disambiguation features used by our system can be divided into three categories: collocation fea­tures, syntactic features and semantic-role based fea­tures. We discuss each in turn below.

We used the ranking-based feature selection method from Ye and Baldwin (2006b) to select the most rele­vant feature based on our training data. This method works in two steps. Firstly, we calculated the infor­mation gain, gain ratio and Chi-squared statistics for each feature, and used these values to generate 3 sets of rankings for the features. We then summed up the individual ranks, and used the sums to create a set of final rankings for the features.

The feature selection process is based on 10-fold cross validation: we divided our training data into 10 pairs of training-test datasets; then for each fold, we extracted the top N% ranked features using our feature selection heuristic from the cv-training set (where N was set to values 5, 10, .., 100), and used these features to test the held-out test set. The best N as determined by the cross validation was then applied to the entire training data set.

Additionally, since we used a maximum entropy-based machine learning package, it was important to determine the best Gaussian smoothing parameter g for the probability distribution. The tuning of g was incorporated into the cross validation process of feature selection.

Given the possible combinations of parameter tuning, we trained the following three classifiers for the preposition sense disambiguation task:

Non-tuned Using all the original features and 10.0 for the Gaussian smoothing parameter.

Smoothing-tuned Using all the original features but automatically tuned Gaussian smoothing param­eter.

Fully-tuned Using both automatically tuned fea­tures and Gaussian smoothing parameter.

The overall precision (%) obtained by the three clas­sifiers for the fine-grained senses are as follows:

The best overall results were achieved when both the features and the Gaussian smoothing parameters were automatically tuned, achieving a 1.4% absolute precision gain over the non-tuned system. However, such parameter tuning may not always be useful: the same tuning process was found to be detrimental in a Senseval-2 verb sense disambiguation task (Ye and Baldwin, 2006b). Consistent with the findings of Ye and Baldwin (2006b), the improvement caused by the tuning of the Gaussian smoothing parame­ter is only marginal compared with the improvement caused by the tuning of the features.

We also evaluated our features based on their cate­gories and types. Collocation features performed the best among the three feature categories. Without any parameter tuning, the collocation-feature-only clas­sifier achieved an overall precision of 67.4% on the test set; the semantic-role-feature-only classifier and the syntactic-feature-only classifier achieved preci­sion of 46.9% and 50.5% respectively.

The best-performing individual features are the bag-of-words features and bag-of-synsets features.

In this paper, we have described a maximum entropy based preposition sense disambiguation system that uses a rich set of features. We have shown that this system performed well above the majority class baseline of 39.6% precision. Our analysis showed that the most important disambiguation features are collocation-based features. This indicates that the semantics of prepositions can be learnt mostly from their surrounding context, and not syntactic proper­ties or verb-preposition semantics.

The research in this paper has been supported by the Aus­tralian Research Council through Discovery Project grant num­ber DP0663879.