SemEval 2010 Task 9, "Noun Compound Inter­pretation Using Paraphrasing Verbs", requires sys­tems to rank paraphrases of noun compounds according to which paraphrases were most fre­quently produced for each compound by human annotators (Butnariu et al., 2010). This paper de­scribes a system which ranks a paraphrase for a given compound by computing the probability of the paraphrase occurring given that we have previ­ously observed that paraphrase co-occurring with other paraphrases in the candidate paraphrase list. These co-occurrence statistics can be built using either the compounds from the test set or the train­ing set, with no significant difference in results.

The model is informed by two observations: people tend to use general, semantically light para­phrases more often than detailed, semantically heavy ones, and most paraphrases provided for a specific compound indicate the same interpreta­tion of that compound, varying mainly according to level of semantic detail.

Given these two properties of the data, the ob­jective of our system was to test the theory that conditional probabilities can be used to estimate a sub-typing or Is-A relation between paraphrases. No information about the compounds was used, nor were the frequencies provided in the training set used.

Most research on the disambiguation of noun com­pounds involves automatically categorizing the compound into one of a pre-defined list of seman­tic relations. Paraphrasing compounds is an alter­native approach to the disambiguation task which has been explored by (Lauer, 1995) and (Nakov, 2008). Paraphrases of semantic relations may be verbs, prepositions, or "prepositional verbs" like found in and caused by. (Lauer, 1995) catego­rized compounds using only prepositions. (Nakov, 2008) and the current task use only verbs and prepositional verbs, however, many of the para­phrases in the task data are effectively just prepo­sitions with a copula, e.g. be in, be for, be of.

The paraphrasing approach may be easier to integrate into applications such as translation, query-expansion and question-answering — its output is a set of natural language phrases rather than an abstract relation category. Also, most sets of pre-defined semantic relations have only one or maybe two levels of granularity. This can often lead to semantically converse relations falling under the same abstract category, for ex­ample a headache tablet is a tablet for prevent­ing headaches, while headache weather is weather that induces headaches — but both compounds would be assigned the same relation (perhaps in­strumental or causal) in many taxonomies of se­mantic relations. Paraphrases of compounds using verbs or verb-preposition combinations can pro­vide as much or as little detail as is required to adequately disambiguate the compound.

The first step in our model is to generate a condi­tional probability table by going over all the com­pounds in the data and calculating the probabil­ity of each paraphrase occurring given that we ob­served another given paraphrase co-occurring for the same compound. We compute the conditional probability of every paraphrase with all other para­phrases individually. We could use either the train­ing or the test set to collect these co-occurrence statistics, as the frequencies with which the para­phrases are ranked are not used — we simply note how many times each paraphrase co-occurred as a possible paraphrase for the same compound with each other paraphrase. For the submitted system we used the test data, but subsequently we confirmed that using only the training data for this step is not detrimental to the system's performance.

For each paraphrase in the data, the conditional probability of that paraphrase is computed with re­spect to all other paraphrases in the data. For any two paraphrases B and A:

As described in the previous section, we antic­ipate that more general, less specific paraphrases will be produced more often than their more de­tailed sub-senses. Therefore, we score each para­phrase by summing its conditional probability with each other paraphrase provided for the same compound.

For a list of paraphrases A provided for a given compound, we score a paraphrase b in that list by summing its conditional probability individually with every other paraphrase in the list.

This gives the more general, broad coverage, paraphrases a higher score, and also has a cluster­ing effect whereby paraphrases that have not co-occurred with the other paraphrases in the list very often for other compounds are given a lower score — they are unusual in the context of this para­phrase list.

Table 1 shows the results of the top 3 systems in the task. Our system achieved the second high­est correlation according to the official evaluation measure, Spearman's rank correlation coefficient. Results were also provided using Pearson's corre­lation coefficient and the cosine of the vector of scores for the gold standard and submitted pre­dictions. Our system performed best using the cosine measure, which measures how closely the predicted scores match the gold standard frequen­cies, rather than the rank correlation. This could be helpful as the scores provide a scale of accept­ability.

As mentioned in the system description, we collected the co-occurrence statistics for our sub­mitted prediction from the test set of paraphrases alone. Since our model does not use the frequen­cies provided in the training set, we chose to use the test set as it was larger and had more annota­tors. This could be perceived as an unfair use of the test data, as we are using all of the test com­pounds and their paraphrases to calculate the po­sition of a given paraphrase relative to other para­phrases.

This is a kind of clustering which would not be possible if only a few test cases were provided. To check that our system did not need to collect co­occurrence probabilities on exactly the same data as it made predictions on, we submitted a second set of predictions for the test based on the proba­bilities from the training compounds alone.

These predictions actually achieved a slightly better score for the official evaluation measure, with a Spearman rho of 0.444, and a cosine of 0.631. This suggests that the model does not need to collect co-occurrence statistics from the same compounds as it makes predictions on, as long as sufficient data is available.

We have described a system which uses a simple statistical method, conditional probability, to es­timate a sub-typing relationship between possible paraphrases of noun compounds. From a list of candidate paraphrases for each noun compound, those which were judged by this method to be good "parent senses" of other paraphrases in the list were scored highly in the rankings.

The system does require a large dataset of com­pounds with associated plausible paraphrases, but it does not require a training set of human pro­vided rankings and does not use any information about the noun compound itself, aside from the list of plausible paraphrases that were provided by the human annotators.

Given the simplicity of our model and its per­formance compared to other systems which used more intensive approaches, we believe that our ini­tial observations on the data are valid: people tend to produce general, semantically light paraphrases more often than specific or detailed paraphrases, and most of the paraphrases provided for a given compound indicate a similar interpretation, vary­ing instead mainly in level of semantic weight or detail.

We have also shown that conditional probabil­ity is an effective way to compute the sub-typing relation between paraphrases.

Acknowledgement

This research was supported by a grant under the FP6 NEST Programme of the European Commis­sion (ANALOGY: Humans the Analogy-Making Species: STREP Contr. No 029088).