We create a word-category co-occurrence matrix (WCCM) having English word types wen as one di­mension and English thesaurus categories cen as an­other. We used the Macquarie Thesaurus (Bernard, 1986) both as a very coarse-grained sense inventory and a source of words that together represent each category (concept). The WCCM is populated with co-occurrence counts from a large English corpus (we used the British National Corpus (BNC)). A par­ticular cell mij, corresponding to word wen and con­cept cen, is populated with the number of times wenco-occurs with any word that has cejn as one of its senses (i.e., wen co-occurs with any word listed un­der concept cejn in the thesaurus).

A particular cell mij-, corresponding to word wenand concept cejn, is populated with the number of times wien co-occurs with any word that has cejnas one of its senses (i.e., wien co-occurs with any word listed under concept cejn in the thesaurus). This matrix, created after a first pass of the corpus, is the base word-category co-occurrence matrix (base WCCM) and it captures strong associations between a sense and co-occurring words (see dis­cussion of the general principle in Resnik (1998)). From the base WCCM we can determine the num­ber of times a word w and concept c co-occur, the number of times w co-occurs with any concept, and the number of times c co-occurs with any word. A statistic such as PMI can then give the strength of association between w and c. This is similar to how Yarowsky (1992) identifies words that are indicative of a particular sense of the target word.

Words that occur close to a target word tend to be good indicators of its intended sense. Therefore, we make a second pass of the corpus, using the base WCCM to roughly disambiguate the words in it. For each word, the strength of association of each of the words in its context (±5 words) with each of its senses is summed. The sense that has the highest cu­mulative association is chosen as the intended sense. A new bootstrapped WCCM is created such that each cell mi j, corresponding to word wien and con­cept cejn, is populated with the number of times wienco-occurs with any word used in sense cejn.

Mohammad and Hirst (2006a) used the DPCs created from the bootstrapped WCCM to attain near-upper-bound results in the task of determin­ing word sense dominance. Unlike the McCarthy et al. (2004) dominance system, this approach can be applied to much smaller target texts (a few hundred sentences) without the need for a large similarly-sense-distributed text. Mohammad and Hirst (2006b) used the DPC-based monolingual dis­tributional measures of concept-distance to rank word pairs by their semantic similarity and to correct real-word spelling errors, attaining markedly better results than monolingual distributional measures of word-distance. In the spelling correction task, the

celebrity f celestial body mode of state of the

conduct atmosphere t practical expose to j lesson elements

weather _*

Figure 1 : The cross-lingual candidate senses of Chi­nese words M 4 and JAit.

distributional concept-distance measures performed better than all WordNet-based measures as well, ex­cept for the Jiang and Conrath (1997) measure.

Given a Chinese word wch in context, we use a Chinese-English bilingual lexicon to determine its different possible English translations. Each En­glish translation wen may have one or more possi­ble coarse senses, as listed in an English thesaurus. These English thesaurus concepts (cen) will be re­ferred to as cross-lingual candidate senses of the Chinese word wch Figure 1 depicts examples.

We create a cross-lingual word-category co­occurrence matrix (CL-WCCM) with Chinese word types wch as one dimension and English thesaurus concepts cen as another.

The matrix is populated with co-occurrence counts from a large Chinese corpus; we used a collection of LDC-distributed corpora—Chinese Treebank En­glish Parallel Corpus, FBIS data, Xinhua Chinese-English Parallel News Text Version 1.0 beta 2, Chi­nese English News Magazine Parallel Text, Chinese

News Translation Text Part 1, and Hong Kong Paral­lel Text. A particular cell mij, corresponding to word wch and concept cj, is populated with the number of times the Chinese word wch co-occurs with any Chinese word having cen as one of its cross-lingual candidate senses. For example, the cell for X'S. ('space') and 'celestial body' will have the sum of the number of times X^. co-occurs with XW, □ , XW , M, It4, and so on (see Figure 2). We used the Macquarie Thesaurus (Bernard, 1986) (about 98,000 words). The possible Chinese translations of an English word were taken from the Chinese-English Translation Lexicon version 3.0 (Huang and Graff, 2002) (about 54,000 entries).

This base word-category co-occurrence matrix (base WCCM), created after a first pass of the cor­pus, captures strong associations between a cate­gory (concept) and co-occurring words. For ex­ample, even though we increment counts for both AS-'celestial body' and X'£- celebrity' for a par­ticular instance where XS. co-occurs with 14, X*?. will co-occur with a number of words such as Xfà, X\>U , and 0 that each have the sense of ce­lestial body in common (see Figure 2), whereas all their other senses are likely different and distributed across the set of concepts. Therefore, the co­occurrence count of X$. and 'celestial body' will be relatively higher than that of X'Ë and 'celebrity'.

As in the monolingual case, a second pass of the corpus is made to disambiguate the (Chinese) words in it. For each word, the strength of associ­ation of each of the words in its context (±5 words) with each of its cross-lingual candidate senses is summed. The sense that has the highest cumula­tive association with co-occurring words is chosen as the intended sense. A new bootstrapped WCCM is created by populating each cell mij, correspond­ing to word wch and concept cen, with the number of times the Chinese word wch co-occurs with any Chinese word used in cross-lingual sense ce". A statistic such as PMI is then applied to these counts to deter­mine the strengths of association between a target concept and co-occurring words, giving the distri­butional profile of the concept.

Mohammad et al. (2007) combined German text with an English thesaurus using a German-English bilingual lexicon to create German-English DPCs. These DPCs were used to determine semantic dis­tance between German words, showing that state-of-the-art accuracies for one language can be achieved using a knowledge source (thesaurus) from another.

Given that a published thesaurus has about 1000 categories and the size of the vocabulary N is at much smaller matrices (about 1000 x N) than the tra­ditional word-word co-occurrence matrix (N x N). Therefore the WCCMs are relatively inexpensive both in terms of memory and computation.

The naïve Bayes classifier has the following formula to determine the intended sense cnb:

cj eC wieW

where C is the set of possible senses (as listed in the Macquarie Thesaurus) and W is the set of words that co-occur with the target (we used a window of ±5 words).

Traditionally, prior probabilities of the senses (P(cj)) and the conditional probabilities in the like­lihood (nwieWP(wi\cJ-)) are determined by sim­ple counts in sense-annotated data. We approx­imate these probabilities using counts from the word-category co-occurrence matrix (monolingual or cross-lingual), thereby obviating the need for manually-annotated data.

For the English Lexical Task, mi j is the number of times the English word wi co-occurs with the En­glish category cj—as listed in the word-category co-occurrence matrix (WCCM). For the Multilin­gual Chinese-English Lexical Task, mij is the num­ber of times the Chinese word wi co-occurs with the English category cj—as listed in the cross-lingual word-category co-occurrence matrix (CL-WCCM).

We calculate the pointwise mutual information be­tween a sense of the target word and a co-occurring word using the following formula:

mij is the count in the WCCM or CL-WCCM (as de­scribed in the previous subsection). For each sense of the target, the sum of the strength of association (PMI) between it and each ofthe co-occurring words (in a window of ±5 words) is calculated. The sense with the highest sum is chosen as the intended sense.

Note that this PMI-based classifier does not capital­ize on prior probabilities of the different senses.

The English Lexical Sample Task training and test instances respectively for 100 target words (50 nouns and 50 verbs). WordNet 2.1 is used as the sense inventory for most of the target words, but certain words have one or more senses from OntoNotes (Hovy et al., 2006). Many of the fine­grained senses are grouped into coarser senses.

Our approach relies on representing a sense with a number of near-synonymous words, for which a thesaurus is a natural source. Even though the ap­proach can be ported to WordNet, there was no easy

Table 1: English Lexical Sample Task: Results obtained using the PMI-based classifier on the training data and the naïve Bayes classifier on both training and test data way of representing OntoNotes senses with near-synonymous words. Therefore, we asked four na­tive speakers of English to map the WordNet and OntoNotes senses of the 100 target words to the Macquarie Thesaurus and use it as our sense inven­tory. We also wanted to examine the effect of using a very coarse sense inventory such as the categories in a published thesaurus (811 in all).

The annotators were presented with a target word, its WordNet/OntoNotes senses, and the Macquarie senses. WordNet senses were represented by syn­onyms, gloss, and example usages. The OntoNotes senses were described through syntactic patterns and example usages (provided by the task organizers). The Macquarie senses (categories) were described by the category head (a representative word for the category) and five other words in the category. Specifically, words in the same semicolon group as the target were chosen. Annotators 1 and 2 labeled each WordNet/OntoNotes sense of the first 50 target words with one or more appropriate Macquarie cat­egories. Annotators 3 and 4 labeled the senses of the other 50 words. We combined all four annotations into a WordNet-Macquarie mapping file by taking, for each target word, the union of categories chosen by the two annotators.

The English Lexical Substitution Task has 1710 test instances for 171 target words (nouns, verbs, adjec­tives, and adverbs) (McCarthy and Navigli, 2007). Some instances were randomly extracted from an Internet corpus, whereas others were selected man­ually from it. The target word might or might not be part of a multiword expression. The task is not tied to any particular sense inventory.

The Multilingual Chinese-English Lexical Sample Task training and test data (Jin et al., 2007) have 2686 and 935 instances respectively for 40 target words (19 nouns and 21 verbs). The instances are taken from a corpus of People's Daily News. The organizers used the Chinese Semantic Dictionary (CSD), developed by the Institute of Computational Linguistics, Peking University, both as a sense in­ventory and bilingual lexicon (to extract a suitable English translation of the target word once the in­tended Chinese sense is determined).

In order to determine the English translations of Chinese words in context, our system relies on Chi­nese text and an English thesaurus. As the thesaurus is used as our sense inventory, the first author and a native speaker of Chinese mapped the English trans­lations of the target to appropriate Macquarie cate­gories. We used three examples (from the training data) per English translation for this purpose.

Both the naïve Bayes classifier and the PMI-based one were applied to the training data. For each in­stance, the Macquarie category c that best captures the intended sense of the target was determined. The instance was labeled with all the WordNet senses that are mapped to c in the WordNet-Macquarie mapping file (described earlier in Section 4.1).

We used the PMI-based classifier for the English Lexical Substitution Task. Once it identifies a suit­able thesaurus category as the intended sense for a target, ten candidate substitutes are chosen from that category. Specifically, the category head word and up to nine words in the same semicolon group as the target are selected (words within a semicolon group are closer in meaning). Of the ten candidates, the single-word expression that is most frequent in the BNC is chosen as the best substitute; the motivation is that the annotators, who created the gold standard, were instructed to give preference to single words over multiword expressions as substitutes.

In the Multilingual Chinese-English Lexical Sample Task, both the naïve Bayes classifier and the PMI-based classifier were applied to the training data. For each instance, the Macquarie category, say c, that best captures the intended sense of the target word is determined. Then the instance is labeled with all the English translations that are mapped to c in the English translations-Macquarie mapping file (described earlier in Section 4.3).