In this paper, we present our system that was ap­plied to the cross lingual substitution for two tasks in SEMEVAL 2010, Tasks 2 and 3. We adopt the same approach for both tasks with some dif­ferences in the basic set-up. Our basic approach relies on applying a word sense disambiguation (WSD) system to the English data that comes from a parallel corpus for English and a language of relevance to the task, language 2 (12). Then we automatically induce the English word sense cor­respondences to 12. Accordingly, for a given test target word, we return its equivalent 12 words as­suming that we are able to disambiguate the target word in context.

We approach the problem of multilingual lexical substitution from a WSD perspective. We adopt the hypothesis that the different word senses of ambiguous words in one language probably trans­late to different lexical items in another language. Hence, our approach relies on two crucial compo­nents: a WSD module for the source language (our target test words, in our case these are the English target test words) and an automatic word align­ment module to discover the target word sense cor­respondences with the foreign words in a second language. Our approach to both tasks is unsuper­vised since we don't have real training data anno­tated with the target words and their corresponding translations into 12 at the onset of the problem.

Accordingly, at training time, we rely on auto­matically tagging large amounts of English data (target word instances) with their relevant senses and finding their 12 correspondences based on au­tomatically induced word alignments. Each of these English sense and 12 correspondence pairs has an associated translation probability value de­pending on frequency of co-occurrence. This in­formation is aggregated in a look-up table over the entire training set. An entry in the table would have a target word sense type paired with all the observed translation correspondences 12 word types. Each of the 12 word types has a probabil­ity of translation that is calculated as a normal­ized weighted average of all the instances of this 12 word type with the English sense aggregated across the whole parallel corpus. This process re­sults in an English word sense translation table (WSTT). The word senses are derived from Word-Net (Fellbaum, 1998). We expand the English word sense entry correspondences by adding the translations of the members of target word sense synonym set as listed in WordNet.

For alignment, we specifically use the GIZA++ software for inducing word alignments across the

GIZA++ to the parallel corpus in both directions

English to 12 and 12 to English then take only the

intersection of the two alignment sets, hence fo-129 cusing more on precision of alignment rather than recall.

For each language in Task 3 and Task 2, we use TreeTagger to do the preprocessing for all languages. The preprocessing includes segmenta­tion, POS tagging and lemmatization. Since Tree-Tagger is independent of languages, our system does not rely on anything that is language spe­cific; our system can be easily applied to other languages. We run GIZA++ on the parallel cor­pus, and obtain the intersection of the alignments in both directions. Meanwhile, every time a target English word appears in a sentence, we apply our WSD system on it, using the sentence as context. From this information, we build a WSST from the English sense(s) to their corresponding foreign words. Moreover, we use WordNet as a means of augmenting the translation correspondences. We expand the word sense to its synset from WordNet adding the 12 words that corresponded to all the member senses in the synset yielding more trans­lation variability.

At test time, given a test data target word, we apply the same WSD system that is applied to the training corpus to create the WSTT Once the tar­get word instance is disambiguated in context, we look up the corresponding entry in the WSTT and return the ranked list of 12 correspondences. We present results for best and for oot which vary only in the cut off threshold. In the BEST condition we return the highest ranked candidate, in the oot con­dition we return the top 10 (where available).

Given the above mentioned pipeline, Tasks 2 and 3 are very similar. Their main difference lies in the underlying WSD system applied.

Our work mainly investigates the influence of WSD on providing machine translation candi­dates. Carpuat & Wu (2007) and Chan et al.(2007) show WSD improves MT. However, in (Carpuat & Wu, 2007) classical WSD is missing by ignor­ing predefined senses. They treat translation can­didates as sense labels, then find linguistic fea­tures in the English side, and cast the disambigua­tion process as a classification problem. Of rele­vance also to our work is that related to the task of English monolingual lexical substitution. For example some of the approaches that participated in the SemEval 2007 excercise include the follow­ing. Yuret (2007) used a statistical language model based on a large corpus to assign likelihoods to each candidate substitutes for a target word in a sentence. Martinez et al. (2007) uses WordNet to find candidate substitutes, produce word sequence including substitutes. They rank the substitutes by ranking the word sequence including that substi­tutes using web queries. In (Giuliano C. et al, 2007), they extract synonyms from dictionaries. They have 2 ways of ranking of the synonyms: by similarity metric based on LSA and by occur­rence in a large 5-gram web corpus. Dahl et al. (2007) also extract synonyms from dictionaries. They present two systems. The first one scores substitutes based on how frequently the local con­text match the target word. The second one in­corporates cosine similarity. Finally, Hassan et al. (2007) extract candidates from several linguistic resources, and combine many techniques and ev­idences to compute the scores such as machine translation, most common sense, language model and so on to pick the most suitable lexical substi­tution candidates.

In this paper we presented a word sense disam­biguation based system for multilingual lexical substitution. The approach relies on having a WSD system for English and an automatic word alignment method. Crucially the approach relies on having parallel corpora. For Task 2 we apply a supervised WSD system to derive the English word senses. For Task 3, we apply an unsuper­vised approach to the training and test data. Both of our systems that participated in Task 2 achieve a decent ranking among the participating systems. For Task 3 we achieve the highest ranking on sev­eral of the language pairs: French, German and Italian.

In the future, we would like to investigate the

usage of the Spanish and Italian WordNets for the 131 task. We would like to also expand our exami­nation to other sources of bilingual data such as comparable corpora. Finally, we would like to in­vestigate using unsupervised clustering of senses (Word Sense Induction) methods in lieu of the WSD approaches that rely on WordNet.