The UvT-WSDl system described in this paper took part in two similar SemEval-2 tasks: Cross-Lingual Word Sense Disambiguation (Lefever and Hoste, 2010) and Cross-Lingual Lexical Substitu­tion (Mihalcea et al., 2010). In each task, a num­ber of words is selected for which the senses are to be determined for a number of instances of these words. For each word, a number of samples in context is provided, where each sample consists of one sentence, with the word to be disambiguated marked.

Because of the cross-lingual nature of the tasks, a word sense corresponds to a translation in an­other language, rather than a sense description in the same language. In the Cross-lingual Lexical Substitution task, the target language is Spanish. The task is to find Spanish substitutes for the En­glish words marked in the test samples. In the

Cross-Lingual Word Sense Disambiguation task, we participate for English-Dutch and English-Spanish. The Word Sense Disambiguation task provides training data for all five languages, in the form of the sentence-aligned EuroParl parallel cor­pus (Koehn, 2005). This is the source of training data the UvT-WSDl system uses for both tasks.

The system may output several senses per in­stance, rather than producing just one sense pre­diction. These are evaluated in two different ways. The scoring type "best" expects that the system outputs the best senses, in the order of its con­fidence. The scoring type "out of five/ten" ex­pects five or ten guesses, and each answer weighs the same. These metrics are more extensively described in (Mihalcea et al., 2010). The UvT-WSDl system participates in both scoring types, for both tasks. The system put forth in this paper follows a similar approach as described in earlier research by (Hoste et al., 2002).

The UvT-WSDl system uses machine learning techniques to learn what senses/translations are as­sociated with any of the target words. It does so on the basis of a variety of local and global context features, discussed in Section 2.2. At the core of the system are the classifiers, or so called "word experts", one per target word. These are built using the Tilburg Memory Based Learner (TiMBL) (Daelemans et al., 2009), making use of the IB 1 algorithm, an implementation of the k-nearest neighbour classifier.

The core of the system can be subdivided into roughly three stages. In the first stage, the word-aligned parallel corpus is read and for each found instance of one of the target words, features are ex­tracted to be used in the classifier. The class con­sists of the word aligned to the found instance of the target word, i.e. the translation/sense. In this way a word expert is built for each of the target words in the task, yielding a total amount of clas­sifiers equal to the total amount of target words. The test data is processed in a similar way, for each marked occurrence of any of the target words, features are extracted and test instances are cre­ated. Subsequently, the word experts are trained and tested, and on the basis of the training data, a parameter search algorithm (Van den Bosch, 2004) determines the optimal set of classifier parameters for each word expert, including for example the value of k and the distance weighting metric used.

In the last phase, the classifier output of each word expert is parsed. The classifiers yield a dis­tribution of classes per test instance, and these are converted to the appropriate formats for "best" and "out of five/ten" evaluation. For the latter scor­ing type, the five/ten highest scoring senses are selected, for the former scoring type, all classes scoring above a certain threshold are considered "best". The threshold is set at 90% of the score of the highest scoring class.

2.1 Word-Alignment, Tokenisation, Lemmatisation and Part-of-Speech-tagging

The Europarl parallel corpus, English-Spanish and English-Dutch, is delivered as a sentence-aligned parallel corpus. We subsequently run GIZA++ (Och and Ney, 2000) to compute a word-aligned parallel corpus.

This, however, is not the sole input. The tar­get words in both tasks are actually specified as a lemma and part-of-speech tag pair, rather than words. In the Word Sense Disambiguation task, all target lemmas are simply nouns, but in the Cross-Lingual Lexical Substitution task, they can also be verbs, adjectives or adverbs. Likewise, both tasks expect the sense/translation output to also be in the form of lemmas. Therefore the system internally has to be aware of the lemma and part-of-speech tag of each word in the parallel corpus and test data, only then can it successfully find all occur­rences of the target words. In order to get this information, both sides of the word-aligned paral­lel corpus are run through tokenisers, lemmatisers and Part-of-Speech taggers, and the tokenised out­put is realigned with the untokenised input so the word alignments are retained. The test data is also processed this way. For English and Spanish, the software suite Freeling (Atserias et al., 2006) per­formed all these tasks, and for Dutch it was done

The system can extract a variety of features to be used in training and testing. A distinction can be made between local context features and global context features. Local context features are ex­tracted from the immediate neighbours of the oc­currence of the target word. One or more of the following local context features are extractable by the UvT-WSDl system: word features, lemma features, and part-of-speech tag features. In each case, n features both to the right and left of the focus word are selected. Moreover, the system also supports the extraction of bigram features, but these did not perform well in the experiments.

The global context features are made up of a bag-of-words representation of keywords that may be indicative for a given word to sense/translation mapping. The idea is that words are collected which have a certain power of discrimination for the specific target word with a specific sense, and all such words are then put in a bag-of-word representation, yielding as many features as the amount of keywords found. A global count over the full corpus is needed to find these keywords. Each keyword acts as a binary feature, indicating whether or not that particular keyword is found in the context of the occurrence of the target word. The context in which these keywords are searched for is exactly one sentence, i.e. the sentence in which the target word occurs. This is due to the test data simply not supplying a wider context.

The method used to extract these keywords (k) is proposed by (Ng and Lee, 1996) and used also in the research of (Hoste et al., 2002). Assume we have a focus word /, more precisely, a lemma and part-of-speech tag pair of one of the target words. We also have one of its aligned translations/senses s, which in this implementation is also a lemma. We can now estimate P(s\k), the probability of sense s, given a keyword k, by dividing Ns>klocal. (the number of occurrences of a possible local context word k with particular focus word lemma-PoS combination and with a particular sense s) by Nkloral (the number of occurrences of a possible local context keyword k\oc with a particular focus word-PoS combination regardless of its sense). If we also take into account the frequency of a pos­sible keyword k in the complete training corpus (Nif ), we get:

Both SemEval-2 tasks have provided trial data upon which the system could be tested during the development stage. Considering the high config­urability of the various parameters for feature ex­traction, the search space in possible configura­tions and classifier parameters is vast, also due to fact that the TiMBL classifier used may take a wealth of possible parameters. As already men­tioned, for the latter an automatic algorithm of pa-

Table 1: UvT-WSDl results in the Cross-Lingual Lexical Substitution task rameter optimisation was used (Van den Bosch, 2004), but optimisation of the feature extraction parameters has not been automated. Rather, a se­lection of configurations has been manually cho­sen and tested during the development stage.

The following two configurations of features were found to perform amongst the best on the trial data. Therefore they have been selected and submitted for the contest:

1. UvT-WSDl-v (aka UvT-v) - An arbiter-voting system over three classifiers: 1) Word experts with two word features and lemma features on both sides of the focus word. 2)Word experts with global features. 3) Word experts with two word features, two lemma features and two part-of-speech tag features.

2. UvT-WSDl-g (aka UvT-g) - Word experts with global features only.

Table 1 shows a condensed view of the results for the Cross-Lingual Lexical Substitution task. Table 2 shows the final results for the Word-Sense Disambiguation task. Note that UvT-WSDl-v and UvT-WSDl-g are two different configurations of the UvT-WSDl system, and to conserve space these are abbreviated as UvT-v and UvT-g respec­tively. These are also the names used in both tasks (Lefever and Hoste, 2010; Mihalcea et al., 2010) to refer to our system.

Cross-Lingual Word Sense Disambiguation and Cross-Lingual Lexical Substitution have proven to be hard tasks, with scores that are relatively close to baseline. This can be attributed to a noticeable trait in the system output to be inclined to assign the same majority sense to all instances.

!For the Cross-Lingual Lexical Substitution task only, the parameter to recompute the T\ threshold automatically was enabled.