One of the lessons learned from our previous experi­ence at Senseval-3 (Buscaldi et al., 2004; Vazquez et al., 2004) is that the integration of different sys­tems usually works better than a standalone system. In our opinion this reflects the reality where humans do not apply always the same rule in order to disam-biguate the same ambigue word; for instance, if we consider the sentences "He hit a home run" and "The thermometer hit 100 degrees'", in the first case the sport domain helps in determining the right sense for hit, whereas in the latter the disambiguation is car­ried out mostly depending on the fact that the subject of the sentence is an object.

The combination of distinct methods represents itself a major problem. If the methods return dif­ferent answers, how can we select the best one? In this sense the available choices are the following:

• Rule-based selection: a set of rules that can be both hand-made or automatically learned from examples;

• Probability-based: the output of the methods is normalized in the range [0,1] and is considered as a probability. Then the values are multiplied in order to obtain the sense with a maximum probability.

• Vote-based: the output of the methods is con­sidered as a weighted vote. Then a voting scheme is used in order to obtain the most voted sense.

In our previous participation with the R2D2 project (Vazquez et al., 2004) the selection was rule-based, with hand-made rules that attempted to take into ac­count the reliability of the various method. We sub­sequently attempted to learn automatically the rules, but the results of these experiments did not allow to determine clearly which method was to be used in each context.

Working with probabilities can be problematic due to the null probabilities that make necessary the adoption of smoothing techniques. Therefore, we opted for a voting scheme, in this case the fuzzy Borda (Nurmi, 2001; Garcia Lapresta and Martinez

Panero, 2002), one of the best known methods in the field of collective decision making. With this scheme the disambiguation methods are considered as experts providing a preference ranking over the sense of the word.

The methods we choose as experts are the sense probability calculated over SemCor, the Conceptual Density algorithm by (Rosso et al., 2003), the ex­tended Lesk by (Banerjee and Pedersen, 2002), and an algorithm that takes into account the domains of the word to be disambiguated and the context words. In the following sections we describe in detail the fuzzy Borda scheme and each WSD expert.

The original Borda vote-counting scheme was in­troduced in 1770 by Jean Charles de Borda, and adopted by the French Academy of Sciences with the purpose of selecting its members. In the classical Borda count each expert gives a mark to each alter­native, according to the number ofalternatives worse than it. The fuzzy variant (Nurmi, 2001; Garcia Lapresta and Martinez Panero, 2002) is a natural ex­tension that allows the experts to show numerically how much some alternatives are preferred to the oth­ers, evaluating their preference intensities from 0 to 1.

Let R ,R,...,Rm be the fuzzy prefer­ence relations of m experts over n alternatives x1,x2,...,xn. For each expert k we obtain a matrix of preference intensities:

where each rj = ßRk (xi ,Xj ), with /j,Rk : X x X — [0,1] being the membership function of Rk. The number rj G [0,1] is considered as the degree of confidence with which the expert k prefers xi to xj. The final value assigned by the expert k to each al­ternative xi is:

which coincides with the sum of the entries greater than 0.5 in the i-th row in the preference matrix. The threshold 0.5 ensure the relation Rkto be an ordinary preference relation (Garcia Lapresta and Martinez

Panero, 2002).

Therefore, the definitive fuzzy Borda count for an alternative xi is obtained as the sum of the values assigned by each expert:

In order to fill the preference matrix with the correct confidence values, the output weights w1 ,w2,... ,wn of each expert k are transformed to fuzzy confidence values by means of the following transformation:

An example of how fuzzy Borda is used to combine the votes in order to obtain the right sense of the target word is shown in Section 4.

We considered five experts in order to carry out the disambiguation process. Sense probability and the extended lesk were available for every word, while the Conceptual Density was calculated only for nouns. Therefore, all the experts were available only for the nouns. For each expert different con­texts were taken into account, depending on the spe­cific characteristics of each expert.

In this example we will consider only the sense probability and extended Lesk experts for simplic­ity.

Let us consider the following phrase: "And he has kept mum on how his decision might affect a bid for United Airlines , which includes a big stake by British Airways PLC." with affect as target word. We can observe that in WordNet the verb affect has 5 senses. The sense count values are 43 for the first sense, 11 for the second, 4 for both the third and the fourth one, and 0 for the last one. We decided to nor­malize the cases with 0 occurrences to 1. After ap­plying the transformation (3) to the sense counts, we obtain the following preference matrix for the sense probability expert:

sense probability expert are 3.60 for affect (1), 2.38 for affect(2), 0.8 for affect(3) and affect(4), and 0 for affect(5), obtained from the sum of the rows where the value is greater than 0.5.

The extended Lesk expert calculates the following similarity scores for thesenses ofaffect, with context words decision, might, bid and include: respectively 107, 70, 35 , 63 and 71 for senses 1 to 5. After apply­ing the transformation (3) to the weights, we obtain the preference matrix for this expert:

In this case the final fuzzy Borda counts are 2.58 for the first sense, 1.2 for sense 2, 0 for sense 3, 0.64 and 1 . 71 for senses 4 and 5 respectively.

Finally, the sum of Borda counts of every expert for each sense (see Table 4) are used to disambiguate the word.

The system was not tested before SemEval. Our par­ticipation was limited to the All-Word and Coarse­Grained tasks (without the sense inventory provided by the organizers). The results are compared to the best system and the MFS (Most Frequent Sense) baseline. We calculated also the partial results over nouns in the all word task, obtaining that the MFS baseline in this case is about 0. 633, whereas our sys­tem obtains 0. 520.

Table 2: Recall obtained by our system (upv-wsd) in each task we participated in, compared with the most frequent sense baseline and the best system in the task.

The combination of different systems allowed us to attain higher recall than with our previous system used in Senseval-3. However, overall results were not as good as expected. Partial results over the nouns show that the CD expert did not perform as in the Senseval-3 and that the CD formula needs to include sense frequency ranking in order to achieve a good performance. As a further work we plan to add a weight reflecting the reliability of each expert.

We would like to thank the TIN2006-15265-C06-04 research project for partially supporting this work. We would also like to thank Prof. Eugene Levner of the Holon Institute of Technology for inspiring us to use the fuzzy Borda voting scheme.