Local collocations: bigrams and trigrams formed with the words around the target. These features are constituted by lemmas, word-forms, or PoS tags. Other local features are those formed with the previ­ous/posterior lemma/word-form in the context.

Syntactic dependencies: syntactic dependencies were extracted using heuristic patterns, and regular expressions defined with the PoS tags around the tar-get. The following relations were used: object, sub­ject, noun-modifier, preposition, and sibling.

Bag-of-words features: we extract the lemmas of the content words in the whole context, and in a ±4-word window around the target. We also obtain salient bigrams in the context, with the methods and the software described in (Pedersen, 2001).

Domain features: The WordNet Domains re­source was used to identify the most relevant do­mains in the context. Following the relevance for­mula presented in (Magnini and Cavaglia, 2000), we defined 2 feature types: (1) the most relevant do­main, and (2) a list of domains above a predefined threshold.

'The PoS tagging was performed with the fnTBL toolkit (Ngai and Florian, 2001).

Singular Value Decomposition (SVD) is an interest­ing solution to the sparse data problem. This tech­nique reduces the dimensions of the vectorial space finding correlations and collapsing features. It also gives the chance to use unlabeled data as an addi­tional source of correlations.

M 3 Rmxn, a matrix of features-by-document is built from the training corpus and decomposed into three matrices, as shown in Eq. (1). U and V, row and column matrix, respectively, have orthonormal columns and S is a diagonal matrix which contains k eigenvalues in descending order.

We used the singular value matrix (S) and the column matrix (U) to create a projection matrix, which is used to project the data (represented in fea­tures vectors) from the original space to a reduced space. Prior to that we selected the first p columns from the S and U matrices (p < k): tp = UpS"

We have explored two different variants in order to build a matrix, and obtain the SVD features:

SVD One Matrix per Target word (SVD-OMT). For each word (i) we extracted all the fea­tures from the given training (test) corpus, (ii) built the feature-by-document matrix from training cor­pus, (iii) decomposed it with SVD, and (iv) project all the training (test) data. Note that this variant has been only used in the lexical sample task due to its costly computational requirements.

SVD Single Matrix for All target words (SVD- SMA): (i) we extracted bag-of-words features from the British National Corpus (BNC) (Leech, 1992), (ii) built the feature-by-document matrix, (iii) de­compose it with SVD, and (iv) project all the data (train/test).

k-NN is a memory-based learning method, where the neighbors are the k most similar contexts, repre­sented by feature vectors (ci), of the test vector (/). The similarity among instances is measured by the cosine of their vectors. The test instance is labeled with the sense obtaining the maximum sum of the weighted votes of the k most similar contexts. The vote is weighted depending on its (neighbor) posi­tion in the ordered rank, with the closest being first. Eq. (2) formalizes k-NN, where Ci corresponds to the sense label of the i-th closest neighbor.

As seen in section 2 we use a variety of heteroge­neous sets of features. Our previous experience has shown that splitting the problem up into more co­herent spaces, training different classifiers in each feature space, and then combining them into a sin­gle classifier is a good way to improve the results (Agirre et al., 2005; Agirre et al., 2006). Depend­ing on the feature type (original features or features extracted from SVD projection) we split different sets of feature spaces. In total we tried 10 features spaces.

For the original features:

• alLfeats: Extracted all original features.

• alLnotdom: All original features except do­main features.

• local: All the original features except domain and bag-of-words features.

• topic: The sum of bag-of-words and domain features.

• bow: Bag-of-word features.

• dom: Domain features.

For the lexical sample task we only use the train­ing data provided. We tuned the classifiers using 3 fold cross-validation on the SemEval lexical sample training data. We tried to optimize several param­eters: number of neighbors, SVD dimensions and best combination of the single k-NNs. We set k as one of 1,3, 5 and 7, and the SVD dimension (d) as one of 50,100,200 and 300. We also fixed the best combination. This is the optimization procedure we followed:

1. For each single classifier and feature set (see section 2), check each parameter combination.

2. Fix the parameters for each single classifier. In our case, k = 5 and k = 7 had similar results, so we postponed the decision. d = 200 was the best dimension for all classifiers, except SVD-OMT[topic] which was d = 50.

3. For the best parameter settings (k = 5; k = 7 and d = 200; d = 50 when SVD-OMT[topic]) make a priori meaningful combinations (due to CPU requirements, not all combination were feasible).

4. Choose the x best combination overall, and op­timize word by word among these combination. We set x = 8 for this work, k was fixed in 5, which was d = 50).

Table 1 shows the best results for 3 fold cross-validation in SemEval lexical sample training cor­pus. The figures show that optimizing each word the performance increases 0.7 percentage points over the best combination.

To train the classifiers for the all-words task we just used Semcor (Miller et al., 1993). In (Agirre et al., 2006) we already tested our approach on the Senseval-3 all-words task. The best performance for the Senseval-3 all-words task was obtained with k = 5 and d = 200, but we decided to to perform further experiments to search for the best combina­tion. We tested the performance of the combination of single k-NN training on Semcor and testing both on the Senseval-3 all-words data (cf. Table 2) and on the training data from SemEval-2007 lexical sample (cf. Table 3).

Note that tables 2 and 3 show contradictory re­sults. Given that in SemEval-2007 lexical sample our results.

Table 2: Results for the best k-NN combinations in Senseval-3 all-words, using Semcor as training cor­pus.

Table 3: Results for the best k-NN combinations in training part of SemEval lexical sample, using Sem-cor as training corpus.

the senses are more coarse grained, we decided to take the best combination on Senseval-3 all-words for the final submission.