Word Sense Disambiguation (WSD) aims to iden­tify the correct sense of ambiguous words in context. Results from the last edition of the Senseval com­petition (Mihalcea et al., 2004) have shown that, for supervised learning, the best accuracies are obtained with a combination of various types of features, to­gether with traditional machine learning algorithms based on feature-value vectors, such as Support Vec­tor Machines (SVMs) and Naive Bayes. While the features employed by these approaches are mostly considered to be "shallow", that is, extracted from corpus or provided by shallow syntactic tools like part-of-speech taggers, it is generally thought that significant progress in automatic WSD would re­quire a "deep" approach in which access to substan­tial body of linguistic and world knowledge could assist in resolving ambiguities. Although the ac­cess to large amounts of knowledge is now possi­ble due to the availability of lexicons like WordNet, parsers, etc., the incorporation of such knowledge has been hampered by the limitations of the mod­elling techniques usually employed for WSD. Using certain sources of information, mainly relational in­formation, is beyond the capabilities of such tech­niques, which are based on feature-value vectors. Arguably, Inductive Logic Programming (ILP) sys­tems provide an appropriate framework for dealing with such data: they make explicit provisions for the inclusion of background knowledge of any form; the richer representation language used, based on first­order logic, is powerful enough to capture contextual relationships; and the modelling is not restricted to being of a particular form (e.g., classification).

We describe the investigation of the use of ILP for WSD in the Lexical Sample task of SemEval-2007 in two different ways: (a) the construction of models that can be used directly to disambiguate words; and (b) the construction of interesting fea­tures to be used by a standard feature-based algo­rithm, namely, SVMs, to build disambiguation mod­els. We call the systems resulting of the two differ­ent approaches "USP-IBM-1" and "USP-IBM-2", respectively. The background knowledge is from 10 different sources of information extracted from cor­pus, lexical resources and NLP tools.

In the rest of this paper we first present the spec­ification of ILP implementations that construct ILP models and features (Section 2) and then describe the experimental evaluation on the SemEval-2007 Lexical Sample task data (Section 3).

Inductive Logic Programming (ILP) (Muggleton, 1991) employs techniques from Machine Learning and Logic Programming to build first-order theo­ries or descriptions from examples and background knowledge, which are also represented by first-order clauses. Functionally, ILP can be characterised by two classes of programs. The first, predictive ILP, is concerned with constructing models (in this case, sets of rules) for discriminating accurately amongst positive and negative examples. The partial spec­ifications provided by (Muggleton, 1994) form the basis for deriving programs in this class:

• B is background knowledge consisting of a fi­nite set of clauses = {Ci, C2,...}

• E is a finite set of examples = E + U E- where:

- Positive Examples. E+ = {ei, e2,...} is a non-empty set of definite clauses

• H, the output of the algorithm given B and E, is acceptable if these conditions are met:

- Prior Satisfiability. B U E- = □ - Posterior Satisfiability. B U H U E- = □ - Posterior Sufficiency. B U H = e\ A e2 A

The second category of ILP programs, descriptive ILP, is concerned with identifying relationships that hold amongst the background knowledge and exam­ples, without a view of discrimination. The partial specifications for programs in this class are based on the description in (Muggleton and Raedt, 1994):

• B is background knowledge

• E is a finite set of examples (this may be

The intuition behind the idea of exploiting a feature-based model constructor that uses first-order features is that certain sources of structured infor­mation that cannot be represented by feature vectors can, by a process of "propositionalization", be iden­tified and converted in a way that they can be accom­modated in such vectors, allowing for traditional learning techniques to be employed. Essentially, this involve two steps: (1) a feature-construction step that identifies all the features, that is, a set of clauses H, that are consistent with the constraints provided by the background knowledge B (descriptive ILP); and (2) a feature-selection step that retains some of the features based on their utility in classifying the examples, for example, each clause must entail at least one positive example (predictive ILP). In order to be used by SVMs, each clause hi in H is con­verted into a boolean feature fi that takes the value 1 (or 0) for any individual for which the body of the clause is true (if the body is false). Thus, the set of clauses H gives rise to a boolean vector for each individual in the set of examples. The fea­tures constructed may express conjunctions on dif­ferent knowledge sources. For example, the follow­ing boolean feature built from a clause for the verb "ask" tests whether the sentence contains the expres­sion "ask out" and the word "dinner". More details on the specifications of predictive and descriptive ILP for WSD can be found in (Specia et al., 2007):

We investigate the performance oftwo kinds ofILP-based models for WSD:

1. ILP models (USP-IBM-1 system): models con­structed by an ILP system for predicting the correct sense ofa word.

H, the output of the algorithm given B and E is acceptable if the following condition is met:

- Posterior Sufficiency. B U H U E = □

2. ILP-assisted models (USP-IBM-2 system): models constructed by SVMs for predicting the correct sense of a word that, in addition to ex­isting shallow features, use features built by an ILP system according to the specification for feature construction in Section 2.

We have investigated the use of ILP as a mech­anism for incorporating shallow and deep knowl­edge sources into the construction of WSD mod­els for the Semeval-2007 Lexical Sample Task data. Results consistently outperform the most frequent sense baseline. It is worth noticing that the knowl­edge sources used here were initially designed for the disambiguation of verbs (Specia et al., 2007) and therefore we believe that further improvements could be achieved with the identification and speci­fication of other sources which are more appropriate for the disambiguation of nouns.