Ambiguity of meaning is inherent in natural lan­guage because the deliverer of words tries to mini­mize the size of the vocabulary set he uses. There­fore, a sizable portion of this vocabulary is polyse-mous and the intended meaning of such words can be encoded in their context.

Due to the knowledge acquisition bottleneck problem and scarcity in training data (Cai et al., 2007), unsupervised corpus based approaches could be favored over supervised ones in word sense disambiguation (WSD) tasks.

Similar efforts in this include work by Cai et al. (Cai et al., 2007) in which they use latent Dirichlet allocation (LDA) topic models to extract the global context topic and use it as a feature along other baseline features. Another technique uses clustering based approach with WordNet as an external resource for disambiguation without rely­ing on training data (Anaya-Sânchez et al., 2007).

To disambiguate a polysemous word in a text document, we use the document topic distribution to represent its context. A document topic distri­bution is the probabilistic distribution of a docu­ment over a set of topics. The assumption is that: given two word senses and the topic distribution of their context, the closeness between these two topic distributions in their topic space is an indi­cation of the similarity between those two senses.

Our motivation behind building this system was the observation that the context of a polysemous word helps determining its sense to some degree. In our word sense induction (WSI) system, we use LDA to create a topic model for the given corpus and use it to infer the topic distribution of the documents containing the ambiguous words.

This paper describes our WSI system which par­ticipated in SemEval-2 word sense induction and disambiguation task (Manandhar et al., 2010).

LDA is a probabilistic model for a collection of dis­crete data (Blei et al., 2003). It can be graphically represented as shown in Figure 1 clS cL three level hierarchical Bayesian model. In this model, the corpus consists of M documents, each is a multino­mial distribution over K topics, which are in turn multinomial distributions over words.

To generate a document d using this probabilis­tic model, a distribution over topics 6a is generated using a Dirichlet prior with parameter a. Then, for each of the Nj words w^n in the document, a topic Zdn is drawn from a multinomial distribu­tion with the parameter 64. Then, a word Wdn is drawn from that topic's distribution over words, given ßij = p(w = i\z = j). Where ßij is the proba­bility of choosing word i given topic j.

We wanted to examine the trade-off between sim­plicity, cost and performance by building a simple language-independent, totally unsupervised, com­putationally cheap system and compare its perfor­mance to other WSI systems participating in the SemEval-2 WSI task (Manandhar et al., 2010). We expect a degradation in precision of our simple ap­proach as the granularity of senses becomes finer; This is due to the degrading sensitivity in mapping between the topics space and the senses space. We note that our simple approach will fail if multiple senses of the same word appear in the same docu­ment; Since these senses will be represented by the same topic distribution of the document, they will be clustered in the same cluster.

Our system is a language-independent system. The used LDA topic model has no knowledge of the training or testing corpus language. Unlike most other WSI and WSD systems, it doesn't make use of part of speech (POS) features which are lan­guage dependent and require POS annotated train­ing data. The only features used are the topics dis­tribution of bag-of-words containing the ambigu­ous word.

First, for each target polysemous word wp (noun or verb), we train a MALLET parallel topic model implementation of LDA on all the training in­stances of that word. Then we use the trained topic model to infer the topics distribution 0i for each of the test instances of that word. For a if-topics topic model, each topics distribution can be repre­sented as a point in a if-dimensional topic space. These points can be clustered into C different clus­ters, each representing a word sense. We used MALLET's if-means clustering algorithm with co­sine similarity to measure the distance between dif­ferent topic distributions in the topic space.

We use the same unsupervised evaluation mea­sures used in SemEval-2 (Manandhar and Kla-paftis, 2009). These measures do not require de­scriptive

The V-measure is used for unsupervised evalu­ation. It is the harmonic mean of the homogene­ity and completeness. Homogeneity is a measure of the degree that each formed cluster consists of data points that belong to a single gold standard (GS) class as defined below.

homogeneity = 1 H (GS)

Where H() is an entropy function, \C\ and \GS\ refer to cluster and class sizes, respectively. N is the number of data points, are data points of class GSi that belong to cluster Cj.

On the other hand, completeness measures the degree that each class consists of data points that belong to a single cluster. It is defined as follows.

completeness

Homogeneity and completeness can be seen as entropy based measures of precision and recall, re­spectively. The V-measure has a range of 0 (worst performance) to 1, inclusive.

The other evaluation measure is the F-score, which is the harmonic mean of precision and re­call. It has a range of 0 to 1 (best performance), inclusive.

The WSI system described earlier was tested on SemEval-1 WSI task (task 2) data (65 verbs, 35 nouns), and participated in the same task in SemEval-2 (task 14) (50 verbs, 50 nouns). The sense induction process was the same in both cases.

Before running our main experiments, we wanted to see how the number of topics if used in the topic model could affect the performance of our system. We tested our WSI system on SemEval-1 data using different if values as shown in Table 1. We found that the V-measure and F-score values increase with increasing if, as more dimensions are added to the topic space, the different senses in this if-dimensional space unfold. This trend stops at a value of if = 400 in a sign to the limited vocabu­lary of the training data. This if value is used in all other experiments.

Next, we evaluated the performance of our sys­tem on SemEval-1 WSI task data. Since no train­ing data was provided for this task, we used an un-annotated version of the test instances to create the LDA topic model. For each target word (verb or noun), we trained the topic model on its given test

V-measure F-score instances. Then we used the generated model's in-ferencer to find the topics distribution of each one of them. These distributions are then clustered in the topic space using the if-means algorithm and the cosine similarity measure was used to evalu­ate the distances between these distributions. The results of this experiment are shown in Table 2.

Our WSI system took part in the main SemEval-2 WSI task (task 14). In the unsupervised evalua­tion, our system had the second highest V-measure value of 15.7 for all words. A break down of the obtained V-measure and F-scores is shown in Table 3.

To analyze the performance of the system, we examined the clustering of the target noun word "promotion" to different senses by our system. We compared it to the GS classes of this word in the answer key provided by the task organizers. For a more objective comparison, we ran the if-means clustering algorithm with K equal to the number of GS classes. Even though the number of formed clusters affects the performance of the system, we assume that the number of senses is known in this analysis. We focus on the ability of the algorithm to cluster similar senses together. A graphical com­parison is given in Figure 2.

The target noun word "promotion" has 27 in­stances and four senses. The lower four rectangles in Figure 2 represent the four different GS classes, and the upper four rectangles represent the four clusters created by our system. Three of the four instances representing a job "promotion" (O) were clustered together, but the fourth one was clus­tered in a different class due to terms like "driv­ing," "troops," and "hostile" in its context. The offer sense of "promotion" (sj) was mainly split between two clusters, cluster 2 which most of its instances has mentions of numbers and monetary units, and cluster 4 which describes business and labor from an employee's eye.

The 13 instances of the third class which carry the sense encourage of the word promotion (□) are distributed among the four different clusters de-

A complete evaluation of all partic­ipating systems is available online at:

Job Offer Encourage Press pending on other topic words that classified them as either belonging to cluster 4 (encouragement in business), cluster 3 (encouragement in conflict or war context), cluster 2 (numbers and money con­text), or cluster 1 (otherwise). We can see that the topic model is unable to detect and extract topic words for the "encourage" sense of the word. Fi­nally, due to the lack of enough training instances of the sense of a promotional issue of a newspaper the topic model inferencer clustered it in the numbers and monetary cluster because it was rich in numbers.

Clustering the topics distributions of the global context of polysemous words in the topic space to induce their sense is cheap as it does not require any annotated data and is language-independent.

Even though the clustering produced by our sys­tem did not fully conform with the set of senses given by the GS classes, it can be seen from the analyzed example given earlier that our cluster­ing carried some different senses. In one case, a GS sense was not captured by the topic model, and instead, other cues from its instances context were used to cluster them accordingly. The in­duced clustering had some noise though.

This simple WSI approach can be used for cheap sense induction or for languages for which no POS tagger has been created yet. This system which had the second highest V-measure score in SemEval-2 WSI task achieves a good trade-off be­tween performance and cost.