The dataset of the SemEval task (Wu and lin, 2010) consists of short texts in Chinese contain­ing target adjectives whose sentiments need to be disambiguated in the given contexts. Those adjec­tives are: y*v big, JN small, many, ^ few, r(i high, low, jf£ thick, W thin, ffî. deep, shallow, Jt heavy, light, huge, Jty*v grave.

Disambiguating sentiment ambiguous adjec­tives is a challenging task for NLP. Previous stud­ies were mostly focused on word sense disam­biguation rather than sentiment disambiguation. Although both problems look similar, the latter is more challenging in our opinion because impreg­nated with more subjectivity. In order to solve the task, one has to deal not only with the semantics of the context, but also with the psychological as­pects of human perception of emotions from the written text.

In our approach, we use Twitter microblogging platform to retrieve emotional messages and form two sets of texts: messages with positive emotions and those with negative ones (Pak and Paroubek, 2010). We use emoticons as indicators of an emo­tion (Read, 2005) to automatically classify texts into positive or negative sets. The reason we use Twitter is because it allows us to collect the data with minimal supervision efforts. It provides an API which makes the data retrieval process much more easier then Web based search or other re­sources.

After the dataset of emotional texts has been obtained, we build a classifier based on n-grams Naïve Bayes approach. We tested two approaches to build a sentiment classifier:

1. In the first one, we collected Chinese texts from Twitter and used them to train a classi­fier to annotate the test dataset.

2. In the second one, we used machine trans­lator to translate the dataset from Chinese to English and annotated it using collected En­glish texts from Twitter as the training data.

We have made the second approach because we were able to collect much more of English texts from Twitter than Chinese ones and we wanted to test the impact of machine translation on the performance of our classifier. We have exper­imented with Google Translate and Yahoo Ba-belfish. Google Translate yielded better results.

In (Yang et al, 2007), the authors use web-blogs to construct a corpora for sentiment analysis and use emotion icons assigned to blog posts as indica­tors of users' mood. The authors applied SVM and CRF learners to classify sentiments at the sentence level and then investigated several strategies to de­termine the overall sentiment of the document. As

the result, the winning strategy is defined by con­sidering the sentiment of the last sentence of the document as the sentiment at the document level.

J. Read in (Read, 2005) used emoticons such as ":-)" and ":-(" to form a training set for the sen­timent classification. For this purpose, the author collected texts containing emoticons from Usenet newsgroups. The dataset was divided into "pos­itive" (texts with happy emoticons) and "nega­tive" (texts with sad or angry emoticons) samples. Emoticons-trained classifiers: SVM and Naïve Bayes, were able to obtain up to 70% accuracy on the test set.

In (Go et al., 2009), authors used Twitter to collect training data and then to perform a senti­ment search. The approach is similar to the one in (Read, 2005). The authors construct corpora by using emoticons to obtain "positive" and "neg­ative" samples, and then use various classifiers. The best result was obtained by the Naïve Bayes classifier with a mutual information measure for feature selection. The authors were able to obtain up to 84% of accuracy on their test set. However, the method showed a bad performance with three classes ("negative", "positive" and "neutral").

In our system, we use a similar idea as in (Go et al., 2009), however, we improve it by using a combination of unigrams, bigrams and trigrams ( (Go et al, 2009) used only unigrams). We also handle negations by attaching a negation particle to adjacent words when forming ngrams.

In our experiments, we used two datasets: a trial dataset containing 100 sentences in Chinese and

■google —yahoo

a test dataset with 2917 sentences. Both datasets were provided by the task organizers. Micro and macro accuracy were chosen as the evaluation metrics.

First, we compared the performance of our method when using Google Translate and Yahoo Babelfish for translating the trial dataset. The re­sults for micro and macro accuracy are shown in Graphs 1 and 2 respectively. The x-axis repre­sents a context window-size, equal to a number of words on both sides of the target adjective. The y-axis shows accuracy values. From the graphs we see that Google Translate provides better results, therefore it was chosen when annotating the test dataset.

Next, we studied the impact of the context win­dow size on micro and macro accuracy. The impact of the size of the context window on the accuracy of the classifier trained on Chinese texts is depicted in Graph 3 and for the classifier trained on English texts with translated test dataset croblogs using emoticons. The techniques used in our approach can be applied to any other language. Our system is fully automate and does not utilize any hand-built lexicon. We were able to achieve up to 64% of macro and 61% of micro accuracy at the SemEval 2010 task

For the future work, we would like to collect more Chinese texts from Twitter or similar mi-croblogging platforms. We think that increasing the training dataset will improve much the accu­racy of the sentiment disambiguation.

Figure 3 : Micro and macro accuracy for the first approach (training on Chinese texts)

Figure 4: Micro and macro accuracy for the sec­ond approach (training on English texts which have been machine translated) in Graph 4.

The second approach achieves better results. We were able to obtain 64% of macro and 61% of micro accuracy when using the second approach but only 63% of macro and 61% of micro accu­racy when using the first approach.

Another observation from the graphs is that Chinese requires a smaller size of a context win­dow to obtain the best performance. For the first approach, a window size of 8 words gave the best macro accuracy. For the second approach, we ob­tained the highest accuracy with a window size of 22 words.

In this paper, we have described our system for disambiguating sentiments of adjectives in Chi­nese texts. Our Naïve Bayes approach uses infor­mation automatically extracted from Twitter mi-