The subjective analysis of a text is becoming impor­tant for many Natural Language Processing (NLP) applications such as Question Answering, Informa­tion Extraction, Text Categorization among others (Shanahan et al., 2006). The resolution of this prob­lem can lead to a complete, realistic and coher­ent analysis of the natural language, therefore ma­jor attention is drawn to the opinion, sentiment and emotion analysis, and to the identification of be­liefs, thoughts, feelings and judgments (Quirk et al., 1985), (Wilson and Wiebe, 2005).

The aim of the Affective Text task is to clas­sify a set of news headlines into six types of emo­tions: "anger", "disgust", "fear", "joy", "sadness" and "surprise". In order to be able to conduct such multi-category analysis, we believe that first we need a comprehensive theory of what a human emotion is, and then we need to understand how the emotion is expressed and transmitted within the nat­ural language. These aspects rise the need of syn­tactic, semantic, textual and pragmatic analysis of a text (Polanyi and Zaenen, 2006). However, some of the major drawbacks in this field are related to the manual or automatic acquisition ofsubjective ex­pressions, as well as to the lack of resources in terms of coverage.

For this reason, our current emotion classification approach is based on frequency and co-occurrence bag of word counts collected from the World Wide Web. Our hypothesis is that words which tend to co-occur across many documents with a given emotion are highly probable to express this emotion.

The rest of the paper is organized as follows. In Section 2 we review some of the related work, in Section 3 we describe our web-based emotion classi­fication approach for which we show a walk-through example in Section 4. A discussion of the obtained results can be found in Section 5 and finally we con­clude in Section 6.

Our approach for emotion classification is based on the idea of (Hatzivassiloglou and McKeown, 1997) and is similar to those of (Turney, 2002) and (Tur-ney and Littman, 2003). According to Hatzivas-siloglou and McKeown (1997), adjectives with the same polarity tended to appear together. For exam­ple the negative adjectives "corrupt and brutal" cooccur very often.

The idea of tracing polarity through adjective co­occurrence is adopted by Turney (2002) for the bi­nary (positive and negative) classification of text re­views. They take two adjectives, for instance "ex­cellent" and "poor" in a way that the first adjective expresses positive meaning, meanwhile the second one expresses negative. Then, they extract all ad­jectives from the review text and combine them with "excellent" and "poor". The co-occurrences of these words are searched on the web, and then the Mutual Information score for the two groups of adjectives is measured. When the adjective of the review ap­pear more often with "excellent", then the review is classified as positive, and when the adjectives appear more often with "poor", then the review is classified as negative.

Following Hatzivassiloglou and McKeown (1997) and Turney (2002), we decided to observe how often the words from the headline co-occur with each one of the six emotions. This study helped us deduce information according to which "birthday" appears more often with "joy", while "war" appears more often with "fear".

Some of the differences between our approach and those of Turney (2002) are mentioned below:

• objectives: Turney (2002) aims at binary text classification, while our objective is six class classification of one-liner headlines. Moreover, we have to provide a score between 0 and 100 indicating the presence of an emotion, and not simply to identify what the emotion in the text is. Apart from the difficulty introduced by the multi-category classification, we have to deal with a small number of content words while Turney works with large list of adjectives.

• word class: Turney (2002) measures polarity using only adjectives, however in our approach we consider the noun, the verb, the adverb and the adjective content words. The motivation of our study comes from (Polanyi and Zaenen, 2006), according to which each content word can express sentiment and emotion. In addition to this issue we saw that most of the headlines contain only nouns and verbs, because they ex­press objectivity.

• search engines: Turney (2002) uses the Al­tavista web browser, while we consider and combine the frequency information acquired from three web search engines.

• word proximity: For the web searches, Tur-ney (2002) uses the NEAR operator and con­siders only those documents that contain the adjectives within a specific proximity. In our approach, as far as the majority of the query words appear in the documents, the frequency count is considered.

• queries: The queries of Turney (2002) are made up of a pair of adjectives, and in our approach the query contains the content words of the headline and an emotion.

There are other emotion classification approaches that use the web as a source of information. For instance, (Taboada et al., 2006) extracted from the web co-occurrences of adverbs, adjectives, nouns and verbs. Gamon and Aue (2005) were looking for adjectives that did not co-occur at sentence level. (Baroni and Vegnaduzzo, 2004) and (Grefenstette et al., 2004) gathered subjective adjectives from the web calculating the Mutual Information score.

Other important works on sentiment analysis are 2005; Wilson and Wiebe, 2005), who used linguistic information such as syntax and negations to deter­mine polarity. Kim and Hovy (2006) integrated verb information from FrameNet and incorporated it into semantic role labeling.

In order to determine the emotions of a headline, we measure the Pointwise Mu­tual Information (MI) of ei and cwj as mi{d,cwjj = iog2h^g^d,), where eie {anger, disgust, fear, joy, sadness, surprise} and cwj are the content words of the headline j. For each headline, we have six MI scores which indicate the presence of the emotion. MI is used in our experiments because it provides information about the independence of an emotion and a bag of words.

To collect the frequency and co-occurrence counts of the headline words, we need large and massive data repositories. To surmount the data sparsity problem, we used as corpus the World Wide Web which is constantly growing and daily updated.

Our statistical information is collected from three web search engines: MyWay, AlltheWeb and Ya-hoo. It is interesting to note that the emotion dis­tribution provided by each one of the search engines for the same headline has different scores. For this reason, we decided to compute an intermediate MI score as aMI = ^"=1/ ".

In the trail data, besides the MI score of an emo­tion and all headline content words, we have calcu­lated the MI for an emotion and each one of the con­tent words. This allowed us to determine the most sentiment oriented word in the headline and then we use this predominant emotion to weight the associ­ation sentiment score for the whole text. Unfortu­nately, we could not provide results for the test data set, due to the high number ofemotion-content word pairs and the increment in processing time and re­turned responses of the search engines.

As a walk through example, we use the Mortar as­sault leaves at least 18 dead headline which is taken from the trial data. The first step in our emotion clas­sification approach consists in the determination of the part-of-speech tags for the one-liner. The non­content words are stripped away, and the rest of the words are taken for web queries. To calculate the MI score of a headline, we query the three search en­gines combining "mortar, assault, leave, dead" with the anger, joy, disgust, fear, sadness and surprise emotions. The obtained results are normalized in a range from 0 to 100 and are shown in Table 1.

As can be seen from the table, the three search engines provide different sentiment distribution for the same headline, therefore in our final experiment we decided to calculate intermediate MI. Comparing our results to those of the gold standard, we can say that our approach detects significantly well the fear, sadness and angry emotions.

Table 2 shows the obtained results for the affective test data. The low performance of our approach is explainable by the minimal knowledge we have used. An interesting conclusion deduced from the trail and test emotion data is that the system detects better the negative feelings such as anger, disgust, fear and sadness, in comparison to the positive emo­tions such as joy and surprise. This makes us believe that according to the web most of the word-emotion combinations we queried are related to the expres­sion of negative emotions.

In the test run, we could not apply the emotion­word weighting, however we believe that it has a significant impact over the final performance. Presently, we were looking for the distribution of all content words and the emotions, but in the future we would like to transform all words into adjectives and then conduct web queries.

Furthermore, we would like to combine the re­sults from the web emotion classification with the polarity information given by SentiWordNet. A-priory we want to disambiguate the headline content words and to determine the polarities of the words and their corresponding senses. For instance, the ad­jective "new" has eleven senses, where new#a#3 and new#a#5 express negativism, new#a#4 and new#a#9 positivism and the rest of the senses are objective.

Emotion classification is a challenging and difficult task in Natural Language Processing. For our first attempt to detect the amount of angry, fear, sadness, surprise, disgust and joy emotions, we have pre­sented a simple web co-occurrence approach. We have combined the frequency count information of three search engines and we have measured the Mu­tual Information score between a bag of content words and emotion.

According to the yielded results, the presented ap­proach can determine whether one sentiment is pre­dominant or not, and most of the correct sentiment assignments correspond to the negative emotions. However, we need to improve the approach in many aspects and to incorporate more knowledge-rich re­sources, as well as to tune the 0-100 emotion scale.

This research has been funded by QALLME number FP6IST-033860 and TEX-MESS number TIN2006-15265-C06-01.