<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="327"/><title>Enhancing the HL-SOT Approach to Sentiment Analysis via a Localized Feature Selection Framework</title><pubinfo>Proceedings of the 5th International Joint Conference on Natural Language Processing,pages 327-335, Chiang Mai, Thailand, November 8 - 13, 2011. ©2011 AFNLP</pubinfo><author surname="Wei" givenname="Wei"><org  name="University of Science and Technology" country="Hong Kong" city="Clear Water Bay"/></author><author surname="Gulla" givenname="Jon Atle"><org  name="University of Science and Technology" country="Hong Kong" city="Clear Water Bay"/></author></firstpageheader><frontmatter><p><b>Enhancing the HL-SOT Approach to Sentiment Analysis via a Localized</b></p><p><b>Feature Selection Framework</b></p><p><b>Wei Wei</b></p><p>Department of Computer and Information Science Norwegian University of Science and Technology wwei@idi.ntnu.no</p><p><b>Jon Atle Gulla</b></p><p>Department of Computer and Information Science Norwegian University of Science and Technology jag@idi.ntnu.no</p></frontmatter><abstract>In this paper, we propose a Localized Fea­ture Selection (LFS) framework tailored to the HL-SOT approach to sentiment analy­sis. Within the proposed LFS framework, each node classifier of the HL-SOT ap­proach is able to perform classification on target texts in a locally customized index term space. Extensive empirical analysis against a human-labeled data set demon­strates that with the proposed LFS frame­work the classification performance of the HL-SOT approach is enhanced with com­putational efficiency being greatly gained. To find the best feature selection algorith­m that caters to the proposed LFS frame­work, five classic feature selection algo­rithms are comparatively studied, which indicates that the TS, DF, and MI al­gorithms achieve generally better perfor­mances than the CHI and IG algorithms. Among the five studied algorithms, the T­S algorithm is best to be employed by the proposed LFS framework. </abstract></header><body><section number="1" title="Introduction"><p>With tens and thousands of review texts being gen­erated online, it becomes increasingly challeng­ing for an individual to exhaustively collect and study the online reviews. Therefore, research on automatic sentiment analysis on review texts has emerged as a popular topic at the crossroads of in­formation retrieval and computational linguistics.</p><p>Sentiment analysis on product reviews aims at extracting sentiment information from texts. It in­cludes two tasks, i.e., labeling a target text<footnote anchor="1"/> with</p><p>!Each product review to be analyzed is called target text</p><p>1) the product's attributes it mentions (attributes identification task), and 2) the corresponding sen­timents mentioned therein (sentiment annotation task). Recently, Wei and Gulla proposed the HL-SOT approach (Wei and Gulla, 2010), i.e., Hier­archical Learning (HL) with Sentiment Ontology Tree (SOT), that is able to achieve the two tasks in one hierarchical classification process. In the HL-SOT approach, each target text is encoded by a vector in a globally unified d-dimensional index term space and is respectively labeled by different nodes<footnote anchor="2"/> of SOT in a hierarchical manner. Although the HL-SOT approach is reported with promising classification performance on tasks of sentimen­t analysis, its computational efficiency, especially as d increases, becomes very low. Furthermore, as d increases it will have more chance to index noisy term into the globally unified index term s­pace so that the classification performance of the HL-SOT approach might be depressed. Hence, we argue that if a locally customized index ter­m space could be constructed for each node re­spectively, both the computational efficiency and the classification performance of the HL-SOT ap­proach would be improved.</p><p>In this paper, we propose a Localized Feature Selection (LFS) framework tailored to the HL-SOT approach. The rationale of the proposed LFS framework draws on the following two observa­tions. Firstly, a feature term that is relevant to a node is usually irrelevant to nodes which stay at another branch of SOT. For example, "ergonomic­s" might be a feature term for the node "design and usability" (see Fig. 1) but it is irrelevant to the node "image quality". Secondly, a feature ter-</p><p>in the following of this paper.</p><footnote label="2">If specified otherwise in the following of this paper the term "node" refers to the classifier of the node.</footnote><page local="2" global="328"/><doubt alpha="47.6" length="21" tooSmall="True" monospace="0.0">lens )       camera -</doubt><doubt alpha="63.1" length="157" tooSmall="False" monospace="0.0">design and usability + ( weight )        (  interface ) design and usability - image quality +        ( noise )   (  resolutionjimage quality - lens + lens -</doubt><doubt alpha="66.7" length="108" tooSmall="True" monospace="0.0">weight + weight - interface +       ( menuJ    (buttonjinterface - noise + noise - resolution + resolution -</doubt><doubt alpha="64.5" length="31" tooSmall="True" monospace="0.0">menu + menu - button + button -</doubt><doubt alpha="100.0" length="6" tooSmall="True" monospace="0.0">camera</doubt><figure caption="Figure 1: an example of part of a SOT for digital camera"></figure><p>m might become insignificant for child nodes of i even if the feature term is significant to <i>i.</i><i> </i>For example, for a sentence commenting on a digital camera like "40D handles noise very well", terms such as "noise" and "well" are significant feature terms for the node "noise". However, the term "noise" becomes insignificant for its child nodes "noise +" and "noise -", since the hierarchical clas­sification characteristic of the HL-SOT approach that a node only processes target texts which are labeled as true by its parent node ensures that each target text handled by the nodes "noise +" and "noise -" is already classified as related to "noise".</p><p>In the proposed LFS framework, the concep­t of "local hierarchy" is defined and introduced as delimitation of local scope of nodes. The lo­calized feature selection process is conducted for each node within its local scope to generate the customized index term space for the node. The proposed LFS framework is empirically analyzed on a human-labeled data set. The experimental results show that with the proposed LFS frame­work the classification performance of the HL-SOT approach is enhanced and the computation­al efficiency is significantly improved. To test which is the best to be employed by the proposed LFS framework, we further comparatively study five classic feature selection algorithms respec­tively based on document frequency (DF) (Man­ning et al., 2008), mutual information (MI) (Man­ning et al., 2008; Church and Hanks, 1990), x<footnote anchor="2"/>-statistic (CHI) (Manning et al., 2008), information gain (IG) (Mitchell, 1997), and term strength (T­S) (Wilbur and Sirotkin, 1992). The comparative­ly experimental results suggest that the TS, DF, and MI algorithms achieve generally better perfor­mance than the CHI and IG algorithms. Among the five employed algorithms, the TS algorithm is the best to be employed by the proposed LFS framework. This paper makes the following con­tributions:</p><p>• We propose a LFS framework to enhance the classification performance and improve the computational efficiency of the HL-SOT ap­proach;</p><p>• We conduct a comparative study on five fea­ture selection algorithms that can be em­ployed in the proposed LFS.</p><p>The remainder of the paper is organized as fol­lows. In section 2, we discuss an overview of related work on sentiment analysis. In section 3, we review the HL-SOT approach proposed in (Wei and Gulla, 2010). In section 4, we present the pro­posed LFS framework. The empirical analysis and the results are presented in section 5. Finally, we conclude the paper and discuss the future work in section 6.</p></section><section number="2" title="Related Work"><p>Research on sentiment analysis was originally per­formed to extract overall sentiments from target texts. However, as shown in the experiments in (Turney, 2002), the whole sentiment of a docu­ment is not necessarily the sum of its parts. Recent work has shifted the focus from overall document sentiment to sentiment analysis based on product attributes (Hu and Liu, 2004; Popescu and Etzioni, 2005; Ding and Liu, 2007; Liu et al., 2005).</p><p>Document overall sentiment analysis is to sum­marize the overall sentiment in the documen­t, which relies on two finer levels of sentiment annotation: <i>word-level sentiment annotation </i>and <i>phrase-level sentiment annotation. </i>The <i>word­level sentiment annotation </i>is to utilize the polar­ity annotation of words in each sentence and sum­marize the overall sentiment of each sentiment-bearing word to infer the overall sentiment within the text (Hatzivassiloglou and Wiebe, 2000; An-dreevskaia and Bergler, 2006; Esuli and Sebas­tian', 2005; Esuli and Sebastiani, 2006; Hatzi-vassiloglou and McKeown, 1997; Kamps et al., 2004; Devitt and Ahmad, 2007; Yu and Hatzivas-siloglou, 2003).<page local="3" global="329"/> The <i>phrase-level sentiment anno­tation </i>focuses sentiment annotation on phrases not words with concerning that atomic units ofexpres-sion is not individual words but rather appraisal groups (Whitelaw et al., 2005). In (Wilson et al., 2005), the concepts of <i>prior polarity </i>and <i>contex­tual polarity </i>were proposed. This paper present­ed a system that is able to automatically identify the <i>contextual polarity </i>for a large subset of senti­ment expressions. In (Turney, 2002), an unsuper-vised learning algorithm was proposed to classify reviews as recommended or not recommended by averaging sentiment annotation of phrases in re­views that contain adjectives or adverbs. However, the performances of these approaches are not satis­factory for sentiment analysis on product reviews, where sentiment on each attribute of a produc­t could be so complicated that it is unable to be expressed by overall document sentiment.</p><p>Attributes-based sentiment analysis is to ana­lyze sentiment based on each attribute of a produc­t. In (Hu and Liu, 2004), mining product features was proposed together with sentiment polarity an­notation for each opinion sentence. In that work, sentiment analysis was performed at the produc­t attributes level. In (Liu et al., 2005), a system with framework for analyzing and comparing con­sumer opinions of competing products was pro­posed. The system made users be able to clearly see the strengths and weaknesses of each produc­t in the minds of consumers in terms of various product features. In (Popescu and Etzioni, 2005), Popescu and Etzioni not only analyzed polarity of opinions regarding product features but also ranked opinions based on their strength. In (Liu et al., 2007), Liu et al. proposed Sentiment-PLSA that analyzed blog entries and viewed them as a document generated by a number of hidden sen­timent factors. These sentiment factors may also be factors based on product attributes. In (Lu and Zhai, 2008), Lu et al. proposed a semi-supervised topic models to solve the problem of opinion inte­gration based on the topic of a product's attributes. The work in (Titov and McDonald, 2008) present­ed a multi-grain topic model for extracting the rat­able attributes from product reviews. In (Lu et al., 2009), the problem of rated attributes summary was studied with a goal of generating ratings for major aspects so that a user could gain differen­t perspectives towards a target entity. In a most recent research work (Wei and Gulla, 2010), Wei and Gulla proposed the HL-SOT approach that sufficiently utilizes the hierarchical relationships among a product attributes and solves the senti­ment analysis problem in a hierarchical classifi­cation process. However, the HL-SOT approach proposed in (Wei and Gulla, 2010) uses a global­ly unified index term space to encode target texts for different nodes which is deemed to limit the performance of the HL-SOT approach. There­fore, the LFS framework proposed in this paper aims at overcoming the weakness of the HL-SOT approach and consequently improving its perfor­mance by generating a locally customized index term space for each node.</p></section><section number="3" title="The HL-SOT Approach Review"><p>In the HL-SOT approach (Wei and Gulla, 2010), each target text is indexed by a vector x <i>G X, X = </i>Rd. Weight vectors Wj(1 &lt; i &lt; N) define linear-threshold classifiers of each node i in SOT so that the target text x is labeled true by node i if x is labeled true by i's parent node and Wj • x &gt; 0j. The parameters Wj and 0j are learned from the training data set: <i>D</i><i> </i><i>=</i><i> </i><i>{(r,l)\r</i><i> </i>G X, l G Y}, where Y denotes the set of label vectors. In the training process, when a new instance rt is observed, each row vector wi)t is updated by a regularized least squares estimator given by:</p><doubt alpha="50.0" length="38" tooSmall="False" monospace="0.0">wi,t=(I+Si,Q(i,t-i)Sv2(i,t-i)+r*rtT)-1</doubt><doubt alpha="43.8" length="32" tooSmall="False" monospace="0.0">XSi,Q(i,t —1))h,Î2)1i,ÎQ(i)t-l))</doubt><p>where <i>I</i><i> </i>is a <i>d</i><i> </i>x <i>d</i><i> </i>identity matrix, <i>Q(i,t</i><i> </i><i>—</i><i> </i><i>1)</i><i> </i>denotes the number of times the parent of node <i>i </i>observes a positive label before observing the in­stance rt, Si,Q(i,t<b>-i) </b>= K,r<b>^^-i)</b>] is a d x <i>Q(i,t</i><i> </i><i>—</i><i> </i><i>1)</i><i> </i>matrix whose columns are the instances a Q(i, <i>t—1)</i>-dimensional vector of the correspond­ing labels observed by node i. The Formula 1 re­stricts that the weight vector wi;t of the classifier i is only updated on the examples that are positive for its parent node. Then the label vector yrt is computed for the instance rt, before the real label vector <i>lrt </i>is observed. Then the current threshold vector <i>9t </i>is updated by:</p><doubt alpha="47.7" length="44" tooSmall="False" monospace="0.0">ri1'riQ(i,t-1),and,li,i2,■■■,li,iQ(i,t-1))is</doubt><doubt alpha="39.1" length="23" tooSmall="False" monospace="0.0">0t+1=°t+e(Vrt— lrt),(2)</doubt><page local="4" global="330"/><p>where e is a small positive real number that de­notes a corrective step for the current threshold vector <i>9t</i>. After the training process for each node of SOT, each target text is to be labeled by each node i parameterized by the weight vector <i>W</i><i>i </i>and the threshold <i>9i </i>in the hierarchical classification process.</p></section><section number="4" title="The Localized Feature Selection"><p>In this section, we propose the LFS framework to generate a locally customized index term space for each node of SOT respectively. We first discuss why localized feature selection is needed for the HL-SOT approach. Then we define the concept of local hierarchy of SOT to introduce the local feature selection scope of a node, followed by a presentation on the local hierarchy based feature selection process.</p><subsection number="4.1" title="Why Localized Feature Selection for the HL-SOT"><p>One deficiency of the HL-SOT approach is that it uses a globally unified index term space to index target texts, which cannot efficiently encode fea­ture information required by each local individual node of SOT. When we look into the detailed clas­sification process of each node of SOT, we observe the following two types of phenomena. Firstly, SOT organizes domain knowledge in a tree like structure. Within a particular domain knowledge represented by SOT, nodes that stay in differen­t branches of SOT represent independent different attributes in that domain. In this way, feature terms (e.g., the term "ergonomics") that are relevant to a node (e.g., the node "design and usability") might be irrelevant to other nodes (e.g., the node "im­age quality") that stay at another branches of SOT; Secondly, the HL-SOT approach labels each tar­get text in a hierarchical order which ensures that each target text that comes to be handled by a node has already been labeled as true by its parent n­ode. Due to this characteristic, feature terms (e.g., the term "noise") that are significant to a node i (e.g., the node "noise") might become a trivial ter­ni for i's child nodes (e.g., the nodes "noise +" and "noise -"). Therefore, the purpose of the localized feature selection is to filter out irrelevant terms that are insignificant to each individual node and build a locally customized index term space for the n­ode so that the performance of the node can be improved.</p></subsection><subsection number="4.2" title="Local Feature Selection Scope for a Node"><p>In order to select locally customized feature terms for each individual node, we need to define a suit­able scope, called local feature selection scope<footnote anchor="3"/>, within which the feature selection process can be effectively conducted for the node. Since the HL-SOT approach is a hierarchical classification pro­cess, before we introduce the local scope for a n­ode we first give a formal definition on local hier­archy of SOT.</p><p><b>Definition 1 </b><i>[Local Hierarchy] A local hierarchy Au of SOT is defined to be formed by all the child nodes of u in SOT, where the node u must be a non-leafnode ofthe SOT.</i></p><p>By the Definition 1, we say all the child nodes of u are on the same local hierarchy under u which is denoted by Au. For examples, in Fig. 2 nodes "camera +", "design and usability", "image quali­ty", "lens", "camera -" are deemed on the same lo­cal hierarchy under the node "camera" and nodes "weight +", "weight -" are deemed on the same local hierarchy under the node "weight", etc. In the hierarchical labeling process of the HL-SOT approach, after a target text is labeled as true by a node i it will go further to the local hierarchy un­der i and is to be labeled by all nodes on the local hierarchy <i>A</i><i>i.</i><i> </i>For a target text the labeling pro­cesses of nodes on <i>A</i><i>i </i>locally can be considered as a multi-label classification process where each node is a local label. Therefore, the measurement for selecting terms as features should be calculat­ed among nodes on the same hierarchy. Hence, the local scope for a node is defined within the local hierarchy which the node is on.</p></subsection><subsection number="4.3" title="Local Hierarchy Based Feature Selection"><p>In the proposed LFS framework, local feature se­lection for a node i of SOT is performed within the <i>local scope </i>of the node i. Since nodes on the same local hierarchy share the same local scope, local feature selection process for all nodes of SOT is achieved in local hierarchy based manner. Specif­ically, for the feature selection process on a local hierarchy A, let <i>c1,c2,</i><i> cK </i>denote the K nodes on <i>A. </i>Let D denote the training data set for the HL-SOT approach. Let <i>DCk </i>denote the set of in­stances in D that contains the label of the node ck (1 ^ k ^ K). Let Da denote the training cor­pus for the local hierarchy A which is the set of<page local="5" global="331"/></p><footnote label="3">In this paper, we also call it "local scope" for short.</footnote><p>Figure 2: All local hierarchies of the example SOT: the grey nodes sharing the same parent node in dashed line are called on the same local hierarchy under the parent node all instances in the training data set D that con­tain any label of nodes on the local hierarchy A: Da = <b>U </b>K<b>=1 </b>DCk. Let VCk denote the set of all the vocabularies that appears in DCk. Let <i>sCk </i><i>(w)</i><i> </i>denote the term score that measures the suitability of w as a feature for node ck. Let FCk denote the set of feature terms selected for ck. Let dCk denote the number of features to be selected in FCk. A lo­cal feature selection process for nodes on the local hierarchy A is described in Algorithm 1.</p><p><b>Algorithm 1 </b>Localized Feature Selection Algorithm</p><p><b>DATA INITIALIZATION:</b></p><p>1: <b>for </b>each node ck on A <b>do</b> 2:      Establish Dcfc containing instances being labeled true by <i>c</i><i>k; </i>3:      Establish the vocabulary set ; 4:      Remove stop words from <i>V</i><i>ck </i>; 5: <b>end for</b> 6: Establish the training corpus: Da = Ufc—1 ;</p><p><b>BEGIN </b>7: <b>for </b>each node c  on A <b>do </b>8:      <b>for </b>each term w G <i>V</i><i>Ck </i><b>do</b></p><p>with training corpus Da and data instance set <i>Dck </i>: 9: Calculate      (w) with a specified feature selection algorithm;</p><doubt alpha="60.0" length="10" tooSmall="False" monospace="0.0">10:end for</doubt><doubt alpha="64.8" length="71" tooSmall="False" monospace="0.0">11:Establish feature space       with top      terms fromVck;12:end for</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">END</doubt><p>In the data initialization phase of the Algorith­m 1, the data instance set DCk and vocabulary set <i>VCk </i>for each node on the local hierarchy A as well as the training corpus Da are established. In a lo­cal feature selection process, a term score sCk(w) for each term w g VCk can be calculated by a spec­ified feature selection algorithm, taking Da as the training corpus and DCk as the data instance set in the class ck. The local feature selection process can employ any specific feature selection algorith­m to calculate the term scores. After all terms in VCk are calculated, those terms with top dCk scores are selected to establish the feature space FCk for the node ck. Since the number of terms in VCkvaries from node to node, in order to produce a ra­tional dimensionality dCk for the established fea­ture space FCk, we introduce a feature selection rate, denoted by 7, to control dCk for each node ck, i.e- dCk = r\VCk \ x <footnote anchor="7"/>n.</p><p>After local feature selection processes for all the nods of SOT are accomplished, a locally cus­tomized index term space FCk for each node ck is established. Each target text will be respectively indexed by a customized vector xCk g xCk(xCk = rdck ) when it goes through the hierarchical classi­fication process of the HL-SOT approach. In next section, we will present the empirical analysis on evaluating the proposed LFS framework.</p></subsection></section><section number="5" title="Empirical Analysis"><p>In this section, we conduct extensive experiments to empirically analyze the proposed LFS frame­work. Our experiments are intended to address the following questions: (1) can the classifica­tion performance of the HL-SOT approach be im­proved with the LFS framework; (2) how much computational efficiency can be gained for the HL-SOT to be implemented in the LFS framework; (3) how are the comparative performances produced by different feature selection algorithms when em­ployed in the proposed LFS framework.</p><subsection number="5.1" title="Data Set Preparation"><p>We construct our data set based on the digital camera review data set used in the HL-SOT ap­proach (Wei and Gulla, 2010). In total, the con­structed data set contains 1500 snippets of cus­tomer reviews on digital cameras, where 35 at­tributes of a digital camera are mentioned in the review data.<page local="6" global="332"/> We build an ontology structure to or­ganize the mentioned attributes and label each re­view text with correspondent attributes as well as sentiments, which complying the rule that if a re­view text is assigned with a label of a node then it is assigned with a label of the parent node. We randomly divide the labeled data set into five fold­s so that each fold at least contains one example instance labeled by each attribute node. To catch the statistical significance of experimental results, we perform 5 cross-fold evaluation by using four folds as training data and the other one fold as test­ing data. All the experimental results presented in this section are averaged over 5 runs of each ex­periment.</p></subsection><subsection number="5.2" title="Evaluation Metrics"><p>Since the existing HL-SOT approach is a hierar­chical classification process, we use the same three classic loss functions (Wei and Gulla, 2010) for measuring classification performance. They are respectively the One-error Loss (O-Loss) function, the Symmetric Loss (S-Loss) function, and the Hi­erarchical Loss (H-Loss) function<footnote anchor="4"/>:</p><p>• One-error loss (O-Loss) function is defined as:</p><doubt alpha="38.5" length="26" tooSmall="False" monospace="0.0">Lo(y,l)=«B(3i: ft =It),(3)</doubt><p>where <i>y </i>is the prediction label vector and l is the true label vector; B(S) is a boolean function which is 1 if and only if the statement S is true, otherwise it is 0.</p><p>• Symmetric loss (S-Loss) function is defined as:</p><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">N</doubt><doubt alpha="45.5" length="22" tooSmall="False" monospace="0.0">Ls(y,l)=Y,B(yt=It),(4)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">i=1</doubt><p>• Hierarchical loss (H-Loss) function is defined as:</p><doubt alpha="54.8" length="31" tooSmall="False" monospace="0.0">Lh(y,l)=B(yt=liA Vj€A(i),yä=h),</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(5)</doubt><p>where A denotes a set of nodes that are ancestors of node i in SOT.</p></subsection><subsection number="5.3" title="Performance Comparison"><p>In this section, we conduct experiments to show performance improvement from the proposed LF-S framework. The performance considered here include both classification performance and com­putational efficiency. We use the existing HL-SOT approach as a baseline. Since the HL-SOT approach used terms' document frequencies (D-F) (Manning et al., 2008) algorithm to select fea­tures to build the globally unified index term s­pace, employing the same DF feature selection al­gorithm we apply the proposed LFS framework on the HL-SOT approach and call the implemented method "DF-SOT". The only difference between HL-SOT and DF-SOT is the index term space for each node of SOT, i.e., in the HL-SOT all the n­odes using the globally unified index term space while in the DF-SOT each node respectively us­ing a locally customized index term space. In this way, the performance difference between the two methods will indicate the effect of the proposed LFS framework.</p><footnote label="4">Since the three loss functions are respectively well-defined by each formula and self-explained by their names, due to the space limitation, we do not present more explana­tion.</footnote><subsubsection number="5.3.1" title="Comparison on Classification Performance"><p>We conduct experiments to investigate whether the classification performance of the HL-SOT can be improved when it is implemented with the LFS framework. Fig. 3 presents the experimental re­sults of classification accuracies between HL-SOT and DF-SOT. In the experiments, the dimension­ality d of the globally unified index term space of the HL-SOT approach is set to 270, which is large enough for the HL-SOT approach to reach its best performance level. The feature selection rate 7 for the locally customized index term space of the DF-SOT approach is set to 0.2 and 0.3, which brings respectively 80% and 70% vocabulary reduction. The value of the corrective step e is set to varying from 0.005 to 0.05 with each step of 0.005 so that each running approach can achieve its best perfor­mance with a certain value of e. From Fig. 3, we can observe that when 7 = 0.2 the DF-SOT ap­proach reaches its best performance with 0.6953 (e = 0.02) on O-Loss, 1.5516 (e = 0.045) on S-Loss, and 1.0578 (e = 0.04) on H-Loss, and that when 7 = 0.3 the DF-SOT approach reach­es its best performance with 0.6953 (e = 0.015) on O-Loss, 1.5531 (e = 0.02) on S-Loss, and 1.0547 (e = 0.025) on H-Loss, which outperform­s the best performance of the HL-SOT approach on O-Loss 0.6984 (e = 0.025), on S-Loss 1.6188 (e = 0.025), and on H-Loss 1.0969 (e = 0.05). This indicates that with the proposed LFS frame­work, compared with the HL-SOT approach, the DF-SOT approach generally improves the classifi­cation performance.</p><page local="7" global="333"/><figure caption="Figure 3: Classification Performance (A Smaller Loss Value Means Better Classification Performance)"></figure></subsubsection><subsubsection number="5.3.2" title="Comparison on Computational Efficiency"><p>We conduct further experiments to analyze com­putational efficiency gained through the proposed LFS framework. All the experiments are conduct­ed on a normal personal computer containing an Intel Pentium D CPU (2.4 GHz, Dual Core) and 4G memory. Fig. 4 summarizes the computation­al time consumed by experiment runs respectively for HL-SOT <i>(d</i><i> </i><i>=</i><i> </i>270) and DF-SOT (7 = 0.2 and 7 = 0.3). From Fig. 4, we can observe that the HL-SOT approach consumes 15917695 ms to inish an experimental run, although the DF-SOT approachonly takes respectively 2.29% (with 7 = 0.2 ) and 4.91% (with 7 = 0.2 ) of computa­tional time as the existing HL-SOT approach con­sumes and achieves even better classiication per­formance than the HL-SOT approach (see Fig.3). This conirms that much computational eficien-cy can be gained for the HL-SOT approach to be implemented in the LFS framework while better classiication performance is ensured. Since the computational complexity of each node classiier of DF-SOT is the same as HL-SOT, the compu­tational eficiency gained from the proposed LFS framework should be attributed to the dimension reduction of the index term space.</p></subsubsection></subsection><subsection number="5.4" title="Comparative Study on Feature Selection Algorithms"><p>The proposed LFS framework for the HL-SOT ap­proach can employ various feature selection algo­rithms to select local features for each individu­al node. In this section, we conduct intensive ex­periments to comparatively study ive classic fea­ture selection algorithms employed within the LFS framework. The ive employed feature selection algorithms are respectively document frequency (DF) (Manning et al., 2008) based feature selec­tion algorithm, mutual information (MI) (Manning et al., 2008; Church and Hanks, 1990) based fea­ture selection algorithm, %<footnote anchor="2"/>-statistic (CHI) (Man­ning et al., 2008) based feature selection algorith­m, information gain (IG) (Mitchell, 1997) based feature selection algorithm as well as term strength (TS) (Wilbur and Sirotkin, 1992) based feature s­election algorithm<footnote anchor="5"/>.</p><figure caption="Figure 4: Time Consuming (ms)"></figure><p>In the experiments, the feature selection rate 7 is set to 0.2 and 0.3 respectively. The value of the corrective step e varies from 0.005 to 0.05 with each step of 0.005. The experimental re­sults are summarized in Fig. 5. From Fig. 5 it is observed that DF, MI, and TS feature selec­tion algorithms achieve generally better perfor­mances than CHI and IG feature selection algo­rithms when they are employed in the proposed LFS framework. Speciically, the TS algorithm is generally the best among the ive employed algo­rithms while the DF algorithm can also achieve as<page local="8" global="334"/></p><footnote label="5">Due to the space limitation, details of the studied feature selection algorithms are not reviewed here. The mechanism of each algorithm can be read in the related references.</footnote><doubt alpha="0.0" length="242" tooSmall="True" monospace="0.0">0.005    0.01    0.015    0.02    0.025    0.00    0.005    0.04    0.045    0.05 0.005    0.01    0.015    0.02    0.025    0.00    0.005    0.04    0.045    0.05 0.005    0.01    0.015    0.02    0.025    0.00    0.005    0.04    0.045 0.05</doubt><p>Corrective Step Corrective Step Corrective Step</p><doubt alpha="34.5" length="55" tooSmall="False" monospace="0.0">(a)O-Loss(Y= 0.2) (b) S-Loss(7= 0.2) (c) H-Loss(7= 0.2)</doubt><doubt alpha="0.0" length="78" tooSmall="True" monospace="0.0">0.005    0.01    0.015    0.02    0.025    0.03    0.035    0.04    0.045 0.05</doubt><p>Corrective Step</p><doubt alpha="32.1" length="56" tooSmall="False" monospace="0.0">(d) O-Loss(7= 0.3) (e) S-Loss(7= 0.3) (f) H-Loss(7= 0.3)</doubt><figure caption="Figure 5: Comparative Performances on the Employed Feature Selection Algorithms"></figure><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">o J</doubt><doubt alpha="100.0" length="1" tooSmall="True" monospace="0.0">I</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">O</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">o-J</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">X</doubt><doubt alpha="66.7" length="3" tooSmall="True" monospace="0.0">'Sa</doubt><p>comparable good performance as the TS algorithm does. This is due to that both the TS and the DF al­gorithms favor high frequency terms and vocabu­laries used in customer reviews on a specific prod­uct are usually overlapping. When 7 = 0.3, it can be also observed that the MI algorithm achieves as comparable good performance as the TS algo­rithm does. This is because, in customer reviews, although some vocabularies are rarely used they always occur as significant features in some spe­cific categories. For example, "ergonomics" is a rare term but almost always appears in the class of "design and usability". Therefore, the MI algorith­m can also achieve relatively better performance through favoring rare terms that always co-occur with specific classes.</p></subsection></section><section number="6" title="Conclusions and Future Work"><p>In this paper, we propose a LFS framework tai­lored to the HL-SOT approach to sentiment analy­sis. In the proposed LFS framework, significan­t feature terms of each node can be selected to construct the locally customized index term space for the node so that the classification performance and computational efficiency of the existing HL-SOT approach are improved. The effectiveness of the proposed LFS is validated against a human­labeled data set.  Further comparative study on five employed feature selection algorithms with­in the proposed LFS framework indicates that the TS, DF, and MI algorithms achieve generally bet­ter performance than the CHI and IG algorithms. Among the five employed algorithms, the TS algo­rithm is the best to be employed by the proposed LFS framework.</p><p>Although the proposed LFS framework shows its effectiveness of improving on the HL-SOT ap­proach, its improvement on the classification per­formance is not so obvious compared with its much improvement on computational efficiency. Due to the limited number of instances in the train­ing data set, the classification performance stil­l suffers from the problem that unobserved terms appear in testing cases. This problem is inherent­ly raised by the bag-of-word model. A concept-based indexing scheme that can infer concepts of unobserved terms might alleviate the problem. We plan to investigate on this issue in the future work.</p></section><section title="Acknowledgments"><p>The authors would like to thank the anony­mous reviewers for the helpful comments on the manuscript. This work is funded by the Research Council of Norway under the VERDIKT research programme (Project No.: 183337).</p><page local="9" global="335"/></section><references><p>Alina Andreevskaia and Sabine Bergler. 2006. Min­ing wordnet for a fuzzy sentiment: Sentiment tag extraction from wordnet glosses. In <i>Proceedings of 11th Conference of the European Chapter of the As­sociation for Computational Linguistics.</i></p><p>Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicog­raphy. <i>Computational Linguistics, </i>16(1):22-29.</p><p>Ann Devitt and Khurshid Ahmad. 2007. Sentiment polarity identification in financial news: A cohesion-based approach. In <i>Proceedings of 45th Annual Meeting ofthe Association for Computational Lin­guistics.</i></p><p>Xiaowen Ding and Bing Liu. 2007. The utility of lin­guistic rules in opinion mining. In <i>Proceedings of the 30th Annual International ACM SIGIR Confer­ence on Research and development in Information Retrieval.</i></p><p>Andrea Esuli and Fabrizio Sebastiani. 2005. Deter­mining the semantic orientation of terms through gloss classification. In <i>Proceedings of 14th ACM Conference on Information and Knowledge Man­agement.</i></p><p>Andrea Esuli and Fabrizio Sebastiani. 2006. Senti-wordnet: A publicly available lexical resource for opinion mining. In <i>Proceedings of5th International Conference on Language Resources and Evaluation.</i></p><p>Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjec­tives. In <i>Proceedings of35th Annual Meeting ofthe Association for Computational Linguistics.</i></p><p>Vasileios Hatzivassiloglou and Janyce M. Wiebe. 2000. Effects of adjective orientation and grad-ability on sentence subjectivity. In <i>Proceedings of 18th International Conference on Computational Linguistics.</i></p><p>Minqing Hu and Bing Liu. 2004. Mining and sum­marizing customer reviews. In <i>Proceedings of10th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.</i></p><p>Jaap Kamps, Maarten Marx, R. ort. Mokken, and Maarten de Rijke. 2004. Using WordNet to mea­sure semantic orientation of adjectives. In <i>Proceed­ings of4th International Conference on Language Resources and Evaluation.</i></p><p>Bing Liu, Minqing Hu, and Junsheng Cheng. 2005. Opinion observer: analyzing and comparing opin­ions on the web. In <i>Proceedings of 14th Interna­tional World Wide Web Conference.</i></p><p>Yang Liu, Xiangji Huang, Aijun An, and Xiaohui Yu. 2007. ARSA: a sentiment-aware model for predict­ing sales performance using blogs. In <i>Proceedings</i> <i>ofthe 30th Annual International ACM SIGIR Con­ference on Research and development in Information Retrieval.</i></p><p>Yue Lu and Chengxiang Zhai. 2008. Opinion inte­gration through semi-supervised topic modeling. In <i>Proceedings of17th International World Wide We­b Conference.</i><i></i></p><p>Yue Lu, ChengXiang Zhai, and Neel Sundaresan. 2009. Rated aspect summarization of short com­ments. In <i>Proceedings of18th International World Wide Web Conference.</i></p><p>Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze, 2008. <i>Introduction to Information Retrieval, </i>chapter 13, pages 271-278. Cambridge University Press.</p><p>Tom Mitchell. 1997. <i>Machine Learning. </i>McGraw-</p><p>Hill.</p><p>Ana-Maria Popescu and Oren Etzioni. 2005. Extract­ing product features and opinions from reviews. In <i>Proceedings ofHuman Language Technology Con­ference and Empirical Methods in Natural Lan­guage Processing Conference.</i><i></i></p><p>Ivan Titov and Ryan T. McDonald. 2008. Modeling online reviews with multi-grain topic models. In</p><p>Peter D. Turney. 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervised classi­fication of reviews. In <i>Proceedings of 40th Annual Meeting ofthe Association for Computational Lin­guistics.</i></p><p>Wei Wei and Jon Atle Gulla. 2010. Sentiment learning on product reviews via sentiment ontology tree. In <i>Proceedings of48th Annual Meeting ofthe Associa­tion for Computational Linguistics.</i><i></i></p><p>Casey Whitelaw, Navendu Garg, and Shlomo Arga-mon. 2005. Using appraisal taxonomies for senti­ment analysis. In <i>Proceedings of14th ACMConfer-ence on Information and Knowledge Management.</i></p><p>J. W. Wilbur and K. Sirotkin. 1992. The automatic i­dentification of stop words. <i>Journal of the American Society for Information Science, </i>18:45-55.</p><p>Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase­level sentiment analysis. In <i>Proceedings of Hu­man Language Technology Conference and Empir­ical Methods in Natural Language Processing Con­ference.</i></p><p>Hong Yu and Vasileios Hatzivassiloglou. 2003. To­wards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In <i>Proceedings of8th Conference on Em­pirical Methods in Natural Language Processing.</i></p></references></body></article>