<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="45"/><title>Holistic Sentiment Analysis Across Languages: Multilingual Supervised Latent Dirichlet Allocation</title><pubinfo>Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,pages 45-55, MIT, Massachusetts, USA, 9-11 October 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Boyd-Graber" givenname="Jordan"><org  name="University of Maryland" country="USA" city="College Park"/></author><author surname="Resnik" givenname="Philip"><org  name="University of Maryland" country="USA" city="College Park"/></author></firstpageheader><frontmatter><p><b>Holistic Sentiment Analysis Across Languages: Multilingual Supervised Latent Dirichlet Allocation</b></p><p><b>Jordan Boyd-Graber</b></p><p>UMD iSchool and UMIACS University of Maryland College Park, MD jbg@umiacs.umd.edu</p><p><b>Philip Resnik</b></p><p>Department of Linguistics</p><p>and UMIACS</p><p>University of Maryland</p><p>College Park, MD</p><p>resnik@umd.edu</p></frontmatter><abstract></abstract></header><body><section title=""><p>In this paper, we develop multilingual super­vised latent Dirichlet allocation (MlSLDA), a probabilistic generative model that allows insights gleaned from one language's data to inform how the model captures properties of other languages. MlSLDA accomplishes this by jointly modeling two aspects of text: how multilingual concepts are clustered into themat-ically coherent topics and how topics associ­ated with text connect to an observed regres­sion variable (such as ratings on a sentiment scale). Concepts are represented in a general hierarchical framework that is flexible enough to express semantic ontologies, dictionaries, clustering constraints, and, as a special, degen­erate case, conventional topic models. Both the topics and the regression are discovered via posterior inference from corpora. We show MlSLDA can build topics that are consistent across languages, discover sensible bilingual lexical correspondences, and leverage multilin­gual corpora to better predict sentiment.</p><p>Sentiment analysis (Pang and Lee, 2008) offers the promise of automatically discerning how people feel about a product, person, organization, or issue based on what they write online, which is potentially of great value to businesses and other organizations. However, the vast majority of sentiment resources and algorithms are limited to a single language, usu­ally English (Wilson, 2008; Baccianella and Sebas-tiani, 2010). Since no single language captures a majority of the content online, adopting such a lim­ited approach in an increasingly global community risks missing important details and trends that might only be available when text in multiple languages is taken into account.</p><p>Up to this point, multiple languages have been addressed in sentiment analysis primarily by trans­ferring knowledge from a resource-rich language to a less rich language (Banea et al., 2008), or by ig­noring differences in languages via translation into English (Denecke, 2008). These approaches are lim­ited to a view of sentiment that takes place through an English-centric lens, and they ignore the poten­tial to share information between languages. Ide­ally, learning sentiment cues holistically, <i>across </i>lan­guages, would result in a richer and more globally consistent picture.</p><p>In this paper, we introduce Multilingual Super­vised Latent Dirichlet Allocation (MlSLDA), a model for sentiment analysis on a multilingual cor­pus. MlSLDA discovers a consistent, unified picture of sentiment across multiple languages by learning "topics," probabilistic partitions of the vocabulary that are consistent in terms of both meaning and rel­evance to observed sentiment. Our approach makes few assumptions about available resources, requiring neither parallel corpora nor machine translation.</p><p>The rest of the paper proceeds as follows. In Sec­tion 1, we describe the probabilistic tools that we use to create consistent topics bridging across languages and the MlSLDA model. In Section 2, we present the inference process. We discuss our set of seman­tic bridges between languages in Section 3, and our experiments in Section 4 demonstrate that this ap­proach functions as an effective multilingual topic model, discovers sentiment-biased topics, and uses multilingual corpora to make better sentiment pre­dictions across languages. Sections 5 and 6 discuss related research and discusses future work, respec­tively.</p><page local="2" global="46"/></section><section number="1" title="Predictions from Multilingual Topics"><p>As its name suggests, MlSLDA is an extension of Latent Dirichlet allocation (LDA) (Blei et al., 2003), a modeling approach that takes a corpus of unan-notated documents as input and produces two out­puts, a set of "topics" and assignments of documents to topics. Both the topics and the assignments are probabilistic: a topic is represented as a probability distribution over words in the corpus, and each doc­ument is assigned a probability distribution over all the topics. Topic models built on the foundations of LDA are appealing for sentiment analysis because the learned topics can cluster together sentiment-bearing words, and because topic distributions are a parsimonious way to represent a document.<footnote anchor="1"/></p><p>LDA has been used to discover latent structure in text (e.g. for discourse segmentation (Purver et al., 2006) and authorship (Rosen-Zvi et al., 2004)). MlSLDA extends the approach by ensuring that this latent structure — the underlying topics — is consis­tent across languages. We discuss multilingual topic modeling in Section 1.1, and in Section 1.2 we show how this enables supervised regression regardless of a document's language.</p><subsection number="1.1" title="Capturing Semantic Correlations"><p>Topic models posit a straightforward generative pro­cess that creates an observed corpus. For each docu­ment d, some distribution <i>9d </i>over unobserved topics is chosen. Then, for each word position in the doc­ument, a topic z is selected. Finally, the word for that position is generated by selecting from the topic indexed by z. (Recall that in LDA, a "topic" is a distribution over words).</p><p>In monolingual topic models, the topic distribution is usually drawn from a Dirichlet distribution. Us­ing Dirichlet distributions makes it easy to specify sparse priors, and it also simplifies posterior infer­ence because Dirichlet distributions are conjugate to multinomial distributions. However, drawing top­ics from Dirichlet distributions will not suffice if our vocabulary includes multiple languages. If we are working with English, German, and Chinese at the same time, a Dirichlet prior has no way to fa­vor distributions z suchthat p(good|z), p(gut|z), and p(hao|z) all tend to be high at the same time, or low at the same time. More generally, the structure of our model must encourage topics to be consistent across languages, and Dirichlet distributions cannot encode correlations between elements.</p><footnote label="1">The latter property has also made LDA popular for infor­mation retrieval (Wei and Croft, 2006)).</footnote><p>One possible solution to this problem is to use the multivariate normal distribution, which can produce correlated multinomials (Blei and Lafferty, 2005), in place of the Dirichlet distribution. This has been done successfully in multilingual settings (Cohen and Smith, 2009). However, such models complicate inference by not being conjugate.</p><p>Instead, we appeal to tree-based extensions of the Dirichlet distribution, which has been used to induce correlation in semantic ontologies (Boyd-Graber et al., 2007) and to encode clustering constraints (An-drzejewski et al., 2009). The key idea in this ap­proach is to assume the vocabularies of all languages are organized according to some shared semantic structure that can be represented as a tree. For con-creteness in this section, we will use WordNet (Miller, 1990) as the representation of this multilingual se­mantic bridge, since it is well known, offers conve­nient and intuitive terminology, and demonstrates the full flexibility of our approach. However, the model we describe generalizes to any tree-structured rep­resentation of multilingual knowledge; we discuss some alternatives in Section 3.</p><p>WordNet organizes a vocabulary into a rooted, di­rected acyclic graph of nodes called synsets, short for "synonym sets." A synset is a child of another synset if it satisfies a hyponomy relationship; each child "is a" more specific instantiation of its parent concept (thus, hyponomy is often called an "isa" relationship). For example, a "dog" is a "canine" is an "animal" is a "living thing," etc. As an approximation, it is not unreasonable to assume that WordNet's structure of meaning is language independent, i.e. the concept encoded by a synset can be realized using terms in different languages that share the same meaning. In practice, this organization has been used to create many alignments of international WordNets to the original English WordNet (Ordan and Wintner, 2007; Sagot and Fiser, 2008; Isahara et al., 2008).</p><p>Using the structure of WordNet, we can now de­scribe a generative process that produces a distribu­tion over a multilingual vocabulary, which encour­ages correlations between words with similar meanings regardless of what language each word is in.<page local="3" global="47"/> For each synset h, we create a multilingual word distribution for that synset as follows:</p></subsection></section><section number="1." title="Draw transition probabilities ß h ~ Dir (r h )"></section><section number="2." title="Draw stop probabilities uj h  ~ Dir (n h )"><p>3. For each language l, draw emission probabilities for that synset <i>4</i><i>&gt;h,i </i>~ Dir <i>(n</i><i>h,i).</i></p><p>For conciseness in the rest of the paper, we will refer to this generative process as <i>multilingual Dirichlet hierarchy, </i>or MultDirHier(t, k, n).<footnote anchor="2"/> Each ob­served token can be viewed as the end result of a sequence of visited synsets A. At each node in the tree, the path can end at node i with probability or it can continue to a child synset with probability wi;0. If the path continues to another child synset, it visits child j with probability ßi;j. If the path ends at a synset, it generates word k with probability <i>&lt;fi</i><i>iti,k</i>.<i><footnote anchor="3"/></i>The probability of a word being emitted from a path with visited synsets r and final synset h in language l is therefore</p><doubt alpha="35.0" length="20" tooSmall="False" monospace="0.0">p(w,A=r,h|1,ß,u, 0)=</doubt><doubt alpha="45.2" length="31" tooSmall="False" monospace="0.0">Wßi,jWi,0J(1 -Uh,l)&lt;f&gt;h,l,w■(1)</doubt><p>Note that the stop probability <i>uoh </i>is independent of language, but the emission &lt;h l is dependent on the language. This is done to prevent the following sce­nario: while synset A is highly probable in a topic and words in language 1 attached to that synset have high probability, words in language 2 have low prob­ability. If this could happen for many synsets in a topic, an entire language would be effectively si­lenced, which would lead to inconsistent topics (e.g.</p><footnote label="2">Variables T h , and K h  are hyperparameters. Their mean is fixed, but their magnitude is sampled during inference (i.e. sr T h, ' i — is constant, but T h  i is not). For the bushier bridges,</footnote><doubt alpha="50.0" length="6" tooSmall="True" monospace="0.0">Th,k '</doubt><p>(e.g. dictionary and flat), their mean is uniform. For GermaNet, we took frequencies from two balanced corpora of German and English: the British National Corpus (University of Oxford, 2006) and the Kern Corpus of the Digitales Wörterbuch der Deutschen Sprache des 20. Jahrhunderts project (Geyken, 2007). We took these frequencies and propagated them through the multilingual hierarchy, following LDAWN's (Boyd-Graber et al., 2007) formulation of information content (Resnik, 1995) as a Bayesian prior. The variance of the priors was initialized to be 1.0, but could be sampled during inference.</p><footnote label="3">Note that the language and word are taken as given, but the path through the semantic hierarchy is a latent random variable.</footnote><p>Topic 1 is about baseball in English and about travel in German). Separating path from emission helps ensure that topics are consistent across languages.</p><p>Having defined topic distributions in a way that can preserve cross-language correspondences, we now use this distribution within a larger model that can discover cross-language patterns of use that predict sentiment.</p><subsection number="1.2" title="The MlSLDA Model"><p>We will view sentiment analysis as a regression prob­lem: given an input document, we want to predict a real-valued observation y that represents the senti­ment of a document. Specifically, we build on super­vised latent Dirichlet allocation (SLDA, (Blei and McAuliffe, 2007)), which makes predictions based on the topics expressed in a document; this can be thought of projecting the words in a document to low dimensional space of dimension equal to the number of topics. Blei et al. showed that using this latent topic structure can offer improved predictions over re­gressions based on words alone, and the approach fits well with our current goals, since word-level cues are unlikely to be identical across languages. In addition to text, SLDA has been successfully applied to other domains such as social networks (Chang and Blei, 2009) and image classification (Wang et al., 2009). The key innovation in this paper is to extend SLDA by creating topics that are globally consistent across languages, using the bridging approach above.</p><p>We express our model in the form of a probabilis­tic generative latent-variable model that generates documents in multiple languages <i>and </i>assigns a real-valued score to each document. The score comes from a normal distribution whose sum is the dot prod­uct between a regression parameter n that encodes the influence of each topic on the observation and a variance <i>a<footnote anchor="2"/></i>. With this model in hand, we use sta­tistical inference to determine the distribution over latent variables that, given the model, best explains observed data. The generative model is as follows:</p><p>1. For each topic i = 1 ■ ■ ■ <i>K</i>, draw a topic distribution <i>[ß</i><i>i, </i><i>U</i><i>i, </i><i>fa}</i><i> </i>from MultDirHier(t, k, n).</p><doubt alpha="66.7" length="45" tooSmall="False" monospace="0.0">2. For each documentd=1...M with language 1d:</doubt><p>(a) Choose a distribution over topics <i>6d ~ </i>Dir (a).</p><page local="4" global="48"/></subsection></section><section number="3." title="Choose a"><p>Norm (nTz.</p><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">Zd</doubt><p>response variable <i>a<footnote anchor="2"/>)</i>, where zd = <i>2</i><i>^n=l zd,n.</i><i></i></p><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">fromy</doubt><p>Crucially, note that the topics are not indepen­dent of the sentiment task; the regression encourages terms with similar effects on the observation <i>y </i>to be in the same topic. The consistency of topics de­scribed above allows the same regression to be done for the entire corpus regardless of the language of the underlying document.</p></section><section number="2" title="Inference"><p>Finding the model parameters most likely to explain the data is a problem of statistical inference. We em­ploy stochastic EM (Diebolt and Ip, 1996), using a Gibbs sampler for the E-step to assign words to paths and topics. After randomly initializing the topics, we alternate between sampling the topic and path of a word <i>(zd,n,</i><i> </i>Ad,ra) and finding the regression pa­rameters n that maximize the likelihood. We jointly sample the topic and path conditioning on all of the other path and document assignments in the corpus, selecting a path and topic with probability</p><doubt alpha="44.1" length="34" tooSmall="False" monospace="0.0">p(zn=k, An=r|z_n,A_n,Wn, n, a,6) =</doubt><doubt alpha="50.0" length="54" tooSmall="False" monospace="0.0">p(yd\z,n,a)p(An=r|zn=k,A_n,Wn,t,k, n)p(zn=k|z_n,a).(2)</doubt><p>Each of these three terms reflects a different influence on the topics from the vocabulary structure, the doc­ument's topics, and the response variable. In the next paragraphs, we will expand each of them to derive the full conditional topic distribution.</p><p>As discussed in Section 1.1, the structure of the topic distribution encourages terms with the same meaning to be in the same topic, even across lan­guages. During inference, we marginalize over pos­sible multinomial distributions ß, w, and &lt;&lt;, using the observed transitions from i to j in topic k; Tk i j, stop counts in synset i in topic k, Ok i 0; continue counts in synsets i in topic k, Ok)i;1; and emission counts in synset i in language l in topic k,    i l. The</p><doubt alpha="87.7" length="57" tooSmall="True" monospace="0.0">Multilingual Topics Text Documents   Sentiment Prediction</doubt><p>Figure 1: Graphical model representing MlSLDA. Shaded nodes represent observations, plates denote repli­cation, and lines show probabilistic dependencies.</p><p>probability of taking a path r is then</p><doubt alpha="52.9" length="17" tooSmall="False" monospace="0.0">p(An=r|zn=k,A_n)=</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">n</doubt><doubt alpha="42.9" length="7" tooSmall="False" monospace="0.0">(i,j)€r</doubt><doubt alpha="60.0" length="10" tooSmall="False" monospace="0.0">°fc,i,i+wi</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">Bk,i,j+</doubt><doubt alpha="47.1" length="34" tooSmall="False" monospace="0.0">j'Bk,i,j'+        EsGO^Ok,i,s+wi,s</doubt><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">)</doubt><p>Transition</p><doubt alpha="66.7" length="33" tooSmall="False" monospace="0.0">Ok,r£„d,0+wrendFk,rend,wn+nr£nd,l</doubt><doubt alpha="53.2" length="47" tooSmall="False" monospace="0.0">Es€0,1°k,rend,s+wr£„d,^w'Frend,w'+nrend,w'v-*-'</doubt><p>Emission</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(3)</doubt><p>Equation 3 reflects the <i>multilingual </i>aspect of this model. The conditional topic distribution for SLDA (Blei and McAuliffe, 2007) replaces this term with the standard Multinomial-Dirichlet. However, we believe this is the first published SLDA-style model using MCMC inference, as prior work has used variational inference (Blei and McAuliffe, 2007;</p><p>Chang and Blei, 2009; Wang et al., 2009).</p><p>Because the observed response variable depends on the topic assignments of a document, the condi­tional topic distribution is shifted toward topics that explain the observed response. Topics that move the predicted response yd toward the true yd will be fa­vored. We drop terms that are constant across all</p><p>(b) For each word in the document n = 1 <i>...Nd, </i>choose a topic assignment zd_„ ~ Mult <i>(6d) </i>and a path Ad n ending at word wd n according </p><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">to Equa</doubt><p><i>Zd n</i><i>, </i><i>4&gt;Zd</i><i> </i><i>n</i><i> </i>}.</p><page local="5" global="49"/><p>topics for the effect of the response variable,</p></section><section number="3" title="Bridges Across Languages"><doubt alpha="46.2" length="13" tooSmall="False" monospace="0.0">p(yd|z,n,a)«1</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">exp</doubt><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">a2</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">yd-</doubt><doubt alpha="63.6" length="11" tooSmall="False" monospace="0.0">Ek'Nd,k'nk'</doubt><p>Other words' influence</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(4)</doubt><p>This word's influence</p><p>The above equation represents the <i>supervised </i>aspect of the model, which is inherited from SLDA.</p><p>Finally, there is the effect of the topics already assigned to a document; the conditional distribution favors topics already assigned in a document,</p><doubt alpha="50.0" length="14" tooSmall="False" monospace="0.0">p(zn=k|z_n,a)=</doubt><doubt alpha="50.0" length="8" tooSmall="False" monospace="0.0">Td,k+ «k</doubt><doubt alpha="58.3" length="12" tooSmall="False" monospace="0.0">Ek'Td,k'+ak'</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(5)</doubt><p>This term represents the <i>document </i>focus of this model; it is present in all Gibbs sampling inference schemes for LDA (Griffiths and Steyvers, 2004).</p><p>Multiplying together Equations 3, 4, and 5 allows us to sample a topic using the conditional distribution from Equation 2, based on the topic and path of the other words in all languages. After sampling the path and topic for each word in a document, we then find new regression parameters n that maximize the likelihood conditioned on the current state of the sampler. This is simply a least squares regression using the topic assignments zd to predict yd.</p><p>Prediction on documents for which we don't have an observed y<i>d </i>is equivalent to marginalizing over y<i>d </i>and sampling topics for the document from Equa­tions 3 and 5. The prediction for y<i>d </i>is then the dot product of n and the empirical topic distribution zd.</p><p>We initially optimized all hyperparameters using slice sampling. However, we found that the regres­sion variance a was not stable. Optimizing a seems to balance between modeling the language in the doc­uments and the prediction, and thus is sensitive to documents' length. Given this sensitivity, we did not optimize a for our prediction experiments in Section 4, but instead kept it fixed at 0.25. We leave optimizing this variable, either through cross valida­tion or adapting the model, to future work.</p><p>In Section 1.1, we described connections across lan­guages as offered by semantic networks in a general way, using WordNet as an example. In this section, we provide more specifics, as well as alternative ways of building semantic connections across languages.</p><p><b>Flat </b>First, we can consider a degenerate mapping that is nearly equivalent to running SL DA indepen­dently across multiple languages, relating topics only based on the impact on the response variable. Con­sider a degenerate tree with only one node, with all words in all languages associated with that node. This is consistent with our model, but there is really no shared semantic space, as all emitted words must come from this degenerate "synset" and the model only represents the output distribution for this single node.</p><p><b>WordNet </b>We took the alignment of GermaNet to WordNet 1.6 (Kunze and Lemnitzer, 2002) and re­moved all synsets that were had no mapped German words. Any German synsets that did not have English translations had their words mapped to the lowest extant English hypernym (e.g. "beinbruch," a bro­ken leg, was mapped to "fracture"). We stemmed all words to account for inflected forms not being present (Porter and Boulton, 1970). An example of the paths for the German word "wunsch" (wish, request) is shown in Figure 2(a).</p><p><b>Dictionaries </b>A dictionary can be viewed as a many to many mapping, where each entry ei maps one or more words in one language <i>si </i>to one or more words <i>ti </i>in another language. Entries were taken from an English-German dictionary (Richter, 2008) a Chinese-English dictionary (Denisowski, 1997), and a Chinese-German dictionary (Hefti, 2005). As with WordNet, the words in entries for English and German were stemmed to improve coverage. An example for German is shown in Figure 2(b).</p><p><b>Algorithmic Connections </b>In addition to hand-curated connections across languages, one could also consider automatic means of mapping across lan­guages, such as using edit distance or local con­text (Haghighi et al., 2008; Rapp, 1995) or us­ing a lexical translation table obtained from paral­lel text (Melamed, 1998). While we experimented<page local="6" global="50"/></p><p>Figure 2: Two methods for constructing multilingual distributions over words. On the left, paths to the German word "wunsch" in GermaNet are shown. On the right, paths to the English word "room" are shown. Both English and German words are shown; some internal nodes in GermaNet have been omitted for space (represented by dashed lines). Note that different senses are denoted by different internal paths, and that internal paths are distinct from the per-language expression.</p><p>with these techniques, constructing appropriate hier­archies from these resources required many arbitrary decisions about cutoffs and which words to include. Thus, we do not consider them in this paper.</p></section><section number="4" title="Experiments"><p>We evaluate MlSLDA on three criteria: how well it can discover consistent topics across languages for matching parallel documents, how well it can discover sentiment-correlated word lists from non-aligned text, and how well it can predict sentiment.</p><subsection number="4.1" title="Matching on Multilingual Topics"><p>We took the 1996 documents from the Europarl cor­pus (Koehn, 2005) using three bridges: GermaNet, dictionary, and the uninformative flat matching.<footnote anchor="4"/> The model is unaware that the translations of documents in one language are present in the other language. Note that this does not use the supervised framework (as there is no associated response variable for Eu-roparl documents); this experiment is to demonstrate the effectiveness of the multilingual aspect of the model. To test whether the topics learned by the model are consistent across languages, we represent each document using the probability distribution <i>9d</i>over topic assignments. Each <i>9d </i>is a vector of length K and is a language-independent representation of the document.</p><footnote label="4">For English and German documents in all experiments, we removed stop words (Loper and Bird, 2002), stemmed words (Porter and Boulton, 1970), and created a vocabulary of the most frequent 5000 words per language (this vocabulary limit was mostly done to ensure that the dictionary-based bridge was of manageable size). Documents shorter than fifty content words were excluded.</footnote><p>For each document in one language, we computed the Hellinger distance between it and all of the docu­ments in the other language and sorted the documents by decreasing distance. The translation of the docu­ment is somewhere in that set; the higher the normal­ized rank (the percentage of documents with a rank lower than the translation of the document), the better the underlying topic model connects languages.</p><p>We compare three bridges against what is to our knowledge the only other topic model for unaligned text, Multilingual Topics for Unaligned Text (Boyd-<page local="7" global="51"/></p><doubt alpha="54.2" length="24" tooSmall="False" monospace="0.0">Graber and Blei, 2009).5</doubt><footnote label="5">The bipartite matching was initialized with the dictionary weights as specified by the Multilingual Topics for Unaligned Text algorithm. The matching size was limited to 250 and the bipartite matching was only updated on the initial iteration then held fixed. This yielded results comparable to when the matching</footnote><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">■gm</doubt><p>Dictionary -MuTo -GermaNet -Flat-</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">en</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">ai o</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">ai</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">IIIII</doubt><doubt alpha="0.0" length="19" tooSmall="False" monospace="0.0">0.0 0.2 0.4 0.6 0.8</doubt><p>Average Parallel Document Rank</p><p>Figure 3: Average rank of paired translation document recovered from the multilingual topic model. Random guessing would yield 0.5; MlSLDA with a dictionary based matching performed best.</p><p>Figure 3 shows the results of this experiment. The dictionary-based bridge had the best performance on the task, ranking a large proportion of documents (0.95) below the translated document once enough topics were available. Although GermaNet is richer, its coverage is incomplete; the dictionary structure had a much larger vocabulary and could build a more complete multilingual topics. Using comparable in­put information, this more flexible model performed better on the matching task than the existing multi­lingual topic model available for unaligned text. The degenerate flat bridge did no better than the baseline of random guessing, as expected.</p></subsection><subsection number="4.2" title="Qualitative Sentiment-Correlated Topics"><p>One of the key tasks in sentiment analysis has been the collection of lists of words that convey senti­ment (Wilson, 2008; Riloff et al., 2003). These resources are often created using or in reference to resources like WordNet (Whitelaw et al., 2005; Baccianella and Sebastiani, 2010). MlSLDA pro­vides a method for extracting topical and sentiment-correlated word lists from multilingual corpora. If</p><p>was updated more frequently.</p><p>a WordNet-like resource is used as the bridge, the resulting topics are distributions over synsets, not just over words.</p><p>As our demonstration corpus, we used the Amherst Sentiment Corpus (Constant et al., 2009), as it has documents in multiple languages (English, Chinese, and German) with numerical assessments of senti­ment (number of stars assigned to the review). We segmented the Chinese text (Tseng et al., 2005) and used a classifier trained on character n-grams to re­move English-language documents that were mixed in among the Chinese and German language reviews.</p><p>Figure 4 shows extracted topics from German-English and German-Chinese corpora. MlSLDA is able to distinguish sentiment-bearing topics from content bearing topics. For example; in the German-English corpus, "food" and "children" topics are not associated with a consistent sentiment signal, while "religion" is associated with a more negative sentiment. In contrast, in the German-Chinese cor­pus, the "religion/society" topic is more neutral, and the gender-oriented topic is viewed more negatively. Negative sentiment-bearing topics have reasonable words such as "pages," "kong pa" (Chinese for "I'm afraid that... ") and "tuo" (Chienese for "discard"), and positive sentiment-bearing topics have reason­able words such as "great," "good," and "juwel" (Ger­man for "jewel").</p><p>The qualitative topics also betray some of the weaknesses of the model. For example, in one of the negative sentiment topics, the German word "gut" (good) is present. Because topics are distributions over words, they can encode the presence of nega­tions like "kein" (no) and "nicht" (not), but not collo­cations like "nicht gut." More elaborate topic models that can model local syntax and collocations (John­son, 2010) provide options for addressing such prob­lems.</p><p>We do not report the results for sentiment predic­tion for this corpus because the baseline of predicting a positive review is so strong; most algorithms do ex­tremely well by always predicting a positive review, ours included.</p></subsection><subsection number="4.3" title="Sentiment Prediction"><p>We gathered 330 film reviews from a German film review site (Vetter et al., 2000) and combined them with a much larger English film review corpus of over<page local="8" global="52"/></p><doubt alpha="100.0" length="2" tooSmall="True" monospace="0.0">Ol</doubt><doubt alpha="0.0" length="4" tooSmall="False" monospace="0.0">-0.8</doubt><doubt alpha="14.3" length="7" tooSmall="False" monospace="0.0">~~T~0.8</doubt><doubt alpha="72.2" length="97" tooSmall="True" monospace="0.0">(harry) harry (volume) band (sky) himmel (universe) all (vampire) vampir (last) letzt (part) teil</doubt><doubt alpha="65.8" length="79" tooSmall="False" monospace="0.0">PÊ^U (harry) &amp; (belt)^(sky)115(both)n|i(section)MM%(vampire)SfJ(strong)më(last)</doubt><doubt alpha="77.0" length="135" tooSmall="True" monospace="0.0">(god) gott (lord) herr (religion) religion (universe) all (world) welt (science) wissenschaft (medicine) medizin (society) gesellschaft</doubt><doubt alpha="37.5" length="8" tooSmall="True" monospace="0.0">±$ (god)</doubt><doubt alpha="54.5" length="11" tooSmall="True" monospace="0.0">5fc£ (lord)</doubt><doubt alpha="44.4" length="9" tooSmall="True" monospace="0.0">£15(both)</doubt><doubt alpha="75.0" length="32" tooSmall="True" monospace="0.0">(religion) (science) (community)</doubt><doubt alpha="0.0" length="22" tooSmall="False" monospace="0.0">-1.6 -0.8"^--^--^0.0 ^</doubt><doubt alpha="16.7" length="12" tooSmall="False" monospace="0.0">y~ -^4 0.4*f</doubt><doubt alpha="67.1" length="82" tooSmall="True" monospace="0.0">(book) buch (itself) sich (that) dass (much) viel (no) kein (good) gut (when) wenn</doubt><doubt alpha="50.0" length="8" tooSmall="True" monospace="0.0">" (book)</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">1(both)</doubt><doubt alpha="63.6" length="22" tooSmall="False" monospace="0.0">ASKh(mostly) "% (book)</doubt><doubt alpha="71.6" length="67" tooSmall="True" monospace="0.0">(woman) frau (point) punkt (man) mann (equal) gleich (fast) schnell</doubt><doubt alpha="60.9" length="64" tooSmall="False" monospace="0.0">(good) gut (quick)        (sentence) satzj£(a little) (two) zwei</doubt><p>(woman) (story) story mA (man) (treasure) schatz t (female) (attractive) attraktiv <i>m </i>(male) (elegant) elegant &amp;1Ê (female)        (gem) juwel</p><p>£F (good) S (set) S (treasure) ! (handsome) M (both) <i>$M</i><i> </i>(story) jJ\ (small)</p><p>(a) German / English (b) German / Chinese</p><p>Figure 4: Topics, along with associated regression coefficient n from a learned 25-topic model on German-English (left) and German-Chinese (right) documents. Notice that theme-related topics have regression parameter near zero, topics discussing the number of pages have negative regression parameters, topics with "good," "great," "hao" (good) and "Uberzeugt" (convinced) have positive regression parameters. For the German-Chinese corpus, note the presence of "gut" (good) in one of the negative sentiment topics, showing the difficulty of learning collocations.</p><p>Table 1: Mean squared error on a film review corpus. All results are on the same German test data, varying the training data. Over-fitting prevents the model learning on the German data alone; adding English data to the mix allows the model to make better predictions.</p><p>5000 film reviews (Pang and Lee, 2005) to create a multilingual film review corpus.<footnote anchor="6"/></p><p>The results for predicting sentiment in German documents with 25 topics are presented in Table 1. On a small monolingual corpus, prediction is very poor. The model over-fits, especially when it has the entire vocabulary to select from. The slightly better performance using GermaNet and a dictionary as topic priors can be viewed as basic feature selec­tion, removing proper names from the vocabulary to prevent over-fitting.</p><footnote label="6">We followed Pang and Lee's method for creating a nu­merical score between 0 and 1 from a star rating. We then converted that to an integer by multiplying by 100; this was done because initial data preprocessing assumed integer values (although downstream processing did not as­sume integer values). The German movie review corpus is available at http://www.umiacs.umd.edu/~jbg/ static/downloads_and_media.html</footnote><p>One would expect that prediction improves with a larger training set. For this model, such an improve­ment is seen even when the training set includes no documents in the target language. Note that even the degenerate flat bridge across languages provides use­ful information. After introducing English data, the model learns to prefer smaller regression parameters (this can be seen as a form of regularization).</p><p>Performance is best when a reasonably large cor­pus is available including some data in the target language. For each bridge, performance improves dramatically, showing that MlSLDA is successfully able to incorporate information learned from both languages to build a single, coherent picture of how sentiment is expressed in both languages. With the GermaNet bridge, performance is better than both the degenerate and dictionary based bridges, showing that the model is sharing information both through the multilingual topics and the regression parameters. Performance on English prediction is comparable to previously published results on this dataset (Blei and McAuliffe, 2007); with enough data, a monolin­gual model is no longer helped by adding additional multilingual data.</p><doubt alpha="59.3" length="27" tooSmall="False" monospace="0.0">Si'lâ([I'm afraid that...])</doubt><doubt alpha="54.5" length="11" tooSmall="True" monospace="0.0">#$ (myself)</doubt><doubt alpha="62.5" length="16" tooSmall="True" monospace="0.0">([really isn't])</doubt><p>ffi (discard)     (female) weiblich (soon) bald<page local="9" global="53"/></p><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>god</p></td><td class="cell"><p>himmel</p></td><td class="cell"><p>diet</p></td><td class="cell"><p>essen</p></td><td class="cell"><p>book</p></td><td class="cell"><p>buch</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>us</p></td><td class="cell"><p>gedanken</p></td><td class="cell"><p>food</p></td><td class="cell"><p>diät</p></td><td class="cell"><p>books</p></td><td class="cell"><p>immer</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>religion</p></td><td class="cell"><p>glaube</p></td><td class="cell"><p>eat</p></td><td class="cell"><p>verlieren</p></td><td class="cell"><p>one</p></td><td class="cell"><p>leben</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>church</p></td><td class="cell"><p>unsere</p></td><td class="cell"><p>weight</p></td><td class="cell"><p>befinden</p></td><td class="cell"><p>life</p></td><td class="cell"><p>art</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>human</p></td><td class="cell"><p>kirche</p></td><td class="cell"><p>eating</p></td><td class="cell"><p>körper</p></td><td class="cell"><p>person</p></td><td class="cell"><p>lesen</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>wahrheit</p></td><td class="cell"><p>healthy</p></td><td class="cell"><p>rezepte</p></td><td class="cell"><p>people</p></td><td class="cell"><p>thema</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>fat</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>autor</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>-1</p></td><td class="cell"><p></p></td><td class="cell"><p>-0.4</p></td><td class="cell"><p>0.4</p></td><td class="cell"><p></p></td><td class="cell"><p>.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>movie</p></td><td class="cell"><p>great</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>book</p></td><td class="cell"><p>buch</p></td><td class="cell"><p></p></td><td class="cell"><p>film</p></td><td class="cell"><p>film good</p></td><td class="cell"><p>gesellschaft</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>novel</p></td><td class="cell"><p>roman children</p></td><td class="cell"><p>kind</p></td><td class="cell"><p>episode</p></td><td class="cell"><p>filme business</p></td><td class="cell"><p>genau</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>reader</p></td><td class="cell"><p>leser baby</p></td><td class="cell"><p>kinder</p></td><td class="cell"><p>movies</p></td><td class="cell"><p>episode all</p></td><td class="cell"><p>überzeugt</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>pages</p></td><td class="cell"><p>seiten child</p></td><td class="cell"><p>eltern</p></td><td class="cell"><p>scenes</p></td><td class="cell"><p>star one</p></td><td class="cell"><p>ergebnis</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>books</p></td><td class="cell"><p>geschichte parents</p></td><td class="cell"><p>baby</p></td><td class="cell"><p>separate</p></td><td class="cell"><p>story companies</p></td><td class="cell"><p>mittel</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>tale</p></td><td class="cell"><p>handlung sleep</p></td><td class="cell"><p>nacht</p></td><td class="cell"><p>gives</p></td><td class="cell"><p>gibt right</p></td><td class="cell"><p>verlangen</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Train</p></td><td class="cell"><p>Test</p></td><td class="cell"><p>GermaNet</p></td><td class="cell"><p>Dictionary</p></td><td class="cell"><p>Flat</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>DE</p></td><td class="cell"><p>DE</p></td><td class="cell"><p>73.8</p></td><td class="cell"><p>24.8</p></td><td class="cell"><p>92.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EN</p></td><td class="cell"><p>DE</p></td><td class="cell"><p>7.44</p></td><td class="cell"><p>2.68</p></td><td class="cell"><p>18.3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EN + DE</p></td><td class="cell"><p>DE</p></td><td class="cell"><p><b>1.17</b></p></td><td class="cell"><p><b>1.46</b></p></td><td class="cell"><p><b>1.39</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></subsection></section><section number="5" title="Relationship to Previous Research"><p>The advantages of MLSLDA reside largely in the assumptions that it makes and does not make: docu­ments need not be parallel, sentiment is a normally distributed document-level property, words are ex­changeable, and sentiment can be predicted as a re­gression on a K-dimensional vector.</p><p>By not assuming parallel text, this approach can be applied to a broad class of corpora. Other mul­tilingual topic models require parallel text, either at the document (Ni et al., 2009; Mimno et al., 2009) or word-level (Kim and Khudanpur, 2004; Zhao and Xing, 2006). Similarly, other multilingual sentiment approaches also require parallel text, often supplied via automatic translation; after the translated text is available, either monolingual analysis (Denecke, 2008) or co-training is applied (Wan, 2009). In con­trast, our approach requires fewer resources for a lan­guage: a dictionary (or similar knowledge structure relating words to nodes in a graph) and comparable text, instead of parallel text or a machine translation system.</p><p>Rather than viewing one language through the lens of another language, MLSLDA views all lan­guages through the lens of the topics present in a document. This is a modeling decision with pros and cons. It allows a language agnostic decision about sentiment to be made, but it restricts the expressive­ness of the model in terms of sentiment in two ways. First, it throws away information important to sen­timent analysis like syntactic constructions (Greene and Resnik, 2009) and document structure (McDon­ald et al., 2007) that may impact the sentiment rating. Second, a single real number is not always sufficient to capture the nuances of sentiment. Less critically, assuming that sentiment is normally distributed is not true of all real-world corpora; review corpora often have a skew toward positive reviews. We standardize responses by the mean and variance of the training data to partially address this issue, but other response distributions are possible, such as generalized linear models (Blei and McAuliffe, 2007) and vector ma­chines (Zhu et al., 2009), which would allow more traditional classification predictions.</p><p>Other probabilistic models for sentiment classifi­cation view sentiment as a word level feature. Some models use sentiment word lists, either given or learned from a corpus, as a prior to seed topics so that they attract other sentiment bearing words (Mei et al., 2007; Lin and He, 2009). Other approaches view sentiment or perspective as a perturbation of a log-linear topic model (Lin et al., 2008). Such techniques could be combined with the multilingual approach presented here by using distributions over words that not only bridge different languages but also encode additional information. For example, the vocabulary hierarchies could be structured to encour­age topics that encourage correlation among similar sentiment-bearing words (e.g. clustering words asso­ciated with price, size, etc.). Future work could also more rigorously validate that the multilingual topics discovered by MLSLDA are sentiment-bearing via human judgments.</p><p>In contrast, MlSLDA draws on techniques that view sentiment as a regression problem based on the topics used in a document, as in supervised latent Dirichlet allocation (SLDA) (Blei and McAuliffe, 2007) or in finer-grained parts of a document (Titov and McDonald, 2008). Extending these models to multilingual data would be more straightforward.</p></section><section number="6" title="Conclusions"><p>MlSLDA is a "holistic" statistical model for multi­lingual corpora that does not require parallel text or expensive multilingual resources. It discovers connections across languages that can recover la­tent structure in parallel corpora, discover sentiment-correlated word lists in multiple languages, and make accurate predictions across languages that improve with more multilingual data, as demonstrated in the context of sentiment analysis.</p><p>More generally, MlSLDA provides a formalism that can be used to incorporate the many insights of topic modeling-driven sentiment analysis to multi­lingual corpora by tying together word distributions across languages. MlSLDA can also contribute to the development of word list-based sentiment sys­tems: the topics discovered by MlSLDA can serve as a first-pass means of sentiment-based word lists for languages that might lack annotated resources.</p><p>MlSLDA also can be viewed as a sentiment-informed multilingual word sense disambiguation (WSD) algorithm. When the multilingual bridge is an explicit representation of sense such as WordNet, part of the generative process is an explicit assignment of every word to sense (the path latent variable A); this is discovered during inference.<page local="10" global="54"/> The dictionary-based technique may be viewed as a disambiguation via a transfer dictionary. How sentiment prediction impacts the implicit WSD is left to future work.</p><p>Better capturing local syntax and meaningful col­locations would also improve the model's ability to predict sentiment and model multilingual topics, as would providing a better mechanism for represent­ing words not included in our bridges. We intend to develop such models as future work.</p></section><section number="7" title="Acknowledgments"><p>This research was funded in part by the Army Re­search Laboratory through ARL Cooperative Agree­ment W911NF-09-2-0072 and by the Office of the Director of National Intelligence (ODNI), Intelli­gence Advanced Research Projects Activity (IARPA), through the Army Research Laboratory. All state­ments of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representing the official views or poli­cies of ARL, IARPA, the ODNI, or the U.S. Govern­ment. The authors thank the anonymous reviewers, Jonathan Chang, Christiane Fellbaum, and Lawrence Watts for helpful comments. The authors especially thank Chris Potts for providing help in obtaining and processing reviews.</p></section><references><p>David Andrzejewski, Xiaojin Zhu, and Mark Craven.</p><p>2009. Incorporating domain knowledge into topic mod­eling via Dirichlet forest priors. In <i>ICML.</i></p><p>Andrea Esuli Stefano Baccianella and Fabrizio Sebastiani.</p><p>2010. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In <i>LREC.</i></p><p>Carmen Banea, Rada Mihalcea, Janyce Wiebe, and Samer Hassan. 2008. Multilingual subjectivity analysis using machine translation. In <i>EMNLP.</i></p><p>David M. Blei and John D. Lafferty. 2005. Correlated topic models. In NIPS.</p><p>David M. Blei and Jon D. McAuliffe. 2007. Supervised topic models. In <i>NIPS. </i>MIT Press.</p><p>David M. Blei, Andrew Ng, and Michael Jordan. 2003. Latent Dirichlet allocation. <i>JMLR, </i>3:993-1022.</p><p>Jordan Boyd-Graber and David M. Blei. 2009. Multilin­gual topic models for unaligned text. In <i>UAI.</i></p><p>Jordan Boyd-Graber, David M. Blei, and Xiaojin Zhu.</p><p>2007. A topic model for word sense disambiguation.</p><p>In <i>EMNLP.</i></p><p>Jonathan Chang and David M. Blei. 2009. Relational topic models for document networks. In <i>AISTATS.</i></p><p>Shay B. Cohen and Noah A. Smith. 2009. Shared lo­gistic normal distributions for soft parameter tying in unsupervised grammar induction. In <i>NAACL.</i></p><p>Noah Constant, Christopher Davis, Christopher Potts, and Florian Schwarz. 2009. The pragmatics of expressive content: Evidence from large corpora. <i>Sprache und Datenverarbeitung, </i>33(1-2).</p><p>Kerstin Denecke. 2008. Using SentiWordNet for multilin­gual sentiment analysis. In <i>ICDEW 2008.</i></p><p>Paul    Denisowski. 1997. CEDICT.</p><p>http://www.mdbg.net/chindict/.</p><p>Jean Diebolt and Eddie H.S. Ip, 1996. <i>Markov Chain Monte Carlo in Practice, </i>chapter Stochastic EM: method and application. Chapman and Hall, London.</p><p>Alexander Geyken. 2007. The DWDS corpus: A ref­erence corpus for the German language of the 20th century. In <i>Idioms and Collocations: Corpus-based Linguistic, Lexicographic Studies. </i>Continuum Press.</p><p>Stephan Greene and Philip Resnik. 2009. More than words: Syntactic packaging and implicit sentiment. In <i>NAACL.</i><i></i></p><p>Thomas L. Griffiths and Mark Steyvers. 2004. Finding scientific topics. <i>PNAS, </i>101(Suppl 1):5228-5235.</p><p>Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. Learning bilingual lexicons from monolingual corpora. In <i>ACL, </i>Columbus, Ohio.</p><p>Jan Hefti. 2005. HanDeDict. http://chdw.de.</p><p>Hitoshi Isahara, Fransis Bond, Kiyotaka Uchimoto, Masao Utiyama, and Kyoko Kanzaki. 2008. Development of the Japanese WordNet. In <i>LREC.</i></p><p>Mark Johnson. 2010. PCFGs, topic models, adaptor grammars and learning topical collocations and the structure of proper names. In ACL.</p><p>Woosung Kim and Sanjeev Khudanpur. 2004. Lexical triggers and latent semantic analysis for cross-lingual language model adaptation. <i>TALIP, </i>3(2):94-112.</p><p>Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In <i>MT Summit. </i>http://www.statmt.org/europarl/.</p><p>Claudia Kunze and Lothar Lemnitzer. 2002. Standardiz­ing WordNets in a web-compliant format: The case of GermaNet. In <i>Workshop on Wordnets Structures and Standardisation.</i></p><p>Chenghua Lin and Yulan He. 2009. Joint sentiment/topic model for sentiment analysis. In <i>CIKM.</i></p><p>Wei-Hao Lin, Eric Xing, and Alexander Hauptmann.</p><p>2008. A joint topic and perspective model for ideo­logical discourse. In <i>ECML PKDD.</i></p><page local="11" global="55"/><p>Edward Loper and Steven Bird. 2002. NLTK: the natu­ral language toolkit. In <i>Tools and methodologies for teaching. </i>ACL.</p><p>Ryan McDonald, Kerry Hannan, Tyler Neylon, Mike Wells, and Jeff Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In ACL.</p><p>Qiaozhu Mei, Xu Ling, Matthew Wondra, Hang Su, and ChengXiang Zhai. 2007. Topic sentiment mixture: modeling facets and opinions in weblogs. In <i>WWW.</i></p><p>Ilya Dan Melamed. 1998. <i>Empirical methods for exploit­ing parallel texts. </i>Ph.D. thesis, University of Pennsyl­vania.</p><p>George A. Miller. 1990. Nouns in WordNet: A lexical inheritance system. <i>International Journal of Lexicog­raphy, </i>3(4):245-264.</p><p>David Mimno, Hanna Wallach, Jason Naradowsky, David Smith, and Andrew McCallum. 2009. Polylingual topic models. In <i>EMNLP.</i></p><p>Xiaochuan Ni, Jian-Tao Sun, Jian Hu, and Zheng Chen. 2009. Mining multilingual topics from Wikipedia. In <i>WWW.</i><i></i></p><p>Noam Ordan and Shuly Wintner. 2007. Hebrew Word­Net: a test case of aligning lexical databases across lan­guages. <i>International Journal of Translation, </i>19(1):3958.</p><p>Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In <i>ACL.</i></p><p>Bo Pang and Lillian Lee. 2008. <i>Opinion Mining and Sentiment Analysis. </i>Now Publishers Inc.</p><p>Martin Porter and Richard Boulton. 1970. Snowball stemmer. http://snowball.tartarus.org/credits.php.</p><p>Matthew Purver, Konrad Kording, Thomas L. Griffiths, and Joshua Tenenbaum. 2006. Unsupervised topic modelling for multi-party spoken discourse. In <i>ACL.</i></p><p>Reinhard Rapp. 1995. Identifying word translations in non-parallel texts. In <i>ACL, </i>pages 320-322.</p><p>Philip Resnik. 1995. Using information content to evalu­ate semantic similarity in a taxonomy. In <i>IJCAI, </i>pages 448-453.</p><p>Frank Richter. 2008. Dictionary nice grep. http://www-</p><p>user.tu-chemnitz.de/ fri/ding/. Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003.</p><p>Learning subjective nouns using extraction pattern boot­strapping. In <i>NAACL. </i>Michal Rosen-Zvi, Thomas L. Griffiths, Mark Steyvers, and Padhraic Smyth. 2004. The author-topic model for</p><p>authors and documents. In <i>UAI. </i>Benoît Sagot and Darja Fiser. 2008. Building a Free</p><p>French WordNet from Multilingual Resources. In <i>On-</i> <i>toLex.</i><i></i></p><p>Ivan Titov and Ryan McDonald. 2008. A joint model of text and aspect ratings for sentiment summarization. In ACL, pages 308-316.</p><p>Huihsin Tseng, Pichuan Chang, Galen Andrew, Daniel Ju-rafsky, and Christopher Manning. 2005. A conditional random field word segmenter. In <i>SIGHAN Workshop on Chinese Language Processing.</i></p><p>University of Oxford. 2006. British Na­tional Corpus. http://www.natcorp.ox.ac.uk/. http://www.natcorp.ox.ac.uk/.</p><p>Tobias Vetter, Manfred Sauer, and Philipp Wallutat. 2000. Filmrezension.de: Online-magazin fur filmkritik. http://www.filmrezension.de.</p><p>Xiaojun Wan. 2009. Co-training for cross-lingual senti­ment classification. In ACL.</p><p>Chong Wang, David Blei, and Li Fei-Fei. 2009. Simulta­neous image classification and annotation. In <i>CVPR.</i></p><p>Xing Wei and Bruce Croft. 2006. LDA-based document models for ad-hoc retrieval. In <i>SIGIR.</i></p><p>Casey Whitelaw, Navendu Garg, and Shlomo Argamon. 2005. Using appraisal groups for sentiment analysis.</p><p>In <i>CIKM.</i></p><p>Theresa Ann Wilson. 2008. <i>Fine-grained Subjectivity and Sentiment Analysis: Recognizing the Intensity, Polarity, and Attitudes ofPrivate States. </i>Ph.D. thesis, University of Pittsburgh.</p><p>Bing Zhao and Eric P. Xing. 2006. BiTAM: Bilingual topic admixture models for word alignment. In <i>ACL.</i></p><p>Jun Zhu, Amr Ahmed, and Eric P. Xing. 2009. Medlda: maximum margin supervised topic models for regres­sion and classification. In <i>ICML.</i></p></references></body></article>