<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="521"/><title>Statistical Machine Translation Models for Personalized Search</title><author surname="U" givenname="Rohini"><org  name="AOL India R&amp;D" country="India" city="Bangalore"/></author><author surname="Ambati" givenname="Vamshi"><org  name="Carnegie Mellon University" country="USA" city="Pittsburgh"/></author><author surname="Varma" givenname="Vasudeva"><org  name="Carnegie Mellon University" country="USA" city="Pittsburgh"/></author></firstpageheader><frontmatter><p><b>Statistical Machine Translation Models for Personalized Search</b></p><p><b>Rohini U *                     Vamshi Ambati Vasudeva Varma</b></p><p>AOL India R&amp; D        Language Technologies Institute LTRC, IIIT Hyd</p><p>Bangalore, India            Carnegie Mellon University Hyderabad, India</p><p>Rohini.uppuluri@corp.aol.com          Pittsburgh, USA vv@iiit.ac.in</p><p>vamshi@cs.cmu.edu</p></frontmatter><abstract>Web search personalization has been well studied in the recent few years. Relevance feedback has been used in various ways to improve relevance of search results. In this paper, we propose a novel usage of rele­vance feedback to effectively model the pro­cess of query formulation and better char­acterize how a user relates his query to the document that he intends to retrieve using a noisy channel model. We model a user profile as the probabilities of translation of query to document in this noisy channel us­ing the relevance feedback obtained from the user. The user profile thus learnt is applied in a re-ranking phase to rescore the search results retrieved using an underlying search engine. We evaluate our approach by con­ducting experiments using relevance feed­back data collected from users using a pop­ular search engine. The results have shown improvement over baseline, proving that our approach can be applied to personalization of web search. The experiments have also resulted in some valuable observations that learning these user profiles using snippets surrounding the results for a query gives bet­ter performance than learning from entire document collection. </abstract></header><body><section number="1" title="Introduction"><p>Most existing text retrieval systems, including the web search engines, suffer from the problem of "one</p><p>This work was done when the first and second authors were at IIIT Hyderabad, India.</p><p>size fits all": the decision of which documents to re­trieve is made based only on the query posed, with­out consideration of a particular user's preferences and search context. When a query (e.g. "jaguar") is ambiguous, the search results are inevitably mixed in content (e.g. containing documents on the jaguar cat and on the jaguar car), which is certainly non­optimal for a given user, who is burdened by having to sift through the mixed results. In order to opti­mize retrieval accuracy, we clearly need to model the user appropriately and personalize search according to each individual user. The major goal of person­alized search is to accurately model a user's infor­mation need and store it in the user proile and then re-rank the results to suit to the user's interests using the user profile. However, understanding a user's in­formation need is, unfortunately, a very dificult task partly because it is dificult to model the search pro­cess which is a cognitive process and partly because it is dificult to characterize a user and his prefer­ences and goals. Indeed, this has been recognized as a major challenge in information retrieval research (et. al, 2003).</p><p>In order to address the problem of personalization one needs to clearly understand the actual process of search. First the user has an information need that he would like to fulill. He is the only entity in the process that knows the exact information he needs and also has a vague notion of the document that can full fill his specific information need. A query based search engine is at his disposal for identifying this particular document or set of documents from among a vast repository of them. He then formu­lates a query that he thinks is congruent to the doc­ument he imagines to fulill his need and poses it to the search engine. The search engine now returns a list of results that it calculates as relevant accord­ing to its ranking algorithm.<page local="2" global="522"/> Every user is different and has a different information need, perhaps over­lapping sometimes. The way a user conceives an ideal document that fulfills his need also varies. It is our hypothesis that if one can learn the variations of each user in this direction, effective personalization can be done.</p><p>Most approaches to personalization have tried to model the user's interests by requesting explicit feedback from the user during the search process and observing these relevance judgments to model the user's interests. This is called relevance feed­back, and personalization techniques using it have been proven to be quite effective for improving re­trieval accuracy (Salton and Buckley, 1990; Roc-chio, 1971). These approaches to personalization have considered, user profile to be a collection of words, ontology, a matrix etc.</p><p>We use relevance feedback for personalization in our approach. However we propose a novel usage of relevance feedback to effectively model the process of query formulation and better characterize how a user relates his query to the document that he in­tends to retrieve as discussed in the web search pro­cess above. A user profile learnt from the relevance feedback that captures the query generation process is used as a guide to understand user's interests over time and personalize his web search results.</p><p>Interestingly, a new paradigm has been proposed for retrieval rooted from statistical language mod­eling recently that views the query generation pro­cess through a Noisy channel model (Berger and Lafferty, 1999) . It was assumed that the docu­ment and query are from different languages and the query generation process was viewed as a trans­lation from the document language which is more verbose to the language of the query which is more compact and brief. The noisy channel model pro­posed by Berger and Lafferty (Berger and Lafferty, 1999) inherently captures the dependencies between the query and document words by learning a trans­lation model between them. As we intend to achieve personalized search by personalizing the query for­mulation process, we also perceive the user profile learning through a Noisy Channel Model. In the model, when a user has an information need, he also has an ideal document in mind that fulfills his need.</p><p>The user tries to in a way translate the notion of the ideal document into a query that is more com­pact but congruent to the document. He then poses this query to the search engine and retrieves the re­sults. By observing this above process over time, we can capture how the user is generating a query from his ideal document. By learning this model ofa user, we can predict which document best describes his information need for the query he poses. This is the motive of personalization. In our approach, we learn a user model which is probabilistic model for the noisy channel using statistical translation ap­proaches and from the past queries and their corre­sponding relevant documents provided as feedback by the user.</p><p>The rest of the paper is organized as follows. We first describe the related work on personalized search then we provide the background and the framework that our approach is based upon. we discuss the modeling of a user profile as a transla­tion model. after which we describe applying it to personalized search. we describe our experimental results followed by conclusions with directions to some future work.</p></section><section number="2" title="Related Work"><p>There has been a growing literature available with regard to personalization of search results. In this section, we briefly overview some of the available literature.</p><p>(Pretschner and Gauch, 1999) used ontology to model users interests, which are studied from users browsed web pages. (Speretta and Gauch, 2004) used users search history to construct user profiles. (Liu et al., 2002) performed personalized web search by mapping a query to a set of categories using a user profile and a general profile learned from the user's search history and a category hierarchy re­spectively. (Hatano and Yoshikawa., 2004) consid­ered the unseen factors of the relationship between the web users behaviors and information needs and constructs user profiles through a memory-based collaborative filtering approach.</p><p>To our knowledge, there has been a very little work has been done that explicitly uses language models to personalization of search results. (Croft et al., 2001) discuss about relevance feedback and query expansion using language modeling.<page local="3" global="523"/> (Shen et al., 2005) use language modeling for short term per­sonalization by expanding queries.</p><p>Earlier approaches to personalization have con­sidered, user profile to be a collection of words, on­tology, language model etc. We perceive the user profile learning through a Noisy Channel Model. In the model, when a user has an information need, he also has a vague notion of what is the ideal document that he would like to retrieve. The user then creates a compact query that he thinks would retrieve the document. He then poses the query to the search en­gine. By observing this above process over time, we learn a user profile as the probabilities of translation for the noisy channel that converts his document to the query. We then use this profile in re-ranking the results of a search engine to provide personalized re­sults.</p></section><section number="3" title="Background"><p>In this section, we describe the statistical language modeling and the translation model framework for information retrieval that form a basis for our re­search.</p><p>The basic approach for language modeling for IR was proposed by Ponte and Croft (Ponte and Croft, 1998). It assumes that the user has a reasonable idea of the terms that are likely to appear in the ideal doc­ument that can satisfy his/her information need, and that the query terms the user chooses can distinguish the ideal document from the rest of the collection. The query is thus generated as the piece of text rep­resentative of the ideal document. The task of the system is then to estimate, for each of the documents in the collection, which is most likely to be the ideal document.</p><p>argmaxP (D\Q) where Q is a query and D is a document. The prior probability P(D) is usually assumed to be uniform and a language model P(Q|D) is estimated for ev­ery document. In other words, they estimate a prob­ability distribution over words for each document and calculate the probability that the query is a sam­ple from that distribution. Documents are ranked according to this probability. The basic model has been extended in a variety of ways. Modeling doc­uments as in terms of a noisy channel model by Berger &amp; Lafferty (Berger and Lafferty, 1999), mix­ture of topics, and phrases are considered (Song and Croft., 1999), (Lavrenko and Croft, 2001) explicitly models relevance, and a risk minimization frame­work based on Bayesian decision theory has been developed (Zhai and Lafferty, 2001).</p><doubt alpha="64.7" length="17" tooSmall="False" monospace="0.0">argmaxP(Q\D)P (D)</doubt><p>The noisy channel by Berger and Lafferty (Berger and Lafferty, 1999) view a query as a distilla­tion or translation from a document describing the query generation process in terms of a noisy channel model. In formulating a query to a retrieval system, a user begins with an information need. This infor­mation need is then represented as a fragment of an "ideal document", a portion of the type of document that the user hopes to receive from the system. The user then translates or "distills" this ideal document fragment into a succinct query, selecting key terms and replacing some terms with related terms.</p><p>To determine the relevance of a document to a query, their model estimates the probability that the query would have been generated as a translation of that document. Documents are then ranked ac­cording to these probabilities. More speciically, the mapping from a document term w to a query term qi is achieved by estimating translation mod­els P(q|w). Using translation models, the retrieval model becomes</p><doubt alpha="47.8" length="23" tooSmall="False" monospace="0.0">P= J]oP+    -a)Y^P(ftMP</doubt><p>where P<i>(Qi\GE</i>) is the smoothed or general probability obtained from a large general corpus. P(qi \ w) is an entry in the translation model. It repre­sents the probability of generation of the query word qi for a word w in the document. P (w\D) is the probability of the word w in the document and <i>a </i>is a weighting parameter which lies between 0 and 1.</p></section><section number="4" title="User Profile as a Translation Model"><p>We perceive the user proile learning as learning the channel probabilities of a Noisy Channel Model that generates the query from the document. In the model, when a user has an information need, he also has a vague notion of what is the ideal document that he would like to retrieve. The user then creates a compact query that he thinks would retrieve the document.<page local="4" global="524"/> He then poses the query to the search en­gine. By observing this above process over time, we can learn how the user is generating a query from his notion of an ideal document. By learning this, we can predict which document best describes his information need. The learnt model, called a user profile, is thus capable of personalizing results for that particular user. Hence, the user profile here is a translation model learnt from explicit feedback of the user using statistical translation approaches. Ex­plicit feedback consists of the past queries and their corresponding relevant documents provided as feed­back by the user. A translation model is a proba­bilistic model consisting of the triples, the source word, the target word and the probability of trans­lation. The translation model here is between doc­ument words and queries words. Therefore the user profile as a translation model in our approach will consist of triples of a document word, a query word and the probability of the document word generating the query word.</p></section><section number="5" title="Personalized Search"><p>In this section, we describe how we perform person­alized search using the proposed translation model based user profile. First, a user profile is learnt using the translation model process then the re-ranking is done using the learnt user profile.</p><subsection number="5.1" title="Learning user profile"><p>In our approach, a user profile consists of a statisti­cal translation model. A translation model is a prob­abilistic model consisting of the triples, the source word, the target word and the probability of trans­lation. Our user profiles consists of the following triples, a document word, a query word and the prob­ability of the document word generating the query word.</p><p>Consider a user u, let { (Qj, Dj}, i = 1,2,N} represent the past history of the user u. where <i>Q</i><i>i </i>is the query and Di is the concatenation of all the relevant documents for the query Qi and let Di = (wi, w2,wn} be the words in it. The user profile learnt from the past history of user consists of the following triples of the form (q, wi, p(q|wi)) where q is a word in the query Qi and wi is a word in the document Di.</p><p>Translation model is typically learnt from paral­lel texts i.e a set of translation pairs consisting of source and target language sentences. In learning the user profile, we first extract parallel texts from the past history of the user and then learn the trans­lation model which is essentially the user profile. In the subsections below, we describe the process in de­tail.</p><subsubsection number="5.1.1" title="Extracting Parallel Texts"><p>By viewing documents as samples of a verbose language and the queries as samples of a concise language, we can treat each document-query pair as a translation pair, i.e. a pair of texts written in the verbose language and the concise language respec­tively. The extracted parallel texts consists of pairs of the form (Qi, Dre1} where Dre1 is the concatena­tion of contexts extracted from all relevant document for the query Qi.</p><p>We believe that short snippets extracted in the context of the query would be better candidates for Dre1 than using the whole document. This is be­cause there can be a lot of noisy terms which need not right in the context of the query. We believe a short snippet usually N (we considered 15) words to the left and right of the query words, similar to a short snippet displayed by search engines can bet­ter capture the context of the query. In deed we experimented with different context sizes for Dre1. The first is using the whole document i.e., consider­ing the query and concatenation of all the relevant documents as a pair in the parallel texts extracted which is called Ddocuments The second is using just a short text snippet from the document in the con­text of query instead of the whole document which is called <i>Dsnippets</i><i> </i>Details are described in the ex­periments section.</p></subsubsection><subsubsection number="5.1.2" title="Learning Translation Model"><p>According to the standard statistical translation model (Brown et al., 1993), we can find the optimal model M * by maximizing the probability of gener­ating queries from documents or<page local="5" global="525"/></p><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">N</doubt><doubt alpha="58.3" length="24" tooSmall="False" monospace="0.0">M * = argmaxUP (Qi|Di,M)</doubt><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">Mi=1</doubt><p>To find the optimal word translation probabilities <i>P(qw\dw, M</i>*), we can use the EM algorithm. The details of the algorithm can be found in the literature for statistical translation models, such as (Brown et al., 1993).</p><p>IBM Model1 (Brown et al., 1993) is a simplistic model which takes no account of the subtler aspects of language translation including the way word or­der tends to differ across languages. Similar to ear­lier work (Berger and Lafferty, 1999), we use IBM Model1 because we believe it is more suited for IR because the subtler aspects of language used for ma­chine translation can be ignored for IR. GIZA++ (Och and Ney, 2003), an open source tool which im­plements the IBM Models which we have used in our work for computing the translation probabilities. A sample user profile learned is shown in Table 1.</p></subsubsection></subsection><subsection number="5.2" title="Re-ranking"><p>Re-ranking is a phase in personalized search where the set of documents matching the query retrieved by a general search engine are re-scored using the user profile and then re-ranked in descending order of rank of the document. We follow a similar ap­proach in our work.</p><p>Let D be set of all the documents returned by the search engine. The rank of each document D re­turned for a query Q for user u is computing using his user profile as shown in Equation 1.</p><doubt alpha="52.2" length="46" tooSmall="False" monospace="0.0">P(Q\D,u) =fjaP(qi\GE) + {l-a)YJP(qi\w,u)P(w\D)</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(1)</doubt><p>where P<i>(qi\GE) </i>is the smoothed or general probability obtained from a large general corpus. P <i>(qi </i>\w, u) is an entry in the translation model of the user. It represents the probability of generation of the query word Qi for a word w in the document. <i>P</i>(w|D) is the probability of the word w in the doc­ument and <i>a </i>is a weighting parameter which lies be­tween 0 and 1.</p><subsubsection number="6.0.1" title="Comparison with Contextless Ranking"><p>We test the effectiveness of our user profile by comparing with a contextless ranking algorithm. We used a generative language modeling for IR as the context less ranking algorithm (Query Likelihood model (Ponte and Croft, 1998; Song and Croft., 1999)). This is actually the simplest version of the model described in Equation 1. Each word w can be translated only as itself that is the translation proba­bilities (see Equation 1) are "diagonal".</p><doubt alpha="45.5" length="11" tooSmall="False" monospace="0.0">P(qi\w,u) =</doubt><doubt alpha="59.1" length="22" tooSmall="False" monospace="0.0">1 if q = w 0 Otherwise</doubt><p>This serves as a good baseline for us to see how well the translation model actually captured the user information. For fair testing similar to our approach, for each query, we first retrieve results matching a query using a general search engine (Lucene). Then we rank the results using the formula shown in Equation 2.</p><doubt alpha="25.0" length="40" tooSmall="False" monospace="0.0">P(Q\D) = [] aP(9i\G£) + (1-a)P(&lt;&amp;\D) (2)</doubt><doubt alpha="75.0" length="4" tooSmall="False" monospace="0.0">qt&amp;Q</doubt><p>We used IBM Model1 for learning the translation model (i.e., the user profile). The general English probabilities are computed from all the documents in the lucene's index. Similar to earlier works (Berger and Lafferty, 1999), we simply set the value of <i>a </i>to be 0.05. The values reported are P@10 values aver­age over all 10 sets and the queries for the respec­tive user. Table 3 clearly shows the improvement brought in by the user profile.</p></subsubsection><subsubsection number="6.0.2" title="Experiments with Different Models"><p>We performed an experiment to see if different training models for learning the user profile affected the performance. We experimented with two mod­els. The first is a basic model and used in ear­lier work, IBM Model1. The second is using the GIZA++ default parameters. We observed that user profile learned using IBM Model1 outperformed that using GIZA++ default parameters. We believe this is because, IBM Model1 is more suited for IR because the subtler aspects of language used for ma­chine translation (which are used in GIZA++ default parameters) can be ignored for IR. We obtained an average P@10 value of 0.2333 for IBM Model1 and 0.2075 for GIZA++.</p><table caption="Table 4: Summary of Comparison of different Mod­els and Contexts for learning user profile"></table></subsubsection><subsubsection number="6.0.3" title="Snippet Vs Document"><p>In extracting parallel texts consists of pairs of the form {Qi, Dre1} where Dre1 is the concatenation of contexts extracted from all relevant document for the query Qi we experimented with different context sizes for Dre1.</p><p>We believe that a short snippet extracted in the context of the query would be better candidate for Dre1 than using the whole document. This is be­cause there can be a lot of noisy terms which need not useful in the context of the query. We believe a short snippet usually N (we considered 15) words to the left and right of the query words, similar to a short snippet displayed by search engines can better<page local="7" global="527"/></p><table caption="Table 2: Statistics of the data set of 7 users" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>User</b></p></td><td class="cell"><p><b>No. Q</b></p></td><td class="cell"><p><b>% of Unique words in Q</b></p></td><td class="cell"><p><b>Total Rel</b></p></td><td class="cell"><p><b>Avg. Rel</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>37</p></td><td class="cell"><p>89</p></td><td class="cell"><p>236</p></td><td class="cell"><p>6.378</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>50</p></td><td class="cell"><p>68.42</p></td><td class="cell"><p>178</p></td><td class="cell"><p>3.56</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3</p></td><td class="cell"><p>61</p></td><td class="cell"><p>82.63</p></td><td class="cell"><p>298</p></td><td class="cell"><p>4.885</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p>26</p></td><td class="cell"><p>86.95</p></td><td class="cell"><p>101</p></td><td class="cell"><p>3.884</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>5</p></td><td class="cell"><p>33</p></td><td class="cell"><p>80.76</p></td><td class="cell"><p>134</p></td><td class="cell"><p>4.06</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>6</p></td><td class="cell"><p>29</p></td><td class="cell"><p>78.08</p></td><td class="cell"><p>98</p></td><td class="cell"><p>3.379</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>7</p></td><td class="cell"><p>29</p></td><td class="cell"><p>88.31</p></td><td class="cell"><p>115</p></td><td class="cell"><p>3.965</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 3: Precision @10 results for 7 users" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Set</b></p></td><td class="cell"><p><b>Contextless</b></p></td><td class="cell"><p><b>Proposed</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User1</p></td><td class="cell"><p>0.1433</p></td><td class="cell"><p>0.1421</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User2</p></td><td class="cell"><p>0.1426</p></td><td class="cell"><p>0.2445</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User3</p></td><td class="cell"><p>0.1016</p></td><td class="cell"><p>0.1216</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User4</p></td><td class="cell"><p>0.0557</p></td><td class="cell"><p>0.1541</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User5</p></td><td class="cell"><p>0.1877</p></td><td class="cell"><p>0.3933</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User6</p></td><td class="cell"><p>0.1566</p></td><td class="cell"><p>0.3941</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>User7</p></td><td class="cell"><p>0.1</p></td><td class="cell"><p>0.1833</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Avg</b></p></td><td class="cell"><p>0.1268</p></td><td class="cell"><p><b>0.2332</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 4: Summary of Comparison of different Models and Contexts for learning user profile" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Training Model</b></p></td><td class="cell"><p><b>Document Test</b></p></td><td class="cell"><p><b>Snippet Test</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>IBM Model1</b></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Document Train</p></td><td class="cell"><p>0.2062</p></td><td class="cell"><p>0.2028</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Snippet Train</p></td><td class="cell"><p>0.2333</p></td><td class="cell"><p><b>0.2488</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>GIZA++</b></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Document Train</p></td><td class="cell"><p>0.1799</p></td><td class="cell"><p>0.1834</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Snippet Train</p></td><td class="cell"><p>0.2075</p></td><td class="cell"><p>0.2034</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><figure caption="Figure 1: Comparison of Snippet Vs Document Training using IBM Model1 for training."></figure><p>IBM Model1 : I - Document Training and Document Testing, IBM Model1 : II - Document Training and Snippet Testing, IBM Model1 : III - Snippet Training and Document Testing, IBM Model1 : IV - Snippet Training and Snippet Testing</p><p>capture the context of the query.</p><p>We experimented with two context sizes. The first is using the whole document i.e., considering the query and concatenation of all the relevant doc­uments as a pair in the parallel texts extracted which is called <i>Ddocuments.</i><i> </i>The second is using just a short text snippet from the document in the context of query instead of the whole document which is called <i>D</i><i>snippets. </i>The user profile learning from pairs of parallel texts {Q, Ddocuments} is called <i>Document Train. </i>The user profile learning from pairs of paral­lel texts {Q, Dsnippets} is called <i>Snippet Train. </i>The user profiles are trained using both IBM Model1 and GIZA++ and comparison of the two is shown in Ta­ble 4.</p><p>We also experimented with the size of the context used for testing. Using the document for re-ranking as shown in Equation 1 (called <i>Document Test)</i><footnote anchor="2"/> and using just a short snippet extracted from the docu­ment for testing (called <i>Snippet Test). </i>Table 4 shows the average P@10 over the 10 sets and all queries and users.</p><p>We observed that, not only did the model used for training affected P@10, but also the data used in training and testing, whether it was a snippet or doc­ument, showed a large variation in the performance. Training using IBM Model1 using the snippet and</p><footnote label="2">It is to be noted that Snippet Train and Document Test and training using IBM Model1 is the default configuration used for all the reported results unless explicitly specified.</footnote><figure caption="Figure 2: Comparison of Snippet Vs Document Training using GIZA++ Default parameters for training."></figure><p>GIZA++:I - Document Training and Document Testing, GIZA++:II - Document Training and Snippet Testing, GIZA++:III - Snippet Training and Document Testing, GIZA++:IV - Snippet Training and Snippet Testing</p><p>testing using snippet achieved the best results. This is in agreement with the discussion that the snip­pet surrounding the query captures the context of the query better than a document which may con­tain many words that could possibly be unrelated to the query, therefore diluting the strength of the mod­els learnt. The detailed results for all the users are shown in Figure 1 and Figure 2.</p></subsubsection></subsection></section><section number="6" title="Experiments"><p>We performed experiments evaluating our approach on data set consisting of 7 users. Each user submit­ted a number of queries to a search engine (Google). For each query, the user examined the top 10 docu­ments and identified the set of relevant documents. Table 2 gives the statistics of the data sets. There is no repetition of query for any user though repetition of some words in the query exists (see Table 2). The document collection consists of top 20 documents from google which is actually the set of documents seen by the user while accessing the relevance of the documents. In all, the total size of the document collection was 3,469 documents. We did not include documents of type doc and pdf files.</p><p>To evaluate our approach, we use the 10-fold cross-validation strategy (Mitchell, 1997). We di­vide the data of each user into 10 sets each hav­ing (approximately) equal number of search queries (For example, for user1 had 37 queries in total, we divided this into 10 sets with 4 queries each approx­imately). Learning of user profile is done 10 times, each time leaving out one of the sets from training, but using only the omitted subset for testing. Per­formance is computed in the testing phase for each time and average of the 10 times is taken. In the testing phase, we take each query and re rank the results using the proposed approach using his pro­file learned from nine other sets. For measuring performance for each query, we compute Precision @10 (P@10), a widely used metric for evaluating personalized search algorithms. It is defined as the proportion of relevant documents among the top 10 results for the given ranking of documents. P@10 is computed by comparing with the relevant docu­ments present in the data. All the values presented in the tables are average values which are averaged over all queries for each user, unless otherwise spec­ified. We used Lucene<footnote anchor="1"/>, an open source search en­gine as the general search engine to first retrieve a set of results matching the query.<page local="6" global="526"/></p><footnote label="1"> http://lucene.apache.org</footnote><table caption="Table 1: Sample user profile" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>qw</b></p></td><td class="cell"><p><b>dw</b></p></td><td class="cell"><p><b>P(qw|dw,u)</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>kdd</p></td><td class="cell"><p>0.0176</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>conference</p></td><td class="cell"><p>0.0123</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>journal</p></td><td class="cell"><p>0.0176</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>sigkdd</p></td><td class="cell"><p>0.0088</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>discovery</p></td><td class="cell"><p>0.0211</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>mining</p></td><td class="cell"><p>0.0017</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>journal</p></td><td class="cell"><p>acm</p></td><td class="cell"><p>0.0088</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>music</p></td><td class="cell"><p>0.0375</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>purchase</p></td><td class="cell"><p>0.0090</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>mp3</p></td><td class="cell"><p>0.0090</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>listen</p></td><td class="cell"><p>0.0180</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>mp3.com</p></td><td class="cell"><p>0.0450</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>music</p></td><td class="cell"><p>free</p></td><td class="cell"><p>0.0008</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><section number="7" title="Conclusions and Future Work"><p>Relevance feedback from the user has been used in various ways to improve the relevance of the re­sults for the user. In this paper we have proposed a novel usage of relevance feedback to effectively model the process of query formulation and better characterize how a user relates his query to the doc­ument that he intends to retrieve. We applied a noisy channel model approach for the query and the doc­uments in a retrieval process. The user profile was modeled using the relevance feedback obtained from the user as the probabilities of translation of query to document in this noisy channel. The user pro­file thus learnt was applied in a re-ranking phase to rescore the search results retrieved using general in­formation retrieval models. We evaluate the usage of our approach by conducting experiments using rele­vance feedback data collected from users of a popu­lar search engine. Our experiments have resulted in some valuable observations that learning these user profiles using snippets surrounding the results for a query show better performance than when learn­ing from entire documents.<page local="8" global="528"/> In this paper, we have only evaluated explicit relevance feedback gathered from a user and performed our experiments. As part of future work, we would like to evaluate our ap­proach on implicit feedback gathered probably as click-through data in a search engine, or on the client side using customized browsers.</p></section><references><p>Adam Berger and John D. Lafferty. 1999. Information retrieval as statistical translation. In <i>Research and De­velopment in Information Retrieval, </i>pages 222-229.</p><p>Peter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. 1993. The mathemat­ics of statistical machine translation: parameter esti­mation. <i>Comput. Linguist., </i>19(2):263-311.</p><p>W. Bruce Croft, Stephen Cronen-Townsend, and Victor Larvrenko. 2001. Relevance feedback and person­alization: A language modeling perspective. In <i>DE-LOS Workshop: Personalisation and Recommender Systems in Digital Libraries.</i></p><p>Jamie Allan et. al. 2003. Challenges in information re­trieval language modeling. In <i>SIGIR Forum, </i>volume 37 Number 1.</p><p>K. Sugiyama K. Hatano and M. Yoshikawa. 2004. Adap­tive web search based on user profile constructed with­out any effort from users. In <i>Proceedings ofWWW 2004, </i>page 675 684.</p><p>Victor Lavrenko and W. Bruce Croft. 2001. Relevance-basedlanguage models. In <i>Research andDevelopment in Information Retrieval, </i>pages 120-127.</p><p>F. Liu, C. Yu, and W. Meng. 2002. Personalized web search by mapping user queries to categories. In <i>Pro­ceedings ofthe eleventh international conference on Information and knowledge management, ACM Press, </i>pages 558-565.</p><p>Tom Mitchell. 1997. <i>Machine Learning. </i>McGrawHill.</p><p>Franz Josef Och and Hermann Ney. 2003. A system­atic comparison of various statistical alignment mod­els. <i>Computational Linguistics, </i>29(1):19-51.</p><p>Jay M. Ponte and W. Bruce Croft. 1998. A lan­guage modeling approach to information retrieval. In <i>Research and Development in Information Retrieval,</i> pages 275-281.</p><p>A. Pretschner and S. Gauch. 1999. Ontology based per­sonalized search. In <i>ICTAI., </i>pages 391-398.</p><p>J. J. Rocchio. 1971. Relevance feedback in information retrieval, the smart retrieval system. <i>Experiments in Automatic Document Processing, </i>pages 313-323.</p><p>G. Salton and C. Buckley. 1990. Improving retrieval per­formance by relevance feedback. <i>Journal of the Amer­ican Society of Information Science, </i>41:288-297.</p><p>Xuehua Shen, Bin Tan, and Chengxiang Zhai. 2005. Im­plicit user modeling for personalized search. In <i>Pro­ceedings ofCIKM 2005.</i></p><p>F. Song and W. B. Croft. 1999. A general language model for information retrieval. In <i>Proceedings on the 22nd annual international ACM SIGIR conference,</i> page 279280.</p><p>Micro Speretta and Susan Gauch. 2004. Personalizing search based on user search histories. In <i>Thirteenth International Conference on Information and Knowl­edge Management (CIKM 2004).</i></p><p>Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to ad hoc information retrieval. In <i>Proceedings ofACM SI-GIR'01, </i>pages 334-342.</p></references></body></article>