<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="318"/><title>TITPI: Web People Search Task Using Semi-Supervised Clustering Approach</title><pubinfo>Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007),pages 318-321, Prague, June 2007. ©2007 Association for Computational Linguistics</pubinfo><author surname="Sugiyama" givenname="Kazunari"><org  name="Tokyo Institute of Technology" country="Japan" city="Tokyo"/></author><author surname="Okumura" givenname="Manabu"><org  name="Tokyo Institute of Technology" country="Japan" city="Tokyo"/></author></firstpageheader><frontmatter><p><b>TITPI: Web People Search Task Using Semi-Supervised Clustering Approach</b></p><p><b>Kazunari Sugiyama</b></p><p>Precision and Intelligence Laboratory</p><p>Tokyo Institute of Technology 4259 Nagatsuta, Midori, Yokohama, Kanagawa 226-8503, Japan sugiyama@lr.pi.titech.ac.jp</p><p><b>Manabu Okumura</b></p><p>Tokyo Institute of Technology 4259 Nagatsuta, Midori, Yokohama, Kanagawa 226-8503, Japan oku@pi.titech.ac.jp</p></frontmatter><abstract>Most of the previous works that disam­biguate personal names in Web search re­sults employ agglomerative clustering ap­proaches. However, these approaches tend to generate clusters that contain a single el­ement depending on a certain criterion of merging similar clusters. In contrast to such previous works, we have adopted a semi-supervised clustering approach to integrate similar documents into a labeled document. Moreover, our proposed approach is char­acterized by controlling the fluctuation of the centroid of a cluster in order to generate more accurate clusters. </abstract></header><body><section number="1" title="Introduction"><p>Personal names are often submitted to search en­gines as query keywords, as described in a report<footnote anchor="1"/>indicating that about 10% of the English queries from the search engine <i>ALLTheWeb<footnote anchor="2"/> </i>contain per­sonal names. However, in response to a personal name query, search engines return a long list of search results containing that contains Web pages about several namesakes. For example, when a user submits a personal name like "William Cohen" as a query to the search engine Google<footnote anchor="3"/>, the re­turned results represent more than one person named "William Cohen." In the results, a computer sci­ence professor, an American politician, a surgeon, and others are not classified into separate clusters but mixed together.</p><footnote label="1"> http://tap.stanford.edu/PeopleSearch.pdf</footnote><footnote label="2">http://www.alltheweb.com/</footnote><footnote label="3">http://www.google.com/</footnote><p>Most of the previous works on disambiguating personal names in Web search results employ sev­eral kinds of agglomerative clustering approach as described in Section 2. However, in these ap­proaches, a lot of clusters that contain only one el­ement tend to be generated, depending on a certain criterion for merging similar clusters. In addition, in person search results from the World Wide Web (WWW), we can often observe that a small num­ber of entities have a lot of search-result Web pages, while others have only one or two. In light of these facts, if a labeled Web page that describes a person is introduced, clustering for personal name disam­biguation would be much more accurate. In the fol­lowing, we refer to such a labeled Web page as the <i>"seed page'" </i>Then, in order to disambiguate per­sonal names in Web search results, we introduce semi-supervised clustering that uses the seed page to aid the clustering of unlabeled search-result Web pages. Our semi-supervised clustering approach is characterized by controlling the fluctuation of the centroid of a cluster.</p></section><section number="2" title="Related Work"><p>(Mann and Yarowsky, 2003) first extract biographi­cal information, such as birthdates, birthplaces, oc­cupations, and so on. Then, for each document, they generate a feature vector composed of the ex­tracted biographical information, proper nouns, and the TF-IDF score computed from the documents in the search results. Finally, using this feature vec­tor, they disambiguate personal names by generating clusters based on a bottom-up centroid agglomerative clustering algorithm.<page local="2" global="319"/> (Wan et al., 2005) employ an approach similar to that of (Mann and Yarowsky, 2003), and have developed a system called <i>Web-Hawk.</i></p><p>(Pedersen et al., 2005) recently proposed a method for discriminating names by clustering the instances of a given name into groups. They ex­tract the context of each instance of an ambiguous name and generate second-order context vectors us­ing significant bigrams. The vectors are then clus­tered such that instances that are similar to each other are grouped into the same cluster.</p><p>(Bekkerman and McCallum, 2005) propose the following three unsupervised approaches: (1) an approach based on the hyperlink structures of Web pages; (2) an approach based on agglomera-tive/conglomerative double clustering (Bekkerman et al., 2005); and (3) a hybrid approach combining the first two.</p><p>(Bollegala et al., 2006) first agglomeratively clus­ter a set of documents and then select key phrases from the resulting clusters to distinguish different namesakes. They extract key phrases from the doc­uments and merge the clusters according to the sim­ilarity between the extracted phrases.</p></section><section number="3" title="Our Proposed Approach"><p>In this section, we first review the pure agglomera-tive clustering approach that most of the previous re­lated works employ and then describe our proposed semi-supervised clustering approach.</p><p>In the following discussion, we denote the feature vector of a search-result Web page in a set of search results as follows:</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(1)</doubt><p>where is the number of distinct terms in the Web page , and denotes each term.</p><p>Stop words were eliminated from all Web pages in the search results based on the stopword list<footnote anchor="4"/>, and stemming was performed using Porter stemmer<footnote anchor="5"/>. In our preliminary experiments, we found that gain (Papineni, 2001) is the most effective term weight­ing scheme for generating feature vectors for clus­tering in this kind of task. Using the gain scheme, we also define each element tuf of <b><i>wp </i></b>as follows:</p><footnote label="4">ftp://ftp.cs.cornell.edu/pub/smart/english.stop  5 http://www.tartarus.org/~martir i /PorterStemmer/</footnote><p><b>Algorithm: </b>Agglomerative clustering <b>Input: </b>Set of search-result Web page <i>pi</i><i> (t = </i><b>1</b>,2, <b><i>P = </i></b><b>{pi,p2) ■ ■ *Pn}.</b><b></b></p><p><b>Output: </b>Clusters that contain the Web pages that refer to the same person. <b>Method:</b></p><p>1. Set the each element in <b><i>P </i></b>as initial clusters.</p><doubt alpha="60.3" length="58" tooSmall="False" monospace="0.0">2. Repeat the following steps for all pi(«=1,2,*• • ,n)inP</doubt><p>until all of the similarities between two clusters are less than the predefined threshold.</p><subsection number="2.1" title="Compute the similarity between and"><p>if the similarity is greater than the predefined threshold, then merge pi and pi+i, and recompute the centroid of the cluster using Equation (3), else     is an independent cluster.</p><p>2.2 Compute all ofthe similarities between two clusters.</p><figure caption="Figure 1: Agglomerative clustering algorithm."></figure><doubt alpha="42.1" length="19" tooSmall="False" monospace="0.0">_#(&lt;*)(df{tk)df(tk)</doubt><p>where is the document frequency of term</p><p>, and is the total number of search-result Web pages.</p><p>We also define the centroid vector of a cluster G as follows:</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(2)</doubt><p>where is the weight of the centroid vector of a cluster, and denotes each term.</p></subsection><subsection number="3.1" title="Agglomerative Clustering"><p>In pure agglomerative clustering, initially, each Web page is an individual cluster, and then two clusters with the largest similarity are iteratively merged to generate a new cluster until this similarity is less than a predefined threshold. The detailed algorithm is shown in Figure 1. In this algorithm, the new cen-troid vector of cluster after merging a cluster into its most similar cluster is defined as follows:</p><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">Qnew _</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">te</doubt><doubt alpha="33.3" length="3" tooSmall="True" monospace="0.0">(G)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">+ W</doubt><doubt alpha="20.0" length="5" tooSmall="False" monospace="0.0">n + 1</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(3)</doubt><p>where and    represent the feature vector of a search-result Web page and the number of search-result Web pages in the centroid cluster, re­spectively.</p></subsection><subsection number="3.2" title="Our Proposed Semi-supervised Clustering"><p>As described in Section 1, if a seed page that de­scribes a person is introduced, the clustering for per­sonal name disambiguation would be much more accurate. Therefore, we apply semi-supervised clustering to disambiguate personal names in Web<page local="3" global="320"/></p><p>1. Set the each element in <b><i>P </i></b>as initial clusters.</p><p>if the similarity is greater than the predefined threshold, else     is an independent cluster.</p><doubt alpha="20.0" length="5" tooSmall="False" monospace="0.0">n + 1</doubt><p><b>Algorithm: </b>Semi-supervised clustering</p><doubt alpha="64.7" length="51" tooSmall="False" monospace="0.0">Input:Set of search-result Web pagepi{i =1,2,•••n),</doubt><p>and a seed page <b><i>pseed, </i></b><b><i>P = {pi</i></b>, P2, • • • Fn , <b><i>pseed}. </i></b><b>Output: </b>Clusters that contain the Web pages that refer to the same person. <b>Method:</b></p><p>2. Repeat the following steps for all pi <b>(«=1</b>,2,* • • , <b><i>n) </i></b>in <b><i>P. </i></b>2.1 Compute the similarity between     and .</p><p>if the similarity is greater than the predefined threshold, then merge     into and recompute the centroid of the cluster using Equation (4), else     is stored as other clusters       , namely, .</p><doubt alpha="51.5" length="68" tooSmall="False" monospace="0.0">3. Repeat the following steps for allpj (j =1,2, • • • , m, (m &lt; n))</doubt><p>in until all ofthe similarities between two clusters are less than the predefined threshold.</p></subsection><subsection number="3.1" title="Compute the similarity between and"><p>then merge and , and recompute the centroid ofthe cluster using Equation (3), 3.2 Compute all ofthe similarities between two clusters.</p><figure caption="Figure 2: Semi-supervised clustering algorithm."></figure><p>search results. Our proposed approach is novel in that it controls the fluctuation of the centroid of a cluster when a new cluster is merged into it. In this process, when we merge the feature vector of a search-result Web page into a particular centroid , we weight each element of by the distance between and . As a measure of the distance, we employ the Mahalanobis distance (Hand et al., 2001) that takes into account the correlations of the data set in the clusters. Using Equations (1) and (2), we define the new centroid vector of cluster <b><i>Gnew</i></b>after merging a cluster into its most similar cluster as follows:</p><table caption="Table 1: Personal names and two kinds of seed page."></table><doubt alpha="71.4" length="7" tooSmall="False" monospace="0.0">Qtnew _</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">(G)„WP</doubt><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">+</doubt><doubt alpha="66.7" length="3" tooSmall="True" monospace="0.0">VU?</doubt><doubt alpha="63.6" length="11" tooSmall="False" monospace="0.0">Dmhl(G,wP),</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(4)</doubt><doubt alpha="66.7" length="45" tooSmall="False" monospace="0.0">where and    are the feature vector      of a</doubt><p>search-result Web page and the number of search-result Web pages in the centroid cluster, respec­tively. In Equation (4), the Mahalanobis distance between the centroid vector of cluster and the feature vector of search-result Web page p is defined as follows:</p><doubt alpha="58.1" length="31" tooSmall="False" monospace="0.0">Dmhl(G,wp) =yJ(wP-G)TV-l(WP-G),</doubt><p>where S is the covariance matrix defined by the members in the centroid of a cluster. Figure 2 shows the detailed algorithm of our proposed semi-supervised clustering.</p><p>In our semi-supervised clustering approach, we use the following two kinds of seed page: (a) the article on each person in Wikipedia, and (b) the top ranked Web page in the Web search results. How­ever, not every personal name in the test data ofWeb People Search Task has an corresponding article in Wikipedia. Therefore, if a personal name has an arti­cle in Wikipedia, we used it as the seed page. Other­wise, we used the top ranked Web pages in the Web search results as the seed page. Table 1 shows per­sonal names classified based on each seed page used in our experiment.</p></subsection></section><section number="4" title="Evaluation Results &amp; Discussion"><p>Tables 2 and 3 show evaluation results in each doc­ument set obtained using pure agglomerative clus­tering and our proposed semi-supervised clustering, respectively. "Set 1," "Set 2," and "Set 3" con­tain the names from participants in the ACL con­ference, from biographical articles in the English Wikipedia, and from the US Census, respectively. According to these tables, we found that, although agglomerative clustering outperforms our proposed semi-supervised clustering by 0.21 in the value of purity, our proposed semi-supervised clustering out­performs agglomerative clustering by 0.4 and 0.06 in the values of inverse purity and F-measure, re­spectively. This indicates that our proposed method tends to integrate search-result Web pages into a seed page and a small number of clusters are gen­erated compared with agglomerative clustering. In terms of these facts, it is easier for a user to browse Web pages clustered based on each personal entity. On the other hand, the small values of purity indi­cate that irrelevant search-result Web pages are of­ten contained in the generated clusters. Therefore, we can guess that irrelevant search-result Web pages are integrated into a seed page. In fact, we observed that more than 50 search-result Web pages could be grouped together with a seed page.</p><table class="main" frame="box" rules="all" border="1" regular="True"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Seed page</p></td><td class="cell"><p>Personal name</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(a) Wikipedia article</p></td><td class="cell"><p>Arthur Morgan, George Foster, Harry Hughes, James Davidson, James Hamilton, James Morehead, Jerry Hobbs, John Nelson, Mark Johnson, Neil Clark, Patrick Killen, Robert Moore,</p><p>Stephen Clark, Thomas Fraser, Thomas Kirk, William Dickson (16 names)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(b) The top ranked Web page</p></td><td class="cell"><p>Alvin Cooper, Chris Brockett, Dekang Lin, Frank Keller, James Curran, Jonathan Brooks, Jude Brown, Karen Peterson, Leon Barrett, Marcy Jackson, Martha Edwards, Sharon Goldwater, Stephan Johnson, Violet Howard (14 names)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="4" global="321"/><table caption="Table 2: Evaluation results in each document set obtained using agglomerative clustering."></table><p>Table 3: Evaluation results in each document set ob­tained using our proposed semi-supervised cluster­ing. _</p><p>Table 4 shows the evaluation results obtained us­ing each seed page. The value of F-measure ob­tained using seed page (a) (0.55) is comparable to that obtained using seed page (b) (0.60). In addi­tion, we could observe that some Wikipedia arti­cles are under updating. Therefore, if the Wikipedia articles are continuously updated, the reliability of Wikipedia as a source of seed pages will be promis­ing in the future. Moreover, observing the results of each person in detail, we found that the purity values are improved when we use a seed page that describes the person using more than about 200 words. On the other hand, in the case where a seed page describes a person with less than 150 words, or describes not only the target person but also some other persons, we could not obtain high purity values.</p></section><section number="5" title="Conclusion"><p>In this paper, we described our participating system in the SemEval-2007 Web People Search Task (Ar-tiles et al., 2007). Our system used a semi-supervised clustering which controls the fluctuation of the centroid of a cluster. The evaluation results showed that our proposed method achieves high scores in inverse purity, with the lower scores in pu­rity. This fact indicates that our proposed method tends to integrate search-result Web pages into a seed page. This clustering result makes it easier for a user to browse the results of a person Web search. However, in the generated cluster with a seed page, irrelevant search-result Web pages are also contained. This problem can be solved by in-</p><p>Table 4: Evaluation results based on each seed page obtained using our proposed semi-supervised clus­t<u>ering.</u>_ troducing multiple seed pages. In our experiment, we used the full contents of search-result Web pages and a seed page. We consider that this can cause lower scores in purity. Therefore, in future work, in order to improve the accuracy of clustering, we plan to conduct further experiments by introducing multiple seed pages and using parts of search-result Web pages and seed pages such as words around an ambiguous name.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Document set</p></td><td class="cell"><p>Purity</p></td><td class="cell"><p>Inverse purity</p></td><td class="cell"><p>F-measure (alpha=0.5)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 1</p></td><td class="cell"><p>0.58</p></td><td class="cell"><p>0.51</p></td><td class="cell"><p>0.45</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 2</p></td><td class="cell"><p>0.67</p></td><td class="cell"><p>0.47</p></td><td class="cell"><p>0.53</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 3</p></td><td class="cell"><p>0.72</p></td><td class="cell"><p>0.47</p></td><td class="cell"><p>0.55</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Global average</p></td><td class="cell"><p>0.66</p></td><td class="cell"><p>0.49</p></td><td class="cell"><p>0.51</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Seed page</p></td><td class="cell"><p>Purity</p></td><td class="cell"><p>Inverse purity</p></td><td class="cell"><p>F-measure</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>(alpha=0.5)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(a) Wikipedia article</p></td><td class="cell"><p>0.44</p></td><td class="cell"><p>0.96</p></td><td class="cell"><p>0.55</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(b) The top ranked Web page</p></td><td class="cell"><p>0.47</p></td><td class="cell"><p>0.81</p></td><td class="cell"><p>0.60</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Document set</p></td><td class="cell"><p>Purity</p></td><td class="cell"><p>Inverse purity</p></td><td class="cell"><p>F-measure (alpha=0.5)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 1</p></td><td class="cell"><p>0.53</p></td><td class="cell"><p>0.86</p></td><td class="cell"><p>0.62</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 2</p></td><td class="cell"><p>0.42</p></td><td class="cell"><p>0.89</p></td><td class="cell"><p>0.55</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Set 3</p></td><td class="cell"><p>0.41</p></td><td class="cell"><p>0.92</p></td><td class="cell"><p>0.55</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Global average</p></td><td class="cell"><p>0.45</p></td><td class="cell"><p>0.89</p></td><td class="cell"><p>0.57</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Javier Artiles, Julio Gonzalo, and Satoshi Sekine. 2007. The SemEval-2007 WePS Evaluation: Establishing a Benchmark for the Web People Search Task. In <i>Proceedings ofSemeval 2007, Association for Computational Linguistics.</i></p><p>Ron Bekkerman, Ran El-Yaniv, and Andrew McCallum. 2005. Multi-way Distributional Clustering via Pairwise Interac­tions. In <i>Proceedings ofthe 22nd International Conference on Machine Learning (ICML2005), </i>pages 41-48.</p><p>Ron Bekkerman and Andrew McCallum. 2005. Disambiguat-ing Web Appearances of People in a Social Network. In <i>Proceedings of the 14th International World Wide Web Con­ference (WWW2005), </i>pages 463-470.</p><p>Danushka Bollegala, Yutaka Matsuo and Mitsuru Ishizuka. 2006. Extracting Key Phrases to Disambiguate Personal Names on the Web. In <i>Proceedings ofthe 7th Interna­tional Conference on Computational Linguistics and Intel­ligent Text Processing (CICLing2006)</i>, pages 223-234.</p><p>David J. Hand, Heikki Mannila and Padhraic Smyth. 2001. Principles of Data Mining. MIT Press, 2001.</p><p>Gideon. S. Mann and David Yarowsky. 2003. Unsupervised Personal Name Disambiguation. In <i>Proceedings ofthe 7th Conference on Natural Language Learning (CoNLL-2003), </i>pages 33-40.</p><p>Kishore Papineni. 2001. Why Inverse Document Frequency? In <i>Proceedings ofthe 2nd Meeting ofthe North American Chapter ofthe Association for Computational Linguistics (NAACL 2001), </i>pages 25-32.</p><p>Ted Pedersen, Amruta Purandare, and Anagha Kulkarni. 2005. Name Discrimination by Clustering Similar Contexts. In <i>Proceedings ofthe 6th International Conference on Com­putational Linguistics and Intelligent Text Processing (CI-</i> <i>CLing2005), </i>pages 226-237.</p><p>Xiaojun Wan, Jianfeng Gao, Mu Li, and Binggong Ding. 2005. Person Resolution in Person Search Results: WebHawk. In <i>Proceedings ofthe 14th International Conference on Infor­mation and Knowledge Management (CIKM 2005), </i>pages 163-170.</p></references></body></article>