We consider the problem of disambiguating person names in a Web searching scenario as described by the Web People Search Task in SemEval 2007 (Ar-tiles et al., 2007). Here, the system receives as in­put a set of web pages retrieved from a search en­gine using a given person name as a query. The goal is to determine how many different people are rep­resented for that name in the input web pages, and correctly assign each namesake to its corresponding subset ofweb pages.

There are many challenges towards an effective solution. We are to correctly estimate the number of namesakes for a given person name and group doc­uments referring to the same individual. Moreover, the information sources to be processed are unstruc­tured web pages and there is no certain way of cor­rectly establishing a relation between any two web pages belonging to the same or different individuals.

We have taken several approaches to analyze dif­ferent sources of information provided with the in­put data, and also compared strategies to combine these individual features together. The configuration that achieved the best performance (which were sub­mitted for our run) used a single named entity fea­ture as input to clustering. In the remainder of this paper, we first describe our system in terms of the clustering approach used and alternative features in­vestigated. We then analyze the results on the train­ing set before concluding the paper.

Clustering is the key part for such a task. We have chosen to view the problem as an unsupervised hard clustering problem. First, we view the problem as unsupervised, using the training data for parameter validation, to optimally tune the parameters in the clustering algorithm. Secondly, we observed that the majority of the input pages reference a single indi­vidual, although there are a few that reference mul­tiple individuals sharing the same name. Hence, we view the problem as hard clustering, assigning input pages to exactly one individual, so that the produced clusters do not overlap.

Hard clustering algorithms can be classified as ei­ther partitive or hierarchical. Agglomerative hierar­chical clustering generates a series of nested clusters by merging simple clusters into larger ones, while partitive methods try to find a pre-specified num­ber of clusters that best capture the data. As the correct number of clusters is not given a priori, we chose a method from the second group. We use the Hierarchical Agglomerative Clustering (HAC) algo­rithm (Jain et al., 1999) for all experiments reported in this paper. HAC views each input web page as a separate cluster and iteratively combines the most similar pair of clusters to form a new cluster that replaces the pair. 3 Features

As input to the clustering, we consider several dif­ferent representations of the input documents. Each representation views the input web pages as a vector of features. HAC then computes the cosine similar­ity between the feature vectors for each pair of clus­ters to determine which clusters to merge. We now review the inventory of features studied in our work.

Tokens (T). Identical to the task baseline by (Ar-tiles et al., 2005), we stemmed the words in the web pages using the Porter stemmer (Porter, 1980), to conflate semantically similar English words with the stem. Each stemmed word is considered to be a fea­ture and weighted by its Term Frequency x Inverse Document Frequency (TFxIDF).

Named Entities (NE). We extract the named enti­ties from the web pages using the Stanford Named Entity Recognizer (Finkel et al., 2005). This tagger identifies and labels names of places, organizations and people in the input. Each named entity token is treated as a separate feature, again weighted by TFx IDF. We do not perform stemming for NE fea­tures.

We also consider a more target-centric form of the NE feature, motivated by the observation that person names can be differentiated using their mid­dle names or titles. We first discard all named enti­ties that do not contain any token of the search tar­get, and then discard any token from the remain­ing named entities that appears in the search tar­get. The remaining tokens are then used as features, and weighted by their TFx IDF. For example, for the search target "Edward Fox", the features generated from the name "Edward Charles Morrice Fox" are "Charles" and "Morrice". We call this variation NE targeted (NE-T).

Hostnames and domains (H and D). If two web pages have links pointing to the exact same URL, then there is a good chance that these two web pages refer the same person. However, we find such exact matches of URLs are rare, so we relax the condition and consider their host-names or domain names instead. For example, the URL http://portal.acm.org/guide.cfm has host­name portal.acm.org and domain name acm.org.

As such, for each web page, we can extract the list of hostnames from the links in this page.

We observe that some host/domain names serve as more discriminative evidence than others (e.g., a link to a university homepage is more telling than a link to the list of publications page of Google Scholar when disambiguating computer sci­ence scholars). To model this, we weight each host/domain name by its IDF. Note that we do not use TF as web pages often contain multiple inter­nal links in the form of menus or navigation bars. Using IDF and cosine similarity has been proven effective for disambiguating bibliographic citation records sharing a common author name (Tan et al., 2006).

We also considered a variant where we include the URL of the input web page itself as a "link". We tried this variation only with hostnames, calling this Host with Self URL (H-S).

Page URLs (U). Uniform resource locations (URLs) themselves contain a rich amount of information. For example, the URL http://www.cs.ualberta.ca/~lindek/ itself sug­gests a home page of "lindek" in the Computer Science department, University of Alberta, Canada.

We used the MeURLin system (Kan and Nguyen Thi, 2005) to segment the URL of each web page into tokens as well as to generate additional fea­tures. These features include (a) segmentation of tokens such as "www.allposters.com" to "www", "all", "posters" and "com"; (b) the parts in the URL where the tokens occur, e.g., protocol, domain name, and directory paths; (c) length of the tokens; (d) or­thographic features; (e) sequential n-grams; and (f) sequential bigrams. As each of these features can be seen as a "token", the output of the MeURLin seg­menter for a web page can be seen as a "document", and hence it is possible to compute the TFx IDF co­sine similarity between two such documents.

We present the clustering performances of the var­ious methods in our system based on the different features that we extracted. Each experiment uses HAC with single linkage clustering. Since the num­ber of clusters is not known, when to terminate the agglomeration process is a crucial point and signifi­cantly affects the quality of the clustering result. We empirically determine the best similarity thresholds to be 0.1 and 0.2 for all the experiments on the three different data sets provided. We found that larger values for these data sets do not allow the HAC algo­rithm to create enough clustering hierarchy by caus­ing it to terminate early, and therefore result in many small clusters increasing purity but dramatically suf­fering from inverse purity performance.

Table 1 shows the results of our experiments on the training data sets (ECDL, Wikipedia and Cen­sus). Two different evaluation measures are reported as described by the task: Fa=05 is a harmonic mean of purity and inverse purity of the clustering result, and Fa=02 is a version of F that gives more impor­tance to inverse purity (Artiles et al., 2007).

Among the individual features, Tokens and Named Entity features consistently show close to best performance for all training data sets. In most cases, NE is better than Tokens because some web pages contain lots of irrelevant text for this task (e.g., headers and footers, menus etc). Also, we found that the NEs have far more discriminative power than most other tokens in determining similarity between web pages. The NE variation, NE targeted, performs worse among the token based methods. Although NE targeted aims for highly precise disambiguation, it seems that it throws away too much information so that inverse purity is very much reduced. The other NEs, such as locations and organizations are also very helpful for this task. For example, the or­ganization may indicate the affiliation of a particular name. This explains the superiority of NE over NE targeted for all three data sets.

Among the link based features, Domain gives bet­ter performance over Host as it leads to better in­verse purity. The reason is that there are usually many pages on different hosts from a single domain for a given name (e.g., the web pages belonging to a researcher from university domain). This greatly helps in resolving the name while results in a slight drop in purity. Using a web page's URL itself in the features Host+Self and Domain+Self shows a larger increase in inverse purity at a smaller decrease in pu­rity, hence these have improved F-measure in com­parison to Domain and Host. Not surprisingly, these link based features perform very well for the ECDL data set, compared to the other two. A significant portion of the people in the ECDL data set are most likely present-day computer scientists, likely having extensive an web presence, which makes the task much easier. Although the other two data sets may have popular people with many web pages, their web presence are usually created by others and often scatter across many domains with little hyperlink-age between them. This explains why our link based methods are not very effective for such data sets.

Our final individual feature URL performs worst among all. Although highly precise, its resulting in­verse purity is poor. While the features generated by MeURLin do improve the performance over pure host name and domain on the page URLs, its incor­poration in a richer feature set does not lead to better results, as the other features which have richer infor­mation to process.

Each of the individual features has different de­gree of discriminative power in many different cases. By combining them, we expect to get bet­ter performance than individually. However, we do not obtain significant improvement in any of the data sets. Furthermore, in the Census data set, the com­bined features fail to outperform the individual NE and Tokens features. The relatively poor perfor­mance of the remaining features also degrades the performance of Tokens and NE when combined.

Considering the performances using the harmonic mean, we do not see any clear winner in all of three

Table 1: Experimental results for each training data set of the task: ECDL, Wikipedia and Census. Each experiment uses single link HAC with the similarity threshold values of 0.1 / 0.2. Best Fa=05 performances are shown in bold.

training data sets. In addition, the method showing the best performance does not result in a win with a large margin in each data set. Relatively com­plicated methods do not always perform better over simpler, single featured based methods on all train­ing data sets. Considering the results and Occam's razor (Thorburn, 1915), we conclude that a simple method should most likely work relatively well in many other different settings as well. Therefore, we selected the method based on the individual NE fea­ture with the similarity threshold value of 0.2 for the final test submission run. We are able to achieve the following results for this submission run: pu­rity = 0.73, inverse purity = 0.82, Fa=0 5 = 0.75, F«=o.2 = 0.78.

We described our PSNUS system thatdisambiguates people mentions in web pages returned by a web search scenario, as defined in the inaugural Web People Search Task. As such, we mainly focus on extracting various kinds of information from web pages and utilizing them in the similarity computa­tion of the clustering algorithm. The experimental results show that a simple Hierarchical Agglomera-tive Clustering approach using a single named entity feature seems promising as a robust solution for the various datasets.