Cross-document coreference resolution is the task of identifying if two mentions of the same (or similar) name in different sources refer to the same individ­ual. Deciding if two documents refer to the same individual is a difficult problem because names are highly ambiguous. Automatic techniques for solv­ing this problem are required not only for better ac­cess to information but also in natural language pro­cessing applications such as multidocument summa­rization and information extraction. Here, we con­centrate on the following SemEval 2007 Web Peo­ple Search Task (Artiles et al., 2007): a search en­gine user types in a person name as a query. Instead of ranking web pages, an ideal system should orga­nize search results in as many clusters as there are different people sharing the same name in the doc­uments returned by the search engine. The input is, therefore, the results given by a web search engine using a person name as query. The output is a number of sets, each containing documents referring to the same individual.

As past and recent research (Bagga and Baldwin, 1998; Phan et al., 2006), we have addressed the problem as a document clustering problem. For our first participation in SemEval 2007, we use two ap­proaches: a lexical or bag-of-words approach and a semantic based approach. We have implemented our own clustering algorithms but rely on available ex­traction and summarization technology developed in our laboratory to produce document representations used as input for the clustering procedure.

We have implemented an agglomerative clustering algorithm. The input to the algorithm is a set of document representations implemented as vectors of terms and weights. Initially, there are as many clus­ters as input documents; as the algorithm proceeds clusters are merged until a certain termination condi­tion is reached. The algorithm computes the similar­ity between vector representations in order to decide whether or not to merge two clusters. The similar­ity metric we use is the cosine of the angle between two vectors. This metric gives value one for identi­cal vectors and zero for vectors which are orthogo­nal (non related). Various options have been imple­mented in order to measure how close two clusters are, but for the experiments reported here we have used the following approach: the similarity between two clusters (simc) is equivalent to the "document" similarity (sim ) between the two more similar doc­uments in the two clusters; the following formula is used:

The input for analysis is a set of documents and a person name (first name and last name). The documents are analysed by the default GATEANNIE system (Cunningham et al., 2002) and single document summarization modules (Saggion and Gaizauskas, 2004b) from our summarization toolkit. No attempt is made to analyse or use con­textual information given with the input document. The processing elements include:

• Document tokenisation Sentence splitting Parts-of-speech tagging

• Named Entity Recognition using a gazetteer lookup module and regular expressions

Named entity coreference using an ortho­graphic name matcher

Named entities of type person, organization, ad­dress, date, and location are considered relevant document terms and stored in a special named en­tity called Mention.

Coreference chains are created and analysed and if they contain an entity matching the target person's surname, all elements of the chain are marked. Ex­tractive summaries are created for each document, a sentence belongs to the summary if it contains a mention which is coreferent with the target entity.

Using language resources creation modules from the summarization tool, two frequency tables are created for each document set (or person): (i) an in­verted document frequency table for words (no nor­malisation is applied); and (ii) an inverted frequency table for Mentions (the full entity string is used, no normalisation is applied).

Statistics (term frequencies and tf*idf) are com­puted over tokens and Mentions using the appropri­ate tables (these tools are part of the summarization toolkit) and vector representations created for each document (same as in (Bagga and Baldwin, 1998)). Two types of representations were considered for these experiments: (i) full document or summary (terms in the summary are considered for vector cre­ation); and (ii) words or Mentions.

Four system configurations were prepared for Se-mEval:

System I: vector representations were created for full documents. Words were used as terms and local inverted document frequencies used (word frequencies) for weighting.

System II: vector representations were created for full documents. Mentions were used as terms and local inverted document frequencies used (Mentions frequencies) for weighting.

System III: vector representations were created for person summaries. Words were used as terms and local inverted document frequencies used (word frequencies) for weighting.

• System IV: vector representations were created for person summaries. Mentions were used as terms and local inverted document frequencies used (Mentions frequencies) for weighting.

Because only one system configuration was al­lowed per participant team, we decided to select System II for official evaluation interested in eval­uating the effect of semantic information in the clus­tering process.

Evaluation of the task was carried out using standard clustering evaluation measures of "purity" and "in­verse purity" (Hotho et al., 2003), and the harmonic

Table 1: Results for our configurations omitting one set. System II was the system we evaluated in SemEval 2007.

mean of purity and inverse purity: F-score. We esti­mated the threshold for the clustering algorithm us­ing the ECDL subset of the training data provided by SemEval. We applied the clustering algorithm to each document set and computed purity, inverse pu­rity, and F-score at each iteration of the algorithm, recording the similarity value of each newly created cluster. The similarity values for the best clustering results (best F-score) were recorded, and the max­imum and minimum values discarded. The rest of the values were averaged to obtain an estimate of the optimal threshold. Two different thresholds were obtained: 0.10 for word vectors and 0.12 for named entity vectors.

Results for the test set in SemEval are presented in Table 1 (One set - "Jerry Hobbs" - was ignored when computing these numbers: due to a failure during document analysis this set could not be clus­tered. The error was identified too close to the sub­mission's date to allow us to re-process the cluster). Our official submission System II (SHEF in the offi­cial results) obtained an F-score of 0.66 positioning itself in 5th place (out of 16 systems). Our best con­figuration obtained 0.74 F-score, so a fourth place would be in theory possible.

Our system obtained an F-score greater than the average of 0.60 of all participant systems. Our optimal configurations (System I and System II) both perform similarly with respect to F-score. While System I favours "inverse purity", System III favours "purity". Results for every individual set are reported in the Appendix.

We have presented a system used to participate in the SemEval 2007 Web People Search task. The system uses an in-house clustering algorithm and available extraction and summarization techniques to produce representations needed by the clustering algorithm. Although the configuration we submit­ted was suboptimal, we have obtained good results; in fact all our system configurations produce results well above the average of all participants. Our future work will explore how the use of contextual infor­mation available on the web can lead to better per­formance. We will explore if a similar approach to our method for creating profiles or answering def­inition questions (Saggion and Gaizauskas, 2004a) which uses co-occurence information to identify pieces of information related to a given entity can be applied here.

This work was partially supported by the EU-funded MUSING project (IST-2004-027097) and the EU-funded LIRICS project (eContent project 22236).