<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="10"/><title>Identifying Similar and Co-referring Documents Across Languages</title><pubinfo>The 2nd International Workshop On "Cross Lingual Information Access Addressing the Information Need of Multilingual Societies, 2008</pubinfo><author surname="T" givenname="Pattabhi R K Rao"><org  name="AU-KBC Research Center" country="India" city="Chennai"/></author><author surname="L" givenname="Sobha"><org  name="Anna University" country="India" city="Chennai"/></author></firstpageheader><frontmatter><p><b>Identifying Similar and Co-referring Documents Across Languages</b></p><p><b>Pattabhi R K Rao T</b></p><p>AU-KBC Research Centre, MIT Campus, Anna University, Chennai-44, India.</p><p>pattabhi@au-kbc.org</p><p><b>Sobha L</b></p><p>sobha@au-kbc.org</p></frontmatter><abstract>This paper presents a methodology for finding similarity and co-reference of documents across languages. The similarity between the documents is identified ac­cording to the content of the whole docu­ment and co-referencing of documents is found by taking the named entities present in the document. Here we use Vector Space Model (VSM) for identifying both similar­ity and co-reference. This can be applied in cross-lingual search engines where users get documents of very similar content from different language documents. </abstract></header><body><section number="1" title="Introduction"><p>In this age of information technology revolution, the growth of technology and easy accessibility has contributed to the explosion of text data on the web in different media forms such as online news magazines, portals, emails, blogs etc in different languages. This represents 80% of the unstructured text content available on the web. There is an ur­gent need to process such huge amount of text us­ing Natural Language Processing (NLP) tech­niques. One of the significant challenges with the explosion of text data is to organize the documents into meaningful groups according to their content.</p><p>The work presented in this paper has two parts a) finding multilingual cross-document similarity and b) multilingual cross-document entity co-referencing. The present work analyzes the docu­ments and identifies whether the documents are similar and co-referring. Two objects are said to be similar, when they have some common properties between them. For example, two geometrical fig­ures are said to be similar if they have the same shape. Hence similarity is a measure of degree of resemblance between two objects.</p><p>Two documents are said to be similar if their contents are same. For example a document D1 describes about a bomb blast incident in a city and document D2 also describes about the same bomb blast incident, its cause and investigation details, then D1 and D2 are said to be similar. But if document D3 talks of terrorism in general and ex­plains bomb blast as one of the actions in terrorism and not a particular incident which D1 describes, then documents D1 and D3 are dissimilar. The task of finding document similarity differs from the task of document clustering. Clustering is a task of categorization of documents based on domain/field. In the above example, documents D1, D2, D3 can be said to be in a cluster of crime domain. When documents are similar they share common noun phrases, verb phrases and named entities. While in document clustering, sharing of named entities and noun phrases is not essential but still there can be some noun phrases and named entities in common. Cross-document co-referencing of entities refers to the identification of same entities across the docu­ments. When the named entities present in the documents which are similar and also co-referencing, then the documents are said to be co-referring documents.</p><p>The paper is further organized as follows. In section 2, the motivation behind this paper is ex­plained and in 3 the methodology used is described. Results and discussions are dealt in section 4 and conclusion in section 5.</p><page local="2" global="11"/></section><section number="2" title="Motivation"><p>Dekang Lin (1998) defines similarity from the in­formation theoretic perspective and is applicable if the domain has probabilistic model. In the past decade there has been significant amount of work done on finding similarity of documents and orga­nizing the documents according to their content. Similarity of documents are identified using differ­ent methods such as Self-Organizing Maps (SOMs) (Kohonen et al, 2000; Rauber, 1999), based on On­tologies and taxanomy (Gruber, 1993; Resnik, 1995), Vector Space Model (VSM) with similarity measures like Dice similarity, Jaccard's similarity, cosine similarity (Salton, 1989). Bagga (Bagga et al., 1998) have used VSM in their work for finding co-references across the documents for English documents. Chung and Allan (2004) have worked on cross-document co-referencing using large scale corpus, where they have said ambiguous names from the same domain (here for example, politics) are harder to disambiguate when compared to names from different domains. In their work Chung and Allan compare the effectiveness of dif­ferent statistical methods in cross-document co-reference resolution task. Harabagiu and Maiorano (2000) have worked on multilingual co-reference resolution on English and Romanian language texts. In their system, "SWIZZLE" they use a data-driven methodology which uses aligned bilingual corpora, linguistic rules and heuristics of English and Romanian documents to find co-references. In the Indian context, obtaining aligned bilingual cor­pora is difficult. Document similarity between In­dian languages and English is tough since the sen­tence structure differs and Indian languages are agglutinative in nature. In the recent years there has been some work done in the Indian languages, (Pattabhi et al, 2007) have used VSM for multilin­gual cross-document co-referencing, for English and Tamil, where no bilingual aligned corpora is used.</p><p>One of the methods used in cross-lingual infor­mation retrieval (CLIR) is Latent Semantic Analy­sis (LSA) in conjunction with multilingual parallel aligned corpus. This approach works well for in­formation retrieval task where it has to retrieve most similar document in one language to a query given in another language. One of the drawbacks of using LSA in multilingual space for the tasks of document clustering, document similarity is that it gives similar documents more based on the lan­guage than by topic of the documents in different languages (Chew et al, 2007). Another drawback of LSA is that the reduced dimension matrix is dif­ficult to interpret semantically. The examples in <u>Table 1, illustrate this.</u>_</p><p>In the first example the component <i>(1.2810*car+0.5685*truck) </i>can be inferred as "Vehicle" but in cases such as in second example, the component <i>(1.2810*car+0.5685*bottle) </i>does not have any interpretable meaning in natural lan­guage. In LSA the dimension reduction factor 'k' has very important role to play and the value of 'k' can be found by doing several experiments. The process of doing dimension reduction in LSA is computationally expensive. When LSA is used, it reduces the dimensions statistically and when there is no parallel aligned corpus, this can not be inter­preted semantically.</p><p>Hence, in the present work, we propose VSM which is computationally simple, along with cosine similarity measure to find document similarity as well as entity co-referencing. We have taken Eng­lish and three Dravidian languages viz. Tamil, Te-lugu and Malayalam for analysis.</p></section><section number="3" title="Methodology"><p>In VSM, each document is represented by a vector which specifies how many times each term occurs in the document (the term frequencies). These counts are weighted to reflect the importance of each term and weighting is the inverse document frequency (idf). If a term t occurs in n documents in the collection then the <i>"idf" </i>is the inverse of log n. This vector of weighted counts is called a "bag of words" representation. Words such as "stop words" (or function words) are not included in the representation.</p><p>The documents are first pre-processed, to get syntactic and semantic information for each word in the documents. The preprocessing of documents involves sentence splitting, morph analysis, part-of-speech (POS) tagging, text chunking and named entity tagging. The documents in English are preprocessed using Brill's Tagger (Brill, 1994) for POS tagging and fn-TBL (Ngai and Florian, 2001) for text chunking.<page local="3" global="12"/> The documents in Indian lan­guages are preprocessed, using a generic engine (Arulmozhi et al., 2006) for POS tagging, and text chunking based on TBL (Sobha and Vijay, 2006). For both English and Indian language documents the named entity tagging is done using Named En­tity Recognizer (NER) which was developed based on conditional random field (CRF). The tagset used by the NER tagger is a hierarchical tagset, consists of mainly i) ENAMEX, ii) NUMEX and iii) TIMEX. Inside the ENAMEX there are mainly 11 subtype's viz. a) Person b) Organization c) Lo­cation d) Facilities e) Locomotives f) Artifacts g) Entertainment h) Cuisines i) Organisms j) Plants k) Disease. For the task of multilingual cross-document entities co-referencing, the documents are further processed for anaphora resolution where the corresponding antecedents for each ana-phor are tagged in the document. For documents in English and Tamil, anaphora resolution is done using anaphora resolution system. For documents in Malayalam and Telugu anaphora resolution is done manually. After the preprocessing of docu­ments, the language model is built by computing the term frequency - inverse document frequency (tf-idf) matrix. For the task of finding multilingual cross-document similarity, we have performed four different experiments. They are explained below:</p><table caption="Table 1. LSA Example" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Before Reduction</b></p></td><td class="cell"><p><b>After Reduction</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>{(car),(truck),(flower)}</p></td><td class="cell"><p>{(1.2810*car+0.5685*tr uck),(flower)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>{(car),(bottle),(flower)}</p></td><td class="cell"><p>{(1.2810*car+0.5685*b ottle),(flower)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p><b>E1: </b>The terms are taken from documents after removing the stop words. These are raw terms where no preprocessing of documents is done; the terms are unique words in the document collection.</p><p><b>E2: </b>The terms taken are the words inside the noun phrases, verb phrases and NER expressions after removing the stop words.</p><p><b>E3: T</b>he whole noun phrase/verb phrase/NER expression is taken to be a single term.</p><p><b>E4: T</b>he noun phrase/NER expression along with the POS tag information is taken as a single term.</p><p>The first experiment is the standard VSM im­plementation. The rest three experiments differ in the way the terms are taken for building the VSM. For building the VSM model which is common for all language document texts, it is essential that there should be translation/transliteration tool. First the terms are collected from individual language documents and a unique list is formed. After that, using the translation/transliteration tool the equiva­lent terms in language L2 for language L1 are found. The translation is done using a bilingual dictionary for the terms present in the dictionary. For most of the NERs only transliteration is possi­ble since those are not present in the dictionary. The transliteration tool is developed based on the phoneme match it is a rule based one. All the In­dian language documents are represented in roman notation (wx-notation) for the purpose of process­ing.</p><p>After obtaining equivalent terms in all lan­guages, the VSM model is built. Let S1 and S2 be the term vectors representing the documents D1 and D2, then their similarity is given by equation (1) as shown below.</p><doubt alpha="34.4" length="32" tooSmall="False" monospace="0.0">Sim(S1,S2) =I(W1j x W2j ) -- (1)</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">tj</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">Where,</doubt><p>tj is a term present in both vectors S1and S2. W1j is the weight of term tj in S1 and W2j is the weight of term tj in S2.</p><p>The weight of term tj in the vector S1 is calculated by the formula given by equation (2), below.</p><doubt alpha="40.4" length="52" tooSmall="False" monospace="0.0">Wij=(tf*log(N/df))/[sqrt(Sn2+Si22+......+Sm2)] --(2)</doubt><p>tf = term frequency of term tj</p><p>N=total number of documents in the collection</p><p>df = number of documents in the collection that the term tj occurs in.</p><p>sqrt represents square root The denominator [sqrt(Si1<footnote anchor="2"/>+Si2<footnote anchor="2"/>+......+Sin<footnote anchor="2"/>)] is the co­sine normalization factor. This cosine normalization factor is the Euclidean length of the vector where 'i' is the document number in the collection and Sin<footnote anchor="2"/> is the square of the product of (tf*log(N/df)) for term tn in the vector Si.</p><p>For the task of multilingual cross-document en­tity co-referencing, the words with-in the anaphor tagged sentences are considered as terms for build­ing the language model.</p></section><section number="4" title="Results and Discussion"><p>The corpus used for experiments is collected from online news magazines and online news portals. The sources in English include "The Hindu", "Times of India", "Yahoo News", "New York<page local="4" global="13"/></p><doubt alpha="60.5" length="43" tooSmall="False" monospace="0.0">Times", "Bangkok Post", "CNN", "WISC", "The</doubt><p>Independent". The sources for Tamil include "Di-namani", "Dinathanthi", "Dinamalar", "Dina-karan", and "Yahoo Tamil". The work was primar­ily done using English and Tamil. Later on this was extended for Malayalam and Telugu. The data sources for Malayalam are "Malayala Manorama", "Mathrubhumi", "Deshabhimani", "Deepika" and sources for Telugu include "Eenadu", "Yahoo Te-lugu" and "Andhraprabha". First we discuss about English and Tamil and Later Telugu and Malaya-lam.</p><p>The domains of the news taken include sports, business, politics, tourism etc. The news articles were collected using a crawler, and hence we find in the collection, a few identical news articles be­cause they appear in different sections of the news magazine like in Front page section, in state sec­tion and national section.</p><p>The dataset totally consists of 1054 English news articles, 390 Tamil news articles. Here we discuss results in two parts; in the first part results pertaining to document similarity are explained. In second part we discuss results on multilingual cross-document entity co-referencing.</p><subsection number="4.1" title="Document Similarity"><p>The data collection was done in four instances, spread in a period of two months. At the first in­stance two days news was crawled from different news sources in English as well as Tamil. In the first set 1004 English documents and 297 Tamil documents were collected.</p><p>In this set when manually observed (human judgment) it was found that there are 90 similar documents forming 31 groups, rest of the docu­ments were not similar. This is taken as gold stan­dard for the evaluation of the system output.</p><p>As explained in the previous section, on this set the four experiments were performed. In the first experiment (E1), no preprocessing of the docu­ments was done except that the stop words were removed and the language model was built. In this it was observed that the number of similar docu­ments is 175 forming 25 groups. Here it was ob­served that along with actual similar documents, system also gives other not similar documents (ac­cording to gold standard) as similar ones. This is due to the fact there is no linguistic information given to the system, hence having words alone does not tell the context, or in which sense it is used. And apart from that named entities when split don't give exact meaning, for example in name of hotels "Leela Palace" and "Mysore Pal­ace", if split into words yields three words, "Leela", "Mysore", and "Palace". In a particular document, an event at hotel Leela Palace is de­scribed and the hotel is referred as Leela Palace or by Palace alone. Another document describes about Dussera festival at Mysore Palace. Now here the system identifies both these documents to be similar even though both discuss about different events. The precision of the system was observed to be 51.4%, where as the recall is 100% since all the documents which were similar in the gold stan­dard is identified. Here while calculating the preci­sion; we are considering the number of documents that are given by the system as similar to the num­ber of documents similar according to the gold standard.</p><p>Hence to overcome the above discussed prob­lem, we did the second experiment (E2) where only words which occur inside the noun phrases, verb phrases and named entities are considered as terms for building the language model. Here it is observed that the number of similar documents is 140 forming 30 groups. This gives a precision of 64.2% and 100% recall. Even though we find a significant increase in the precision but still there are large number of false positives given by the system. A document consists of noun phrases and verb phrases, when the individual tokens inside these phrases are taken; it is equivalent to taking almost the whole document. This reduces the noise. The problem of "Leela Palace" and "Mysore Palace" as explained in the previous paragraph still persists here.</p><p>In the third experiment (E3) the whole noun phrase, verb phrase and named entity is considered as a single term for building the language model. Here the phrases are not split into individual to­kens; the whole phrase is a single term for lan­guage model. This significantly reduces the num­ber of false positives given by the system. The sys­tem identifies 106 documents as similar documents forming 30 groups. Now the precision of the sys­tem is 84.9%. In this experiment, the problem of "Leela Palace" and "Mysore Palace" is solved. Though this problem was solved the precision of the system is low, hence we performed the fourth (E4) experiment.</p><p>In the fourth experiment (E4), the part-of-speech (POS) information is given along with the phrase for building the language model.<page local="5" global="14"/> It is observed that the precision of the system increases. The number of similar documents identified is 100 forming 31 groups. This gives a precision of 90% and a recall of 100%.</p><p>Another important factor which plays a crucial role in implementation of language model or VSM is the threshold point. What is the threshold point that is to be taken? For obtaining an answer for this question, few experiments were performed by set­ting the threshold at various points in the range 0.75 to 0.95. When the threshold was set at 0.75 the number of similar documents identified by the system was larger, not true positives but instead false positives. Hence the recall was high and pre­cision was low at 50%. When the threshold was moved up and set at 0.81, the number of similar documents identified was more accurate and the number of false positives got reduced. The preci­sion was found to be 66%. When the threshold was moved up still further and set at 0.90, it was found that the system identified similar documents which were matching with the human judgment. The precision of the system was found to be 90%. The threshold was moved up further to 0.95, think­ing that the precision would further improve, but this resulted in documents which were actually similar to be filtered out by the system. Hence the threshold chosen was 0.9, since the results ob­tained at this threshold point had matched the hu­man judgment. For the experiments E1, E2, E3 and E4 explained above, the threshold is fixed at 0.9.</p><p>A new set of data consisting of 25 documents from 5 days news articles is collected. This is com­pletely taken from single domain, terrorism. These news articles describe specifically the Hyderabad bomb blast, which occurred on August 25th 2007. All these 25 documents were only English docu­ments from various news magazines. This data set was collected specifically to observe the perform­ance of the system, when the documents belonging to single domain are given. In the new data set, from terrorism domain, human judgment for docu­ment similarity was found to have 13 similar docu­ments forming 3 groups. While using this data set the noun phrases, verb phrases and named entities along with POS information were taken as terms to build the language model and the threshold was set at 0.9, it was observed that the system finds 14 documents to be similar forming 3 groups. Here, out of 14 similar documents, only 12 documents match with the human judgment and one document which ought to be identified was not identified by the system. The document which was not identified described about the current event, that is, bomb blast on 25th August in the first paragraph and then the rest of the document described about the simi­lar events that occurred in the past. Hence the simi­larity score obtained for this document with respect to other documents in the group was 0.84 which is lower than the threshold fixed. Hence the recall of the system is 92.3% and the precision of the sys­tem is 85.7%.</p><p>Another data set consisting of 114 documents was taken from tourism domain. The documents were both in Tamil and English, 79 documents in Tamil and 35 documents in English. This data set describes various pilgrim places and temples in Southern India. The human annotators have found 21 similar documents which form a group of three. These similar documents describe about Lord Siva's and Lord Murugan's temples. The system obtained 25 documents as similar and grouped into three groups. Out of 25 documents obtained as similar, four were dissimilar. These dissimilar documents described non-Siva temples in the same place. In these dissimilar documents the names of offerings, festivals performed were referred by the same names as in the rest of the documents of the group, hence these documents obtained similarity score of 0.96 with respect to other documents in the group. Here we get a precision of 84% and a recall of 100%.</p><p>A new data set consisting of 46 documents was taken from various news magazines. This set con­sists of 24 English documents, 11 Tamil docu­ments, 7 Malayalam documents and 4 Telugu documents. This data set describes the earthquake in Indonesia on 12th September 2007 and tsunami warning in other countries. The news articles were collected on two days 13th and 14th September 2007.</p><p>The documents collected were in different font encoding schemes. Hence before doing natural language processing such as morph-analysis, POS tagging etc, the documents were converted to a common roman notation (wx-notation) using the font converter for each encoding scheme.</p><p>Here we have used multilingual dictionaries of place; person names etc for translation. The lan­guage model is built by taking noun phrases and verb phrases along with POS information were as terms.<page local="6" global="15"/> In this set human annotators have found 45 documents to be similar and have grouped them into one group. The document which was identi­fied as dissimilar describes about a Tamil film shooting at Indonesia being done during the quake time. The system had identified all the 46 docu­ments including the film shooting document in the collection to be similar and put into one group. The <i>"film shooting" </i>document consisted of two para­graphs about the quake incident, other two para­graphs consisted of statement by the film producer stating that the whole crew is safe and the shooting is temporarily suspended for next few days. Since this document also contained the content describ­ing the earthquake found in other documents of the group, the system identified this <i>"film shooting" </i>document to be similar. Here one interesting point which was found was that all the documents gave a very high similarity score greater than 0.95. Hence the precision of the system is 97.8% and recall 100%.</p><p>The summary of all these experiments with dif­ferent dataset is shown in the table 2 below.</p><doubt alpha="29.6" length="27" tooSmall="False" monospace="0.0">Average| 89.S %    | 98.07%</doubt><table caption="Table 2. Summary of Results for Document similarity for four different data sets"></table></subsection><subsection number="4.2" title="Document Co-referencing"><p>The documents that were identified as similar ones are taken for entity co-referencing. In this work the identification of co-referencing documents is done for English and Tamil. In this section first we dis­cuss the co-referencing task for English documents in terrorism domain, then for documents in English and Tamil in Tourism domain. In the end of this section we discuss about documents in English and Tamil, which are not domain specific.</p><p>In the first experiment, the document collection in terrorism domain is taken for co-referencing task. This data set of 25 documents in terrorism domain consists of 60 unique person names. In this work we consider only person names for entity co-referencing. In this data set, 14 documents are identified as similar ones by the system. These 14 documents consist of 26 unique person names. .</p><p>The language model is built using only named entity terms and the noun, verb phrases occurring in the same sentence where the named entity oc­curs. POS information is also provided with the terms. Here we find that out of 26 entities, the sys­tem co-references correctly for 24 entities, even though the last names are same. The results ob­tained for these named entities is shown in the be­low table Table 3.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Entity Name</b></p></td><td class="cell"><p><b>No. of links containing the entity</b></p></td><td class="cell"><p><b>Correct</b></p><p><b>Responses</b></p><p><b>obtained</b></p></td><td class="cell"><p><b>Total Re­sponses ob­tained</b></p></td><td class="cell"><p><b>Precision</b></p></td><td class="cell"><p><b>Recall %</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Y S Ra-jasekhar</p><p>Reddy</p></td><td class="cell"><p>7</p></td><td class="cell"><p>7</p></td><td class="cell"><p>7</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Indrasena Reddy</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>K Jana Reddy</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Shivaraj Patil</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Manmohan Singh</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Abdul Sha-</p><p>hel</p><p>Mohammad</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>50</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mohammad Abdullah</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>50</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mohammad Amjad</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mohammad Yunus</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Ibrahim</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dawood Ibrahim</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Madhukar Gupta</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>N Chandra-babu Naidu</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Tasnim Aslam</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mahender Agrawal</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Somnath Chatterjee</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Pervez Musharaff</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sonia Gan­dhi</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Taslima</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>SNo</b></p></td><td class="cell"><p><b>Dataset</b></p></td><td class="cell"><p><b>Preci­sion %</b></p></td><td class="cell"><p><b>Recall</b></p><p><b>%</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>English 1004 and Tamil 297 documents</p></td><td class="cell"><p>90.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2</p></td><td class="cell"><p>English 25 - terrorism domain documents</p></td><td class="cell"><p>85.7</p></td><td class="cell"><p>92.S</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>S</p></td><td class="cell"><p>35 English Docs and Tamil 79 docs - Tour­ism domain</p></td><td class="cell"><p>84.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>4</p></td><td class="cell"><p>46 Docs on Earth Quake incident - 24 English, 11 Tamil, 7 Malayalam, 4 Telugu</p></td><td class="cell"><p>97.8</p></td><td class="cell"><p>100.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="7" global="16"/><p>The results obtained for these named entities are <u>shown in the table 4, below.</u></p><p>lish documents in terrorism domain</p><p>The system identifies the entity names ending with "Reddy" correctly. These names in the docu­ments occur along with definite descriptions which helps the system in disambiguating these names. For example "Y <i>S Rajasekhar Reddy" </i>in most cases is referred to as <i>"Dr. Reddy" </i>along with the defi­nite description <i>"chief minister". </i>Similarly the other name <i>"K Jana Reddy" </i>occurs with the defi­nite description <i>"Home minister". </i>Since here we are taking full noun phrases as terms for building language model, this helps obtaining good results. For entities such as "Abdul Shahel Mohammad" and "Mohammad Abdullah", it is observed that the both names are referred in the documents as "Mohammad" and surrounding phrases do not have any distinguishing phrases such as definite descriptions, which differentiate these names. Both these entities have been involved in masterminding of the Hyderabad bomb blast. Hence the system couldn't disambiguate between these two named entities and identifies both to be same, hence it fails here.</p><p>In the second experiment, the data set in Tour­ism domain consisting of 79 Tamil Documents and 35 English documents is taken for the task of co-referencing. In this data set 25 documents were identified as similar. Now these similar documents of 25 are considered for entity co-referencing task. There are 35 unique names of Gods. Here in this domain, one of the interesting points is that, there are different names to refer to a single God. For example Lord Murugan, is also referred by other names such as "Subramanyan", "Saravana", "Kart-tikeyan", "Arumukan" etc. Simialrly for Lord Siva is referred by "Parangirinathar", "Dharbaranes-wara" etc. It is observed that in certain documents the alias names are not mentioned along with common names. In these instances even human annotators found it tough for co-referencing, hence the system could not identify the co-references. This problem of alias names can be solved by hav­ing a thesaurus and using it for disambiguation.</p><p>The co-referencing system could disambiguate a document which was identified as similar by the system and dissimilar by the human annotator.</p><p>Another experiment is performed where both English and Tamil Documents are taken for entity co-referencing. In this experiment we have taken the data set in which there are 1004 English docu­ments and 297 Tamil documents. The documents are not domain specific. Here 100 documents are identified as similar ones, which contains of 64 English and 36 Tamil documents. Now we con­sider these 100 similar documents for entity co-referencing. In the 100 similar documents, there are 520 unique named entities. The table (Table 5) below shows results of few interesting named enti­ties in this set of 100 similar documents.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">16</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Nasrin</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Bandaru Dattatreya</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>L K Advani</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Average</b></p></td><td class="cell"><p>95.2</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 4. Results for entity co-referencing for English and Tamil Documents in Tourism domain" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Entity Name</b></p></td><td class="cell"><p><b>No. of links con­taining the</b></p></td><td class="cell"><p><b>Correct</b></p><p><b>Responses obtained</b></p></td><td class="cell"><p><b>Total Re­sponses obtained</b></p></td><td class="cell"><p><b>Precision</b></p></td><td class="cell"><p><b>Recall %</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Murugan</p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p>S</p></td><td class="cell"><p>Sv.5</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Shiva</p></td><td class="cell"><p>10</p></td><td class="cell"><p>9</p></td><td class="cell"><p>9</p></td><td class="cell"><p>100</p></td><td class="cell"><p>90</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Parvathi</p></td><td class="cell"><p>10</p></td><td class="cell"><p>9</p></td><td class="cell"><p>11</p></td><td class="cell"><p>Sl.S</p></td><td class="cell"><p>90</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Nala</p></td><td class="cell"><p>5</p></td><td class="cell"><p>5</p></td><td class="cell"><p>5</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Damayan-thi</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Narada</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sanees-warar</p></td><td class="cell"><p>6</p></td><td class="cell"><p>6</p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p>857</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Deivayani</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Vishnu</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Vinayaka</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Indra</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Thiruna-vukkarasar</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mayan</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Average</b></p></td><td class="cell"><p>96.5</p></td><td class="cell"><p>98.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Entity</b></p><p><b>Name</b></p></td><td class="cell"><p><b>No. of links containing the entity</b></p></td><td class="cell"><p><b>Correct</b></p><p><b>Responses obtained</b></p></td><td class="cell"><p><b>Total Re­sponses ob­tained</b></p></td><td class="cell"><p><b>Precision</b></p></td><td class="cell"><p><b>Recall %</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Karunanidhi</p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p><i>v</i></p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Manmohan Singh</p></td><td class="cell"><p>15</p></td><td class="cell"><p>14</p></td><td class="cell"><p></p></td><td class="cell"><p>Sv.5</p></td><td class="cell"><p>9S.S</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sonia Gandhi</p></td><td class="cell"><p>54</p></td><td class="cell"><p>54</p></td><td class="cell"><p>5S</p></td><td class="cell"><p>9S.1</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Shivaraj Patil</p></td><td class="cell"><p>S</p></td><td class="cell"><p>S</p></td><td class="cell"><p>10</p></td><td class="cell"><p>SO</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Prathibha Patil</p></td><td class="cell"><p>24</p></td><td class="cell"><p>24</p></td><td class="cell"><p>26</p></td><td class="cell"><p>92.S</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Lalu Prasad</p></td><td class="cell"><p>5</p></td><td class="cell"><p>5</p></td><td class="cell"><p>5</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="8" global="17"/><p>Table 5. Results for entity co-referencing for English and Tamil Documents not of any specific domain</p></subsection></section><section number="5" title="Conclusion"><p>The VSM method is a well known statistical method, but here it has been applied for multilin­gual cross-document similarity, which is a first of its kind. Here we have tried different experiments and found that using phrases with its POS informa­tion as terms for building language model is giving good performance. In this we have got an average precision of 89.3 and recall of 98.07% for docu­ment similarity. Here we have also worked on mul­tilingual cross-document entity co-referencing and obtained an average precision of 95.6 % and recall of 99.2 %. The documents taken for multilingual cross-document co-referencing are similar docu­ments identified by the similarity system. Consid­ering similar documents, helps indirectly in getting contextual information for co-referencing entities, because obtaining similar documents removes documents which are not in the same context. Hence this helps in getting good precision. Here we have worked on four languages viz. English, Tamil, Malayalam and Telugu. This can be applied for other languages too. Multilingual document similarity and co-referencing, helps in retrieving similar documents across languages.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Atal Bihari Va­jpayee</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>4</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Abdul Kalam</p></td><td class="cell"><p>22</p></td><td class="cell"><p>22</p></td><td class="cell"><p>22</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sania Mirza</p></td><td class="cell"><p>10</p></td><td class="cell"><p>10</p></td><td class="cell"><p>10</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Advani</p></td><td class="cell"><p>8</p></td><td class="cell"><p>8</p></td><td class="cell"><p>8</p></td><td class="cell"><p>100</p></td><td class="cell"><p>100</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Average</b></p></td><td class="cell"><p>95.S</p></td><td class="cell"><p>99.S</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Arulmozhi Palanisamy and Sobha Lalitha Devi. 2006.</p><p><i>HMM based POS Tagger for a Relatively Free Word Order Language, </i>Journal of Research on Computing Science, Mexico. 18:37-48.</p><p>Bagga, Amit and Breck Baldwin. 1998. <i>Entity-Based Cross-Document Coreferencing Using the Vector Space Model, </i>Proceedings of the 36th Annual Meet­ing of the Association for Computational Linguistics and the 17th International Conference on Computa­tional Linguistics (COLING-ACL'98):79-85.</p><p>Brill, Eric. 1994. <i>Some Advances in transformation Based Part of Speech Tagging, </i>Proceedings of the</p><p>Twelfth International Conference on Artificial Intel­ligence (AAAI-94), Seattle, WA</p><p>Peter A. Chew, Brett W. Bader, Tamara G. Kolda, Ah­med Abdelali. 2007. <i>Cross-Language Information Retrieval Using PARAFAC2, </i>In the Proceedings Thirteenth International Conference on Knowledge Discovery and Data Mining (KDD' 07), San Jose, California.:143-152.</p><p>Chung Heong Gooi and James Allan. 2004. <i>Cross-</i> <i>Document Coreference on a Large Scale Corpus, </i>Proceedings of HLT-NAACL: 9-16.</p><p>Dekang Lin. 1998. <i>An Information-Theoretic Definition of Similarity, </i>Proceedings of International Confer­ence on Machine Learning, Madison, Wisconsin,</p><p>July.</p><p>T. R. Gruber. 1993. <i>A translation approach to portable ontologies, </i>Knowledge Acquisition, 5(2):199-220.</p><p>Harabagiu M Sanda and Steven J Maiorano. 2000. <i>Mul­tilingual Coreference Resolution, </i>Proceedings of 6thApplied Natural Language Processing Conference:</p><p>142-149.</p><p>Kohonen, Teuvo Kaski, Samuel Lagus, Krista Salojarvi, Jarkko Honkela, Jukka Paatero,Vesa Saarela, Anti. 2000. <i>Self organisation of a massive document col­lection, </i>IEEE Transactions on Neural Networks, 11(3): 574-585.</p><p>G. Ngai and R. Florian. 2001. <i>Transformation-Based Learning in the Fast Lane, </i>Proceedings of the NAACL'2001, Pittsburgh, PA: 40-47</p><p>R K Rao Pattabhi, L Sobha, and Amit Bagga. 2007. <i>Multilingual cross-document co-referencing, </i>Pro­ceedings of 6th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC), March 29-30, 2007, Portugal:115-119</p><p>Rauber, Andreas Merkl, Dieter. 1999. <i>The SOMLib</i> <i>digital library system, </i>In the Proceedings of the 3rd European Conference on Research and Advanced Technology for Digital Libraries (ECDL'99), Paris,</p><p>France. Berlin: 323-341.</p><p>P. Resnik. 1995. <i>Using information content to evaluate semantic similarity in taxonomy, </i>Proceedings of</p><p>IJCAI: 448-453.</p><p>Salton, Gerald. 1989. <i>Automatic Text Processing: The Transformation, Analysis and Retrieval of Informa­tion by Computer, </i>Reading, MA: Addison Wesley</p><p>Sobha L, and Vijay Sundar Ram. 2006. <i>Noun Phrase Chunker for Tamil, </i>Proceedings of the First National Symposium on Modeling and Shallow Parsing of In­dian Languages (MSPIL), IIT Mumbai, India: 194­198.</p></references></body></article>