<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="174"/><title>BUAP: An Unsupervised Approach to Automatic Keyphrase Extraction from Scientific Articles</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 174-177, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Ortiz" givenname="Roberto"><org  name="YY Technologies" country="USA" city="Mountain View"/></author><author surname="Pinto" givenname="David"><org  name="YY Technologies" country="USA" city="Mountain View"/></author><author surname="Tovar" givenname="Mireya"><org  name="YY Technologies" country="USA" city="Mountain View"/></author><author surname="Jiménez-Salazar" givenname="Héctor"><org  name="YY Technologies" country="USA" city="Mountain View"/></author></firstpageheader><frontmatter><p><b>BUAP: An Unsupervised Approach to Automatic Keyphrase Extraction</b></p><p><b>from Scientific Articles</b></p><p><b>Roberto Ortiz, David Pinto, Mireya Tovar        Hector Jiménez-Salazar</b></p><p>Faculty of Computer Science, BUAP    Information Technologies Dept., UAM Puebla, Mexico DF, Mexico</p><p>korn_resorte2003@hotmail.com,        hgimenezs@gmail.com {dpinto, mtovar}@cs.buap.mx</p></frontmatter><abstract>In this paper, it is presented an unsuper­vised approach to automatically discover the latent keyphrases contained in scien­tific articles. The proposed technique is constructed on the basis of the combi­nation of two techniques: maximal fre­quent sequences and pageranking. We evaluated the obtained results by using micro-averaged precision, recall and F-scores with respect to two different gold standards: 1) reader's keyphrases, and 2) a combined set of author's and reader's keyphrases. The obtained results were also compared against three different base­lines: one unsupervised (TF-IDF based) and two supervised (Naïve Bayes and Maximum Entropy). </abstract></header><body><section number="1" title="Introduction"><p>The task of automatic keyphrase extraction has been studied for several years. Firstly, as semantic metadata useful for tasks such as summarization (Barzilay and Elhadad, 1997; Lawrie et al., 2001; DAvanzo and Magnini, 2005), but later rec­ognizing the impact that good keyphrases would have on the quality of various Nat­ural Language Processing (NLP) applica­tions (Frank et al., 1999; Witten et al., 1999; Turney, 1999; Barker and Corrnacchia, 2000; Medelyan and Witten, 2008). Thus, the selection of important, topical phrases from within the body of a document may be used in order to improve the performance of systems dealing with different NLP problems such as, clustering, question-answering, named entity recognition, information retrieval, etc.</p><p>In general, a keyphrase may be considered as a sequence of one or more words that capture the main topic of the document, as that keyphrase is expected to represent one of the key ideas ex­pressed by the document author. Following the previously mentioned hypothesis, we may take ad­vantage of two different techniques of text analy­sis: maximal frequent sequences to extract a se­quence of one or more words from a given text, and pageranking, expecting to extract those word sequences that represent the key ideas of the au­thor.</p><p>The interest on extracting high quality keyphrases from raw text has motivated forums, such as SemEval, where different systems may evaluate their performances. The purpose of SemEval is to evaluate semantic analysis systems. In particular, in this paper we are reporting the results obtained in Task #5 of SemEval-2 2010, which has been named: "Automatic Keyphrase Extraction from Scientific Articles". We focused this paper on the description of our approach and, therefore, we do not describe into detail the task nor the dataset used. For more information about this information read the "Task #5 Description paper", also published in this proceedings volume (Nam Kim et al, 2010).</p><p>The rest of this paper is structured as follows. Section 2 describes into detail the components of the proposed approach. In Section 3 it is shown the performance of the presented system. Finally, in Section 4 a discussion of findings and further work is given.</p></section><section number="2" title="Description of the approach"><p>The approach presented in this paper relies on the combination of two different techniques for select­ing the most prominent terms of a given text: max­imal frequent sequences and pageranking. In Fig­ure 1 we may see this two step approach, where we are considering a sequence to be equivalent to an n-gram. The complete description of the pro­cedure is given as follows.</p><p>We select maximal frequent sequences which we consider to be candidate keyphrases and, there­after, we ranking them in order to determine which ones are the most importants (according to the pageranking algorithm).<page local="2" global="175"/> In the following subsec­tions we give a brief description of these two tech­niques. Afterwards, we provide an algorithm of the presented approach.</p><p>Scientific articles</p><p>n-grams extraction</p><p>Pageranking</p><p>Keyphrases</p><p>Figure 1 : Two step approach of BUAP Team at the Task #5 of SemEval-2</p><subsection number="2.1" title="Maximal Frequent Sequences"><p><i>Definition: </i>If a sequence <i>p </i>is a subsequence of <i>q </i>and the number of elements in <i>p </i>is equal to <i>n, </i>then the <i>p </i>is called an n-gram in <i>q.</i></p><p><i>Definition: </i>A sequence <i>p = a\ ■ ■ ■ cik </i>is a sub­sequence of a sequence <i>q </i>if all the items <i>ai </i>occur in <i>q </i>and they occur in the same order as in <i>p. </i>If a sequence <i>p </i>is a subsequence of a sequence <i>q </i>we say that <i>p </i>occurs in <i>q.</i></p><p><i>Definition: </i>A sequence <i>p </i>is frequent in <i>S </i>if <i>p </i>is a subsequence of at least <i>ß </i>documents in <i>S </i>where <i>ß </i>is a given frequency threshold. Only one oc­currence of sequence in the document is counted. Several occurrences within one document do not make the sequence more frequent.</p><p><i>Definition: </i>A sequence <i>p </i>is a maximal frequent sequence in <i>S </i>if there does not exists any sequence <i>q </i>in <i>S </i>such that <i>p </i>is a subsequence of <i>q </i>and <i>p </i>is frequent in <i>S.</i></p></subsection><subsection number="2.2" title="PageRanking"><p>The algorithm of PageRanking was defined by Brin and Page in (Brin and Page, 1998). It is a graph-based algorithm used for ranking webpages. The algorithm considers input and output links of each page in order to construct a graph, where each vertex is a webpage and each edge may be the input or output links for this webpage. They denote as <i>In(Vi) </i>the set of input links of webpage <i>Vi, </i>and <i>Out (Vi) </i>their output links. The algorithm proposed to rank each webpage based on the vot­ing or recommendation of other webpages. The higher the number of votes that are cast for a ver­tex, the higher the importance of the vertex. More­over, the importance of the vertex casting the vote determines how important the vote itself is, and this information is also taken into account by the ranking model.</p><p>Although this algoritm has been initially pro­posed for webpages ranking, it has been also used for other NLP applications which may model their corresponding problem in a graph structure. Eq. (1) is the formula proposed by Brin and Page.</p><doubt alpha="37.5" length="16" tooSmall="False" monospace="0.0">S (Vi) =(l-d)+d*</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(i)</doubt><p>where <i>d </i>is a damping factor that can be set be­tween 0 and 1, which has the role of integrat­ing into the model the probability of jumping from a given vertex to another random vertex in the graph. This factor is usually set to 0<b>.85 </b>(Brin and Page, 1998).</p><p>There are some other propossals, like the one presented in (Mihalcea and Tarau, 2004), where a textranking algorithm is presented. The authors consider a weighted version of PageRank and present some applications to NLP using unigrams. They also construct multi-word terms by exploring the conections among ranked words in the graph. Our algorithm differs from textranking in that we use MFS for feeding the PageRanking algorithm.</p></subsection><subsection number="2.3" title="Algorithm"><p>The complete algoritmic description of the pre­sented approach is given in Algorithm 1. Read­ers and writers keyphrases may be quite dif­ferent. In particular, writers usually introduce acronyms in their text, but they use the complete or expanded representation of these acronyms for their keyphrases. Therefore, we have in­cluded a module <i>(Extract-Acronyms) </i>for ex­tracting both, acronyms with their corresponding expanded version, which are used afterwards as output of our system. We have preprocessed the dataset removing stopwords and punctuation sym­bols. Lemmatization (TreeTagger<footnote anchor="1"/> ) and stemming (Porter Stemmer (Porter, 1980)) were also applied in some stages of preprocessing.</p><p>The <i>MaximaLFreqSequences </i>module ex­tracts maximal frequent sequences of words and we feed the PageRaking module <i>(PageRanking)</i> with all these sequences for determining the most important ones.<page local="3" global="176"/> We use the structure of the sci­entific articles in order to determine <i>in </i>and <i>out </i>links of the sequences found. In fact, we use a neighborhood criterion (a pair of MF S in the same sentence) for determining the links between those MFS's. Once the ranking is calculated, we may se­lect those sequences of a given length (unigrams, bigrams and trigrams) as output of our system. We also return a maximum of three acronyms, and their associated multiterm phrases <i>(MultiTerm), </i>as candidate keyphrases. Determining the length and quantity of the sequences (n-grams) was ex­perimentally deduced from the training corpus.</p><footnote>http://www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/</footnote><doubt alpha="61.6" length="151" tooSmall="False" monospace="0.0">Algorithm 1: Algorithm of the Two Step ap­proach for the Task #5 at SemEval-2Input:A document set:D = {d\, d2, • • • }Output:A setK = {Ki,K2,• • • } of</doubt><p>keyphrases for each document <i>d{.</i></p><doubt alpha="31.6" length="19" tooSmall="False" monospace="0.0">Ki = {ki,i,k,2,■■■}</doubt></subsection></section><section number="1" title="foreach di G D do"></section><section number="2" title="AcronymSet = Extract_Acronyms (di);"><doubt alpha="66.7" length="24" tooSmall="False" monospace="0.0">3d\ =Pre_Processing(di);</doubt><p><b>4 </b><i>MF S = </i>Maximal_Freq_Sequences(d|); s      <i>CK = </i>PageRanking(d!, <i>M F S);</i></p></section><section number="6" title="CU = Top_Nine_Unigrams(CK);"><p>7 <i>CT = </i>Top_Three_Trigrams(CK); <i>Ki = CT; NU = </i>0; <i>Acronyms = </i>0; <b>foreach </b><i>unigram </i>G <i>CU </i><b>do</b></p><p><b>if </b><i>unigram </i>G <i>AcronymSet </i><b>then if </b><i>Acronyms &lt; </i>3 <b>then </b><i>Ki = Ki</i><i>[j</i><i> </i><i>{unigram}; EA = </i>MultiTermf/umgram); <i>K% = K% </i>U <i>{EA}; Acronyms++;</i></p><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">end</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">se</doubt><doubt alpha="56.0" length="25" tooSmall="False" monospace="0.0">Ki = Ki[j{unigram}; NU++;</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">27end</doubt><doubt alpha="45.0" length="20" tooSmall="False" monospace="0.0">28returnK = {K\, K2,</doubt><doubt alpha="56.7" length="67" tooSmall="False" monospace="0.0">N = [lb-{2*Acronyms+\CT\+NU)); CB =Top_N_Bigrams(CK,N); Ki=Ki\J CB;</doubt><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">}</doubt><p>In this edition of the Task #5 of SemEval-2 2010, we tested three different runs, which were named: <i>BUAP - </i>1, <i>BUAP - </i>2 and <i>BUAP - </i>3. Definition and differences among the three runs are given in Table 3.</p><p>The results obtained with each run, together with three different baselines are given in the fol­lowing section.</p></section><section number="3" title="Experimental results"><p>In all tables, <i>P, R, F </i>mean micro-averaged pre­cision, recall and F-scores. For baselines, there were provided 1,2,-3 grams as candidates and <i>TFIDF </i>as features. In Table 2, <i>TFIDF </i>is an unsupervised method to rank the candidates based on <i>TFIDF </i>scores. <i>NB</i><i> </i>and <i>ME</i><i> </i>are super­vised methods using Naïve Bayes and maximum entropy in WEKA. In second column, <i>R </i>means to use the reader-assigned keyword set as gold-standard data and <i>C </i>means to use both author-assigned and reader-assigned keyword sets as an­swers.</p><p>Notice from Tables 2 and 3 that we outper­formed all the baselines for the Top 15 candidates. However, the Top 10 candidates were only outper­formed by the .Reader-Assigned keyphrases found. This implies that the <i>Writer </i>keyphrases we ob­tained were not of as good as the <i>Reader </i>ones. As we mentioned, readers and writers assign different keywords. The former write keyphrases based on the lecture done, by the latter has a wider context and their keyphrases used to be more complex. We plan to investigate this issue in the future.</p></section><section number="4" title="Conclusions"><p>We have presented an approach based on the ex­traction of maximal frequent sequences which are then ranked by using the pageranking algorithm. Three different runs were tested, modifying the preprocessing stage and the number of bigrams given as output. We did not see an improve­ment when we used lemmatization of the docu­ments. The run which obtained the best results was ranking by the organizer according to the top 15 best keyphrases, however, we may see that our runs need to be analysed more into detail in order to provide a re-ranking procedure for the best 15 keyphrases found. This procedure may improve the top 5 candidates precision.</p><doubt alpha="0.0" length="13" tooSmall="False" monospace="0.0">1111111111222</doubt><doubt alpha="0.0" length="11" tooSmall="False" monospace="0.0">23 24 25 26</doubt><page local="4" global="177"/><p>Table 3 : The three different runs submitted to the competition</p></section><section title="Acknowledgments"><p>This work has been partially supported by CONA-CYT (Project #106625) and PROMEP (Grant #103.5/09/4213).</p><table caption="Table 1: Description of the three runs submitted to the Task #5 of SemEval-2 2010" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>Run name</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>Description</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP -</i></p></td><td class="cell"><p>1</p></td><td class="cell"><p>This run is exactly the one described in Algorithm 1.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP-</i></p></td><td class="cell"><p>2</p></td><td class="cell"><p>Same as <i>BUAP — </i>1 but lemmatization was applied a priori and stemming at the end.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP-</i></p></td><td class="cell"><p>3</p></td><td class="cell"><p>Same as <i>BU AP — </i>2 but output twice the number of bigrams.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Method</p></td><td class="cell"><p></p></td><td class="cell"><p>by</p></td><td class="cell"><p>top 5 candidates</p></td><td class="cell"><p>top 10 candidates</p></td><td class="cell"><p>top 15 candidates</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>TF - IDF</i></p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>17.80%</p></td><td class="cell"><p>7.39%</p></td><td class="cell"><p>10.44%</p></td><td class="cell"><p>13.90%</p></td><td class="cell"><p>11.54%</p></td><td class="cell"><p>12.61%</p></td><td class="cell"><p>11.60%</p></td><td class="cell"><p>14.45%</p></td><td class="cell"><p>12.87%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>22.00%</p></td><td class="cell"><p>7.50%</p></td><td class="cell"><p>11.19%</p></td><td class="cell"><p>17.70%</p></td><td class="cell"><p>12.07%</p></td><td class="cell"><p>14.35%</p></td><td class="cell"><p>14.93%</p></td><td class="cell"><p>15.28%</p></td><td class="cell"><p>15.10%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>NB</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>16.80%</p></td><td class="cell"><p>6.98%</p></td><td class="cell"><p>9.86%</p></td><td class="cell"><p>13.30%</p></td><td class="cell"><p>11.05%</p></td><td class="cell"><p>12.07%</p></td><td class="cell"><p>11.40%</p></td><td class="cell"><p>14.20%</p></td><td class="cell"><p>12.65%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>21.40%</p></td><td class="cell"><p>7.30%</p></td><td class="cell"><p>10.89%</p></td><td class="cell"><p>17.30%</p></td><td class="cell"><p>11.80%</p></td><td class="cell"><p>14.03%</p></td><td class="cell"><p>14.53%</p></td><td class="cell"><p>14.87%</p></td><td class="cell"><p>14.70%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>ME</i></p></td><td class="cell"><p></p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>16.80%</p></td><td class="cell"><p>6.98%</p></td><td class="cell"><p>9.86%</p></td><td class="cell"><p>13.30%</p></td><td class="cell"><p>11.05%</p></td><td class="cell"><p>12.07%</p></td><td class="cell"><p>11.40%</p></td><td class="cell"><p>14.20%</p></td><td class="cell"><p>12.65%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>21.40%</p></td><td class="cell"><p>7.30%</p></td><td class="cell"><p>10.89%</p></td><td class="cell"><p>17.30%</p></td><td class="cell"><p>11.80%</p></td><td class="cell"><p>14.03%</p></td><td class="cell"><p>14.53%</p></td><td class="cell"><p>14.87%</p></td><td class="cell"><p>14.70%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 2: Baselines</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Method</p></td><td class="cell"><p></p></td><td class="cell"><p>by</p></td><td class="cell"><p>top 5 candidates</p></td><td class="cell"><p>top 10 candidates</p></td><td class="cell"><p>top 15 candidates</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP —</i></p></td><td class="cell"><p>1</p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>10.40%</p></td><td class="cell"><p>4.32%</p></td><td class="cell"><p>6.10%</p></td><td class="cell"><p>13.90%</p></td><td class="cell"><p>11.54%</p></td><td class="cell"><p>12.61%</p></td><td class="cell"><p>14.93%</p></td><td class="cell"><p>18.60%</p></td><td class="cell"><p>16.56%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>13.60%</p></td><td class="cell"><p>4.64%</p></td><td class="cell"><p>6.92%</p></td><td class="cell"><p>17.60%</p></td><td class="cell"><p>12.01%</p></td><td class="cell"><p>14.28%</p></td><td class="cell"><p>19.00%</p></td><td class="cell"><p>19.44%</p></td><td class="cell"><p>19.22%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP —</i></p></td><td class="cell"><p>2</p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>10.40%</p></td><td class="cell"><p>4.32%</p></td><td class="cell"><p>6.10%</p></td><td class="cell"><p>13.80%</p></td><td class="cell"><p>11.46%</p></td><td class="cell"><p>12.52%</p></td><td class="cell"><p>14.67%</p></td><td class="cell"><p>18.27%</p></td><td class="cell"><p>16.27%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>14.40%</p></td><td class="cell"><p>4.91%</p></td><td class="cell"><p>7.32%</p></td><td class="cell"><p>17.80%</p></td><td class="cell"><p>12.14%</p></td><td class="cell"><p>14.44%</p></td><td class="cell"><p>18.73%</p></td><td class="cell"><p>19.17%</p></td><td class="cell"><p>18.95%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>BUAP -</i></p></td><td class="cell"><p>3</p></td><td class="cell"><p><i>R</i></p></td><td class="cell"><p>10.40%</p></td><td class="cell"><p>4.32%</p></td><td class="cell"><p>6.10%</p></td><td class="cell"><p>12.10%</p></td><td class="cell"><p>10.05%</p></td><td class="cell"><p>10.98%</p></td><td class="cell"><p>12.33%</p></td><td class="cell"><p>15.37%</p></td><td class="cell"><p>13.68%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>C</i></p></td><td class="cell"><p>14.40%</p></td><td class="cell"><p>4.91%</p></td><td class="cell"><p>7.32%</p></td><td class="cell"><p>15.60%</p></td><td class="cell"><p>10.64%</p></td><td class="cell"><p>12.65%</p></td><td class="cell"><p>15.67%</p></td><td class="cell"><p>16.03%</p></td><td class="cell"><p>15.85%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>[Barker and Corrnacchia2000] K. Barker and N. Cor-rnacchia. 2000. Using noun phrase heads to extract document keyphrases. In <i>13th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence.</i></p><p>[Barzilay andElhadadl997] R. Barzilay and M. El-hadad. 1997. Using lexical chains for text sum­marization. In <i>ACI/EACI 1997 Workshop on Intel­ligent Scalable Text Summarization, </i>pages 10-17.</p><p>[Brin and Page 1998] S. Brin and L. Page. 1998. The anatomy of a large-scale hypertextual web search engine. In <i>COMPUTER NETWORKS AND ISDN SYSTEMS, </i>pages 107-117. Elsevier Science Pub­lishers B. V.</p><doubt alpha="62.2" length="45" tooSmall="False" monospace="0.0">[DAvanzo and Magnini2005] E.      DAvanzo and</doubt><p>B. Magnini. 2005. A keyphrase-based approach to summarization:the lake system. In <i>Document Understanding Conferences (DUC-2005).</i></p><doubt alpha="59.6" length="52" tooSmall="False" monospace="0.0">[Franketal. 1999] E. Frank, G.W. Paynter, I. Witten,</doubt><p>C. Gutwin, and CG. Nevill-Manning. 1999. Do­main specific keyphrase extraction. In <i>16th Interna­tional Joint Conference on AI, </i>pages 668-673.</p><p>[Lawrieetal.2001] D. Lawrie, W. B. Croft, and A. Rosenberg. 2001. Finding topic words for hi­erarchical summarization. <i>InSIGIR 2001.</i></p><p>[Medelyan and Witten2008] O. Medelyan and I. H. Witten. 2008. Domain independent automatic keyphrase indexing with small training sets. <i>Jour­nal of American Societyfor Information Science and Technology, </i>59(7): 1026-1040.</p><p>[Mihalcea and Tarau2004] R. Mihalcea and P. Tarau. 2004. Textrank: Bringing order into texts. In <i>EMNLP 2004, ACI, </i>pages 404-411.</p><p>[NamKimetal.2010] S. Nam Kim, O. Medelyan, and M.Y. Kan. 2010. Semeval-2010 task5: Auto­matic keyphrase extraction from scientific articles.</p><p>In <i>Proceedings of the Fifth International Workshop on Semantic Evaluations (SemEval-2010). </i>Associa­tion for Computational Linguistics.</p><p>[Porterl980] M.F.Porter. 1980. An algorithm for suf­fix stripping. <i>Program, </i>14(3).</p><p>[Turneyl999] P. Turney. 1999. Learning to extract keyphrases from text. Technical Report ERB-1057. (NRC #41622), National Research Council, Institute for Information Technology.</p><p>[Witten et al. 1999] I. Witten, G. Paynter, E. Frank, C. Gutwin, and G. Nevill-Manning. 1999. Kea:practical automatic key phrase extraction. In <i>fourth ACM conference on Digital libraries, </i>pages 254-256.</p></references></body></article>