<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="158"/><title>SJTULTLAB: Chunk Based Method for Keyphrase Extraction</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 158-161, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Wang" givenname="Letian"><org  name="Shanghai Jiaotong University" country="China" city="Shanghai"/></author><author surname="Li" givenname="Fang"><org  name="Toyo University" country="Japan" city="Saitama"/></author></firstpageheader><frontmatter><p><b>SJTULTLAB: Chunk Based Method for Keyphrase Extraction</b></p><p><b>Letian Wang</b></p><p>Department of Computer Science &amp; Engineering Shanghai Jiao Tong University Shanghai, China <b>koh@s jtu.</b>edu.cn</p></frontmatter><abstract>In this paper we present a chunk based keyphrase extraction method for scientific articles. Different from most previous sys­tems, supervised machine learning algo­rithms are not used in our system. Instead, document structure information is used to remove unimportant contents; Chunk ex­traction and filtering is used to reduce the quantity of candidates; Keywords are used to filter the candidates before generating final keyphrases. Our experimental results on test data show that the method works better than the baseline systems and is comparable with other known algorithms. </abstract></header><body><section number="1" title="Introduction"><p>Keyphrases are sequences of words which cap­ture the main topics discussed in a document. Keyphrases are very useful in many natural lan­guage processing (NLP) applications such as doc­ument summarization, classification and cluster­ing. But it is an expensive and time-consuming job for users to tag keyphrases of a document. These needs motivate methods for automatic keyphrase extraction.</p><p>Most existing algorithms for keyphrase extrac­tion treat this task as a supervised classifica­tion task. The KEA algorithm (Gordon et al., 1999) identifies candidate keyphrases using lex­ical methods, calculates feature values for each candidate, and uses a machine-learning algorithm to predict which candidates are good keyphrases. A domain-specific method (Frank et al., 1999) was proposed based on the Naive Bayes learn­ing scheme. Turney (Turney, 2000) treated a document as a set of phrases, which the learn­ing algorithm must learn to classify as positive or negative examples of keyphrases. Turney (Tur­ney, 2003) also presented enhancements to the</p><doubt alpha="85.7" length="7" tooSmall="False" monospace="0.0">Fang Li</doubt><p>Department of Computer Science &amp; Engineering Shanghai Jiao Tong University Shanghai, China</p><footnote>fli@sjtu.edu.cn</footnote><p>KEA keyphrase extraction algorithm that are de­signed to increase the coherence of the extracted keyphrases. Nguyen and yen Kan (Nguyen and yen Kan, 2007) presented a keyphrase extraction algorithm for scientific publications. They also in­troduced two features that capture the positions of phrases and salient morphological phenom­ena. Wu and Agogino (Wu and Agogino, 2004) proposed an automated keyphrase extraction al­gorithm using a nondominated sorting multi-objective genetic algorithm. Kumar and Srinathan (Kumar and Srinathan, 2008) used n-gram filtra­tion technique and weight of words for keyphrase extraction from scientific articles.</p><p>For this evaluation task, Kim and Kan (Kim and Kan, 2009) tackled two major issues in au­tomatic keyphrase extraction using scientific ar­ticles: candidate selection and feature engineer­ing. They also re-examined the existing features broadly used for the supervised approach.</p><p>Different from previous systems, our system uses a chunk based method to extract keyphrases from scientific articles. Domain-specific informa­tion is used to find out useful parts in a document. The chunk based method is used to extract candi­dates of keyphrases in a document. Keywords of a document are used to select keyphrases from can­didates.</p><p>In the following, Section 2 will describe the ar­chitecture of the system. Section 3 will introduce functions and implementation of each part in the system. Experiment results will be showed in Sec­tion 4. The conclusion will be given in Section 5.</p></section><section number="2" title="System Architecture"><p>Figure 1 shows the architecture of our system. The system accepts a document as input (go through arrows with solid lines), then does the preprocess­ing job and identifies the structure of the docu­ment. After these two steps, the formatted doc­ument is sent to the candidate selection module<page local="2" global="159"/></p><doubt alpha="92.9" length="14" tooSmall="True" monospace="0.0">Input Document</doubt><doubt alpha="75.0" length="8" tooSmall="True" monospace="0.0">( Systei</doubt><doubt alpha="100.0" length="13" tooSmall="True" monospace="0.0">Preprocessing</doubt><doubt alpha="93.9" length="33" tooSmall="True" monospace="0.0">Document Structure Identification</doubt><doubt alpha="85.7" length="21" tooSmall="True" monospace="0.0">Formatted Document(s)</doubt><doubt alpha="94.7" length="19" tooSmall="True" monospace="0.0">Candidate Selection</doubt><doubt alpha="93.8" length="16" tooSmall="True" monospace="0.0">Chunk Extraction</doubt><doubt alpha="93.3" length="15" tooSmall="True" monospace="0.0">Chunk Filtering</doubt><doubt alpha="93.3" length="15" tooSmall="True" monospace="0.0">Chunk Selection</doubt><doubt alpha="94.7" length="19" tooSmall="True" monospace="0.0">Keywords Extraction</doubt><doubt alpha="94.7" length="19" tooSmall="True" monospace="0.0">Keyphrase Selection</doubt><p>Figure 1 : System architecture which first extracts chunks from the document, then uses some rules to filter the extracted chunks. After candidate selection, the system will choose top fifteen (ordered by the position of the first oc­currence in the original document) chunks from the candidates as the keyphrases and output the result ("Outputl" in Figure 1) which is our sub­mitted result. The candidates will also be sent to keyphrase selection module which first extracts keywords from the formatted document, then uses keywords to choose keyphrases from the candi­dates. Keywords extraction needs some training data (go through arrows with dotted lines) which also needs first two steps of our system. The result of keywords selection module will be sent to "Out-put2" as the final result after choosing top fifteen chunks.</p><p>OpenNLP<footnote anchor="1"/> and KEA<footnote anchor="2"/> are used in chunk extrac­tion and keywords extraction respectively.</p></section><section number="3" title="System Description 3.1 Preprocessing"><p>In preprocessing, our system first deletes line breaks between each broken lines to reconnect the broken sentences while line breaks after title and section titles will be reserved. Title and section titles are recognized through some heuristic rules that title occupies first few lines of a document and section titles are started with numbers except abstract and reference. The system then deletes brackets blocks in the documents to make sure no keyphrases will be splitted by brackets blocks (e.g., the brackets in "natural language processing (NLP) applications" could be an obstacle to ex­tracting phrase "natural language processing ap­plications").</p><footnote label="1">http : //opennlp. sourcef orge .net/</footnote><footnote label="2">http : //nzdl .org/Kea/</footnote><subsection number="3.2" title="Document Structure Identification"><p>Scientific articles often have similar structures which start with title, abstract and end with con­clusion, reference. The structure information is used in our system to remove unimportant con­tents in the input document. Based on the anal­ysis of training documents, we assume that each article can be divided into several parts: <i>Title, Ab­stract, Introduction, Related Work, Content, Ex­periment, Conclusion, Acknowledgement </i>and <i>Ref­erence, </i>where <i>Content </i>often contains the descrip­tion of theories, methods or algorithms.</p><p>To implement the identification of document structure, our system first maps each section ti­tle (including document title) to one of the parts in the document structure with some rules derived from the analysis of training documents. For each part except <i>Content, </i>we have a pattern to map the section titles. For example, the section title of <i>Ab­stract </i>should be equal to "abstract", the section ti­tle of <i>Introduction </i>should contain "introduction", the section title of <i>Related Work </i>should contain "related work" or "background", the section title of <i>Experiment </i>should contain "experiment", "re­sult" or "evaluation", the section title of <i>Conclu­sion </i>should contain "conclusion" or "discussion". Section titles which do not match any of the pat­terns will be mapped to the <i>Content </i>part. After mapping section titles, the content between two section titles will be mapped to the same part as the first section title (e.g., the content between the section title "1. Introduction" and "2. Related Work" will be mapped to the <i>Introduction </i>part).</p><p>In our keyphrase analysis, we observed that most keyphrases appear in the first few parts of a document, such as <i>Title, Abstract, </i>and <i>Introduc­tion. </i>We also found that parts like <i>Experiment, Acknowledgement </i>and <i>Reference </i>almost have no keyphrases.<page local="3" global="160"/> Thus, <i>Experiment, Acknowledgement </i>and <i>Reference </i>are removed by our system and other parts are sorted in their original order and outputted as formatted document(s) (see in Fig­ure 1) for further process.</p></subsection><subsection number="3.3" title="Candidate Selection"><p>The purpose of candidate selection is to find out potential keyphrases in a document. Traditional approaches just choose all the possible words se­quences and filters them with part-of-speech tags. This approach may result in huge amount of candi­dates and lots of meaningless candidates for each document.</p><p>Our system uses chunk based method to solve these problems.</p><p>"A chunk is a textual unit of adjacent word tokens which can be mutually linked through unambiguously identi­fied dependency chains with no recourse to idiosyncratic lexical information."<footnote anchor="3"/></p><p>Our approach significantly reduces the quantity of candidates and keep the meanings of origi­nal documents. For example, for an article ti­tle, "Evaluating adaptive resource management for distributed real-time embedded systems", the traditional method will extract lots of meaning­less candidates like "adaptive resource" and "dis­tributed real-time", while our method just extract "adaptive resource management" and "distributed real-time embedded systems" as candidates.</p><subsubsection number="3.3.1" title="Chunk Extraction"><p>The first step of candidate selection is chunk ex­traction which extract chunks from a document. Four tools in OpenNLP, <i>SentenceDetector, Tok-enizer, PosTagger </i>and <i>TreebankChunker, </i>are uti­lized in our system. The system first evokes <i>Sen­tenceDetector </i>to split the formatted document into sentences. Then uses <i>Tokenizer </i>and <i>PosTagger </i>to label all the words with part-of-speech tag. At last, <i>TreebankChunker </i>is used to extract chunks from the document.</p></subsubsection><subsubsection number="3.3.2" title="Chunk filtering"><p>Not all the extracted chunks can be the candidates of keyphrases. Our system uses some heuristic rules to select candidates from extracted chunks.</p><footnote label="3">http://www.ilc.cnr.it/sparkle/wp 1 -prefinaVnode24.html</footnote><p>The types of rules range from statistic informa­tion to syntactic structures. The rules that our sys­tem uses are based on some traditional methods for candidate filtering. They are:</p><p>1. Any chunks in candidates should have less than 5 words.</p><p>2. Any single word chunks in candidates should be found at least twice in a document.</p><p>3. Any chunks in candidates should be noun phrases.</p><p>4. Any chunks in candidates must start with the word with the part-of-speech tag (defined in OpenNLP) NN, NNS, NNP, NNPS, JJ, JJR or JJS and end with the word with the part-of-speech tag NN, NNS, NNP or NNPS. Chunks that do not match these rules will be removed. Chunks that haven't been removed will be the candidate keyphrases of the document.</p></subsubsection></subsection><subsection number="3.4" title="Keyphrase Selection"><p>Our analysis shows that keywords are helpful to extract keyphrases from a document. Thus, key­words are used to select keyphrases from candi­date chunks.</p><subsubsection number="3.4.1" title="Keywords Extraction"><p>KEA is a keyphrase extraction tool, it can also be used to extract keywords with some appropriate parameters. We observed that most keyphrases extracted by KEA only contain one word or two words which describe the key meaning of the doc­ument, even when the max length is set to 5 or more. There are four parameters to be set, in or­der to get best results, we set maximum length of a keyphrase to 2, minimum length of a keyphrase to 1, minimum occurrence of a phrase to 1 and number of keyphrases to extract to 30. Then, the output of the KEA system contains thirty keywords per document.</p><p>As showed in Figure 1, KEA needs training data (provided by the task owner). Our system uses for­matted documents (generated by the first two steps of our system) of training data as the input training data to KEA.</p></subsubsection><subsubsection number="3.4.2" title="Chunk Selection"><p>After extracting thirty keywords from each docu­ment, our system uses these keywords to filter out non-keyphrase chunks from the candidates. The system completes the task in two steps:<page local="4" global="161"/> 1) Re­move candidates of a document that do not have any keywords of the document extracted by KEA; 2) Choose the top fifteen (ordered by the position of the first occurrence in the orginal document) keyphrases as the answer of a document ("Out-put2" in Figure 1).</p></subsubsection></subsection></section><section number="4" title="Experiment Result"><p>Table 1 shows the F-score of two outputs of our system and some baseline systems. The first three methods are the baselines provided by the task owner. TFIDF is an unsupervised method to rank the candidates based on TFIDF scores. NB and ME are supervised methods using Navie Bayes and maximum entropy in WEKA<footnote anchor="4"/>. KEA refers to the KEA system with the parameters that can out­put the best results. OP1 is our system with the "Outputl" as result and OP 2 is our system with the "Output2" as result (see Figure 1). In second column, "R" means to use the reader-assigned keyphrases set as gold-standard data and "C" means to use both author-assigned and reader-assigned keyphrases sets as answers.</p><p>Table 1 : The comparison of F-score of our system with other systems.</p><p>From the table, we can see that, both two out­puts of our system made an improvement over the baseline systems and got better results than the well known KEA system.</p><p>We submitted both results of OP 1 and OP 2 to the evaluation task. Because of some misunder­standing over the result upload system, only the result of OP 1 (with bold style) was successfully submitted.</p><footnote label="4">http://www.cs.waikato.ac.nz/ml/weka/</footnote></section><section number="5" title="Conclusion"><p>We proposed a chunk based method for keyphrase extraction in this paper. In our system, document structure information of scientific articles is used to pick up significant contents, chunk based candi­date selection is used to reduce the quantity of can­didates and reserve their original meanings, key­words are used to select keyphrases from a docu­ment. All these factors contribute to the result of our system.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Method</b></p></td><td class="cell"><p><b>by</b></p></td><td class="cell"><p><b>Top05</b></p></td><td class="cell"><p><b>Top 10</b></p></td><td class="cell"><p><b>Topl5</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>TFIDF</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p>10.44%</p></td><td class="cell"><p>12.61%</p></td><td class="cell"><p>12.87%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p>11.19%</p></td><td class="cell"><p>14.35%</p></td><td class="cell"><p>15.10%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>NB</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p>9.86%</p></td><td class="cell"><p>12.07%</p></td><td class="cell"><p>12.65%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p>10.89%</p></td><td class="cell"><p>14.03%</p></td><td class="cell"><p>14.70%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>ME</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p>9.86%</p></td><td class="cell"><p>12.07%</p></td><td class="cell"><p>12.65%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p>10.89%</p></td><td class="cell"><p>14.03%</p></td><td class="cell"><p>14.70%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>KEA</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p>14.55%</p></td><td class="cell"><p>17.24%</p></td><td class="cell"><p>16.42%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p>14.45%</p></td><td class="cell"><p>17.68%</p></td><td class="cell"><p>17.74%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>OP1</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p><b>15.61%</b></p></td><td class="cell"><p><b>17.60%</b></p></td><td class="cell"><p><b>17.31%</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p><b>15.36%</b></p></td><td class="cell"><p><b>18.41%</b></p></td><td class="cell"><p><b>18.61%</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>OP2</b></p></td><td class="cell"><p>R</p></td><td class="cell"><p>16.08%</p></td><td class="cell"><p>18.42%</p></td><td class="cell"><p>18.05%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>C</p></td><td class="cell"><p>17.91%</p></td><td class="cell"><p>20.52%</p></td><td class="cell"><p>20.36%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Eibe Frank, Gordon W. Paynter, Ian H. Witten, Carl Gutwin, and Craig G. Nevill-manning. 1999. Domain-specific keyphrase extraction, pages 668-673. Morgan Kaufmann Publishers.</p><p>Ian Witten Gordon, Gordon W. Paynter, Eibe Frank, Carl Gutwin, and Craig G. Nevill-manning. 1999. Kea: Practical automatic keyphrase extraction. In <i>Proceedings of Digital Libraries 99 (DU99, </i>pages 254-255. ACM Press.</p><p>Su Nam Kim and Min-Yen Kan. 2009. Re-examining automatic keyphrase extraction approaches in scien­tific articles. In <i>Proceedings of the Workshop on Multiword Expressions: Identification, Interpreta­tion, Disambiguation and Applications, </i>pages 9-16, Singapore, August. Association for Computationai Linguistics.</p><p>Niraj Kumar and Kannan Srinathan. 2008. Automatic keyphrase extraction from scientific documents us­ing n-gram fiitration technique, fn <i>DocEng '08: Proceeding of the eighth ACM symposium on Doc­ument engineering, </i>pages f99-208, New York, NY, USA. ACM.</p><p>Thuy Dung Nguyen and Min yen Kan. 2007. Keyphrase extraction in scientific pubfications. fn <i>In Proc.</i><i> of International Conference on Asian Digi­tal Libraries (ICADL 07, </i>pages 317-326. Springer.</p><p>Peter Turney. 2000. Learning aigorithms for keyphrase extraction. <i>Information Retrieval, </i>2:303-336.</p><p>Peter Turney. 2003. Coherent keyphrase extraction via web mining, fn <i>In Proceedings ofUCAI, </i>pages 434-439.</p><p>Jia-Long Wu and Aiice M. Agogino. 2004. Au­tomating keyphrase extraction with mufti-objective genetic aigorithms. fn <i>HICSS '04: Proceedings of the Proceedings of the 37th Annual Hawaii Interna­tional Conference on System Sciences (HICSS'04) -Track 4, </i>page 40104.3, Washington, DC, USA. IEEE Computer Society.</p></references></body></article>