Keyphrases are noun phrases (NPs) that capture the primary topics of a document. While benefi­cial for applications such as summarization, clus­tering and indexing, only a minority of documents have manually-assigned keyphrases, as it is a time-consuming process. Automatic keyphrase genera­tion is thus a focus for many researchers.

Most existing keyphrase extraction systems view this task as a supervised classification task in two stages: generating a list of candidates - can­didate identification; and using answer keyphrases to distinguish true keyphrases - candidate selec­tion. The selection model uses a set of features that capture the saliency of a phrase as a keyphrase. A major challenge of the keyphrase extraction task lies in the candidate identification process. A narrow candidate list will overlook some true keyphrases (favoring precision), whereas a broad list will produce more errors and require more pro­cessing in latter selection stage (favoring recall).

In our previous system (Nguyen and Kan, 2007), we made use of the document logical struc­ture in the proposed features. The premise of this earlier work was that keyphrases are distributed non-uniformly in different logical sections of a pa­per, favoring sections such as introduction, and related work. We introduced features indicating which sections a candidate occurrs in. For our fielded system in this task (Kim et al., 2010), we further leverage the document logical structure for both candidate identification and selection stages.

Our contributions are as follows: 1) We suggest the use of Google Scholar-based crawler to auto­matically find PDF files to enhance logical struc­ture extraction; 2) We provide a keyphrase distri­bution study with respect to different logical struc­tures; and 3) From the study result, we propose a candidate identification approach that uses logical structures to effectively limit the number of candi­dates considered while ensuring good coverage.

Although we have plain text for all test input, we posit that logical structure recovery is much more robust given the original richly-formatted docu­ment {e.g., PDF), as font and formatting informa­tion can be used for detection. As a bridge be­tween plain text data provided by the organizer and PDF input required to extract formatting fea­tures, we first describe our Google Scholar-based crawler to find PDFs given plain texts. We then detail on the logical structure extraction process.

Google Scholar-based Paper Crawler

Our crawler takes inputs as titles to query Google Scholar (GS) by means of web scraping. It processes GS results and performs approximate ti­tle matching using character-based Longest Com­mon Subsequence similarity. Once a matching ti­tle with high similarity score (> 0.7 experimen­tally) is found, the crawler retrieves the list of available PDFs, and starts downloading until one is correctly stored. We enforce that PDFs accepted should have the OCR texts closely match the pro­vided plain texts in terms of lines and tokens.

In the keyphrase task, we approximate the title inputs to our crawler by considering the first two lines of each plain text provided. For 140 train and 100 test input documents, the crawler down­loaded 117 and 80 PDFs, of which 116 and 76 files are correct, respectively. This yields an accept­able level of performance in terms of (Precision, Recall) of (99.15%, 82.86%) for train and (95%, 76%) for test data.

Logical Structure Extraction

Logical structure is defined as "a hierarchy of log­ical components, for example, titles, authors, affil­iations, abstracts, sections, etc." in (Mao et al., 2003). Operationalizing this definition, we em­ploy an in-house software, called SectLabel (Lu-ong et al., to appear), to obtain comprehensive logical structure information for each document. SectLabel classifies each text line in a scholarly document with a semantic class (e.g., title, header, bodyText). Header lines are furthered classified into generic roles {e.g., abstract, intro, method).

A prominent feature of SectLabel is that it is capable of utilizing rich information, such as font format and spatial layout, from an optical char­acter recognition (OCR) output if PDF files are present. In case PDFs are unavailable, SectLa­bel still handles plain text based logical structure discovery, but with degraded performance.

Phrase Distribution Study

We perform a study of keyphrase distribution on the training data over different logical structures (LSs) to understand the importance of each sec­tion within documents. These LSs include: ti­tle, headers, abstract, introduction (intro), related work (rw), conclusion, and body text (body).

We make a key observation that within a para­graph, important phrases occur mostly in the first n sentences. To validate our hypothesis, we con­sider keyphrase distribution over bodyn, which is the subset of all of the body LS, limited to the first n sentences of each paragraph (n = 1,2, 3 experi­mentally).

Table 1 : Keyphrase distribution over different log­ical structures computed from the 144 training documents. The type counts of author-assigned (ath), reader-assigned (rder) and combined (comb) keyphrases are shown. Sent indicates the number of sentences in each LS. The Den column gives the density of keyphrases for each LS.

Results in Table 1 show that individual LSs (title, headers, abstract, intro, rw, concl) con­tain a high concentration (i.e., density > 0.2) of keyphrases, with title and abstract having the highest density, and intro being the most dominant LS in terms of keyphrase count. With all these LSs and body, we obtain the full setting, covering 1994/2059=96.84% of all keyphrases appearing in the original text, fulltext, while effectively reduc­ing the number of processed sentences by more than two-thirds.

Considering only the first sentence of each para­graph in the body text, bodyi, yields fair keyphrase coverage of 1035/1411=73.35% relative to that of fulltext. The number of lines to be processed is much smaller, about a third, which validates our aforementioned hypothesis.

Keyphrase Extraction

Results from the keyphrase distribution study mo­tivates us to further explore the use of logical structures (LS). The idea is to limit the search scope of our candidate identification system while maintaining coverage. We propose a new ap-

caption, footnote, and reference lines.

Following (Nguyen and Kan, 2007), we use the Naive Bayes model implemented in Weka (Hall et al., 2009) for candidate phrase selection. As dif­ferent learning models have been discussed much previous work, we just list the different features with which we experimented with. Our featuresare as follows (where n indicates a numeric fea­ture; b, a boolean one):

F1-F3 (n): TFxIDF, term frequency, term fre­quency of substrings.

F4-F5 (n): First and last occurrences (word off­set).

F6 (n): Length of phrases in words.

F7 (b): Typeface attribute (available when PDF is present) - Indicates if any part of the candidate phrase has appeared in the document with bold or italic format, a good hint for its relevance as a keyphrase.

F8 (b): InTitle - shows whether a phrase is also part of the document title.

F9 (n): TitleOverlap - the number of times a phrase appears in the title of other scholarly documents (obtained from a dump of the DBLP database).

F10-F14 (b): Header, Abstract, Intro, RW, Concl - indicate whether a phrase appears in head­ers, abstract, introduction, related work or conclu­sion sections, respectively.

F15-F19 (n): HeaderF, AbstractF, IntroF, RWF, ConclF - indicate the frequency of a phrase in the headers, abstract, introduction, related work or conclusion sections, respectively.

We have described and evaluated our keyphrase extraction system for the SemEval-2 Task #5. With the use of logical structure in the candidate identification, our system has demonstrated its su­perior performance over systems that do not use such information. Moreover, we have effectively reduced the numbers of text lines and candidate

Table 5: Performance over different abridged in­puts using the best feature set Fi;3;4;6. "@N" indi­cates the number of top N keyphrase matches.

Table 6: Final results on the test data.

phrases to be processed in the candidate identifi­cation and selection respectively by about half.

Our system takes advantage of the logical struc­ture analysis but not to the extent we had hoped. We had hypothesized that formatting features (F7) such as bold and italics, would help discriminate key phrases, but in our limited experiments for this task did not validate this. Similarly, external knowledge should help in the keyphrase task, but the prior knowledge about keyphrase likelihood (Fg) in DBLP hurt performance in our tests. We plan to further explore these issues for the future.