Keyphrases summarize the content of documents with the most important phrases. They can be valuable in many application areas, ranging from information retrieval to topic detection. However, since manually assigned keyphrases are rarely pro­vided and creating them by hand would be costly and time-consuming, their automatic generation is of great interest nowadays. Recent state-of-the-art systems treat this kind of task as a super­vised learning task, in which phrases of a docu­ment should be classified with respect to their key phrase characteristics based on manually labeled corpora and various feature values.

This paper focuses on the task of keyphrase ex­traction from scientific papers and we shall intro­duce new features that can significantly improve the overall performance. Although the experimen­tal results presented here are solely based on sci­entific articles, due to the robustness and univer­sality of the features, our approach is expected to achieve good results when applied on other do­mains as well.

In keyphrase extraction tasks, phrases are ex­tracted from one document that are the most char­acteristic of its content (Liu et al., 2009; Wit­ten et al., 1999). In these approaches keyphrase extraction is treated as a classification task, in which certain n-grams of a specific document act as keyphrase candidates, and the task is to classify them as proper keyphrases or not.

While Frank et al. (1999) exploited domain spe­cific knowledge to improve the quality of auto­matic tagging, others like Liu et al. (2009) analyze term co-occurence graphs. It was Nguyen and Kan (2007) who dealt with the special characteristics of scientific papers and introduced the state-of-the-art feature set to keyphrase extraction tasks. Here we will follow a similar approach and make sig­nificant improvements by the introduction of novel features.

The SZTERGAK framework treats the reproduc­tion of reader-assigned keyphrases as a supervised learning task. In our setting a restricted set of to­ken sequences extracted from the documents was used as classification instances. These instances were ranked regarding to their posteriori proba­bilities of the keyphrase class, estimated by a Naïve Bayes classifier. Finally, we chose the top-15 candidates as keyphrases.

Our features can be grouped into four main cat­egories: those that were calculated solely from the surface characteristics of phrases, those that took into account the document that contained a keyphrase, those that were obtained from the given document set and those that were based on exter­nal sources of information.

The training and test sets of the shared task (Kim et al., 2010) consisted of 144 and 100 scien­tific publications from the ACL repository, respec­tively. Since the primary evaluation of the shared task was based on the top-15 ranked automatic keyphrases compared to the keyphrases assigned by the readers of the articles, these results are re­ported here. The evaluation results can be seen in Table 2 where the individual effect of each feature is given in combination with the standard features.

It is interesting to note the improvement ob­tained by extending standard features with the simple feature of phrase length. This indicates that though the basic features were quite good, they did not take into account the point that reader keyphrases are likely to consist of several words.

Morphological features, such as POS or suffix features were also among the top-performing ones, which seems to show that most of the keyphrases tend to have some common structure. In contrast, the syntactic feature made some decrease in the performance when it was combined just with the standard ones. This can be due to the fact that the input data were quite noisy, i.e. some inconsisten­cies arose in the data during the pdf to text con­version of articles, which made it difficult to parse some sentences correctly.

It was also interesting to see that Wikipedia fea­ture did not improve the result when it was applied to the whole document. However, our previous ex­periences on keyphrase extraction from scientific abstracts showed that this feature can be very use­ful. Hence, we relaxed the feature to handle occur­rences just from the abstract. This modification of the feature yielded a 14.8% improvement in the F-measure. A possible explanation for this is that Wikipedia has articles of very common phrases (such as Calculation or Result) and the dis­tribution of such non-keyphrase terms is higher in the body of the articles than in abstracts.

The last row of Table 2 contains the result achieved by the complete feature set excluding keyphraseness. As keyphraseness exploits author-assigned keyphrases and - to the best of our knowledge - other participants of the shared task did not utilize author-assigned keyphrases, this re­sult is present in the final ranking of the shared task systems. However, we believe that if the task is to extract keyphrases from an article to gain se­mantic meta-data for an NLP application (e.g. for information retrieval or summarization), author-assigned keyphrases are often present and can be very useful. This latter statement was proved by one of our experiments where we used the au­thor keyphrases assigned to the document itself as a binary feature (instead of using the pool of all keyphrases). This feature set could achieve an F-score of 27.44 on the evaluation set and we believe that this should be the complete feature set in a real-world semantic indexing application.

In this paper we introduced a wide set of new fea­tures that are able to enhance the overall perfor­mance of supervised keyphrase extraction applica­tions. Our features include those calculated simply on surface forms of keyphrase aspirants, those that make use of the document- and corpus-level envi­ronment of phrases and those that rely on exter­nal knowledge. Although features were designed to the specific task of extracting keyphrases from scientific papers, due to their generality it is highly assumable that they can be successfully utilized on different domains as well.

The features we selected in SZTERGAK per­formed well enough to actually achieve the third place on the shared task by excluding the keyphraseness feature and would be the first by using any author-assigned keyphrase-based fea­ture. It is also worth emphasizing that we think that there are many possibilities to further extend the feature set (e.g. with features that take the semantic relatedness among keyphrase aspirants into account) and significant improvement could be achievable.

Acknowledgement

The authors would like to thank the annotators of the shared task for the datasets used in the shared task. This work was supported in part by the NKTH grant (project codename TEXTREND).