Keyphrases provide users overwhelmed by the richness of information currently available with useful insight into document content but at the same time they are a valuable input for a variety of NLP applications such as summarization, cluster­ing and searching. The SemEval 2010 competition included a task targeting the Automatic Keyphrase Extraction from Scientific Articles (Kim et al., 2010). Given a set of scientific articles partic­ipants are required to assign to each document keyphrases extracted from text.

We participated in this task with an unsuper­vised approach for keyphrase extraction that does not only consider a general description of a term to select candidates but also takes into consider­ation context information. The larger context of our work is the extraction of expertise topics for Expertise Mining (Bordea, 2010).

Expertise Mining is the task of automatically extracting expertise topics and expertise profiles from a collection of documents. Even though the Expertise Mining task and the Keyphrase Extrac­tion task are essentially different, it is important to assess the keyphraseness of extracted expertise topics, i.e., their ability to represent the content of a document. Here we will report only relevant findings for the Keyphrase Extraction task, focus­ing on the overlapping aspects of the two afore­mentioned tasks.

After giving an overview of related work in sec­tion 2 we introduce skill types and present our can­didate selection method in section 3. Section 4 de­scribes the features used for ranking and filtering the candidate keyphrases and Section 5 presents our results before we conclude in Section 6.

The current methods for keyphrase extraction can be categorized in supervised and unsupervised ap­proaches. Typically any keyphrase extraction sys­tem works in two stages. In the first stage a gen­eral set of candidates is selected by extracting the tokens of a text. In the second stage unsupervised approaches combine a set of features in a rank to select the most important keyphrases and super­vised approaches use a training corpus to learn a keyphrase extraction model.

Mihalcea and Tarau (2004) propose an unsuper­vised approach that considers single tokens as ver­tices of a graph and co-occurrence relations be­tween tokens as edges. Candidates are ranked us­ing PageRank and adjacent keywords are merged into keyphrases in a post-processing step. The frequency of noun phrase heads is exploited by Barker and Cornacchia (2000), using noun phrases as candidates and ranking them based on term fre­quency and term length.

Kea is a supervised system that uses all n-grams of a certain length, a Naive Bayes classifier and tf-idf and position features (Frank et al., 1999). Turney (2000) introduces Extractor, a supervised system that selects stems and stemmed n-grams as candidates and tunes its parameters (mainly re­lated to frequency, position, length) with a ge­netic algorithm. Hulth (2004) experiments with three types of candidate terms (i.e., n-grams, noun phrase chunks and part-of-speech tagged words that match a set of patterns) and constructs classi­fiers by rule induction using features such as term frequency, collection frequency, relative position and PoS tags.

The candidate selection method is the main dif­ference between our approach and previous work. We did not use only a general description of a term to select candidates, but we also took into consid­eration context information.

Skill types are important domain words that are general enough to be used in different subfields and that reflect theoretical or practical expertise. Consider for instance the following extracts from scientific articles:

...analysis of historical trends... ...duplicate photo detection algorithm ... ...approach for data assimilation... ...methodology for reservoir characterization...

In all four examples the expertise topic (e.g., "historical trends", "duplicate photo detection al­gorithm", "data assimilation", "reservoir charac­terization") is introduced by a skill type (e.g., "analysis", "algorithm", "approach", "methodol­ogy"). Some of these skill types are valid for any scientific area (e.g. "approach", "method", "analysis", "solution") but we can also identify domain specific skill types, e.g., for computer science "implementation", "algorithm", "develop­ment", "framework", for physics "proof", "prin­ciples", "explanation" and for chemistry "law", "composition", "mechanism", "reaction", "struc­ture".

Our system is based on the GATE natural lan­guage processing framework (Cunningham et al., 2002) and it uses the ANNIE IE system included in the standard GATE distribution for text tok-enization, sentence splitting and part-of-speech tagging. The GATE processing pipeline is de­picted in Figure 1, where the light grey boxes em­body components available as part of the GATE framework whereas the dark grey boxes represent components implemented as part of our system.

We manually extract a set of 81 single word skill types for the Computer Science field by analysing word frequencies for topics from the ACM classi­fication system. The skill types that appear most

!ACM classification system: http://www.acm. org/about/class/

GATE Pipeline Document Reset

ANNIE English Tokenizer

ANNIE Sentence Splitter

ANNIE POS Tagger

Skill Types Gazetteer

Topic Extraction JAPE Transducer

Figure 1 : GATE Processing Pipeline frequently in keyphrases given in the training set are "system", "model" and "information". The Skill Types Gazetteer adds annotations for skill types and then the IAPE Transducer uses regular expressions to annotate candidates.

We rely on a syntactic description of a term to discover candidate keyphrases that appear in the right context of a skill type or that include a skill type. The syntactic pattern for a term is defined by a sequence of part-of-speech tags, mainly a noun phrase. We consider that a noun phrase is a head noun accompanied by a set of modifiers (i.e nouns, adjectives) that includes proper nouns, car­dinal numbers (e.g., "P2P systems") and gerunds (e.g., "ontology mapping", "data mining"). Terms that contain the preposition "of" (e.g., "quality of service") or the conjunction "and" (e.g., "search and rescue") were also allowed.

For the ranking stage we use several features al­ready proposed in the literature such as length of a keyphrase, tf-idf and position. We also take into consideration the collection frequency in the con­text of a skill type.

Ranking. Longer candidates in terms of number of words are ranked higher, because they are more descriptive. Keyphrases that appear more frequently with a skill type in the collection of documents are also ranked higher. Therefore we define the rank for a topic as:

The SemEval task organizers provided two sets of scientific articles, a set of 144 documents for training and a set of 100 documents for test­ing. No information was provided about the sci­entific domain of the articles but at least some of them are from Computer Science. The av­erage length of the articles is between 6 and 8 pages including tables and pictures. Three sets of answers were provided: author-assigned keyphrases, reader-assigned keyphrases and com­bined keyphrases (combination of the first two sets). The participants were asked to assign a num­ber of exactly 15 keyphrases per document.

All reader-assigned keyphrases are extracted from the papers, whereas some of the author-assigned keyphrases do not occur explicitly in the text. Two alternations of keyphrase are accepted: A of B / B A and As B. In case that the seman­tics changes due to the alternation, the alternation is not included in the answer set. The traditional evaluation metric was followed, matching the ex­tracted keyphrases with the keyphrases in the an­swer sets and calculating precision, recall and F-score. In both tables the column labels start with a number which stands for the top 5, 10 or 15 candi­dates. The characters P, R, F mean micro-averaged precision, recall and F-scores. For baselines, 1, 2, 3 grams were used as candidates and TF-IDF as features.

In Table 1 the keyphrases extracted by our sys­tem are compared with keyphrases extracted by an unsupervised method that ranks the candidates based on TF-IDF scores and two supervised meth­ods using Naive Bayes (NB) and maximum en-tropy(ME) in WEKA. Our performance is well above the baseline in all cases.

To show the contribution of skill types we in­cluded the results for a baseline version of our system (DUB) that does not rank the candidates using the normalized collection frequency in the context of a skill type Frii but the overall collec­tion frequency (i.e., the number of occurrences of a keyphrase in the corpus). The significantly in­creased results compared to our baseline version show the effectiveness of skill types for keyphrase candidate ranking.

Table 2 presents our results in comparison with results of other participants. Even though our sys­tem considers in the first stage a significantly lim­ited set of candidates the results are very close to the average results of other participants. Our sys­tem performed 8th best out of 19 participants for top 15 keyphrases, 10th best for top 10 keyphrases and 13th best for top 5 keyphrases, which indicates that our approach could be improved by using a more sophisticated ranking method.

In this paper we have reported the performance of an unsupervised approach for keyphrase extrac­tion that does not only consider a general descrip­tion of a term to select keyphrase candidates but also takes into consideration context information. The method proposed here uses term extraction techniques (the syntactic description of a term), classical keyword extraction techniques(TF-IDF, length, position) and contextual evidence (skill types).

We argued that so called "skill types" (e.g., "methods", "approach", "analysis") are a useful instrument for selecting keyphrases from a doc­ument. Another novel aspect of this approach is using the collection of documents available on the Web (i.e., number of hits for a keyphrase) instead of a reference corpus. It would be interesting to evaluate the individual contributions of skill types for Keyphrase Extraction by adding them as a fea­ture in a classical system like KEA.

Future work will include an algorithm for auto­matic extraction of skill types for a domain and an analysis of the performance of each skill type.

This work is supported by Science Foundation Ire­land under Grant No. SFI/08/CE/I1380 (Lion-2).