Keyphrases capture the main topic of the docu­ment in which they appear and can be useful for making the content of a document quickly ac­cessible to a potential reader. They can be pre­sented to the reader directly, in order to provide a short overview of the document, but can also be processed further, e.g. for text summarization, document clustering, question-answering or rela­tion extraction. The task of extracting keyphrases automatically can be performed by generating a list of keyphrase candidates, ranking these can­didates, and selecting the top-ranked candidates as keyphrases. In the KeyWE system, candidates are generated based on an adapted nominal group chunker described in section 3 and ranked using the SVMmrafc algorithm (loachims, 2006), as de­scribed in section 4. The used features are spec­ified in section 5. In section 6, we present the results achieved on the test data provided for the SemEval 2010 Shared Task on Keyphrase Extrac-

Günter Neumann

DFKI - Language Technology Saarbrücken, Germany tion by selecting as keyphrases the top 5, 10, and 15 top-ranked candidates, respectively.

The task of keyphrase extraction came up in the 1990s and was first treated as a supervised learn­ing problem in the GenEx system (Turney, 1999). Since then, the task has evolved and various new approaches have been proposed. The task is usu­ally performed in two steps: 1. candidate ex­traction (or generation) and 2. keyphrase selec­tion. The most common approach towards can­didate extraction is to generate all n-grams up to a particular length and filter them using stopword lists. Lately, more sophisticated candidate extrac­tion methods, usually based on additional linguis­tic information (e.g. POS tags), have been pro­posed and shown to produce better results (e.g. Hulth (2004)). Liu et al. (2009) restrict their can­didate list to verb, noun and adjective words. Kim and Kan (2009) generate regular expression rules to extract simplex nouns and nominal phrases. As the majority of technical terms is in nominal group positions, we assume that the same holds true for keyphrases and apply an adapted nominal group chunker to extract keyphrase candidates. Lhe selection process is usually based on some supervised learning algorithm, e.g. Naive Bayes (Frank et al., 1999), genetic algorithms (Lurney, 1999), neural networks (Wang et al., 2005) or de­cision trees (Medelyan et al., 2009). Unsuper­vised approaches have also been proposed, e.g. by Mihalcea and Tarau (2004) and Liu et al. (2009). However, as for the shared task, annotated train­ing data was available, we opted for an approach based on supervised learning.

Rather than extracting candidates from the full text of the article, we restrict our search for candidates to the first 2000 characters starting with the ab­stract. We also extract title and general terms for use in the feature construction process. From the reduced input text, we extract keyphrase candi­dates based on the output of a nominal group chun­ker.

This approach is inspired by findings from cog­nitive linguistics. Talmy (2000) divides the con­cepts expressed in language into two subsystems: the grammatical subsystem and the lexical sub­system. Concepts associated with the grammati­cal subsystem provide a structuring function and are expressed using so-called closed-class forms (function words, such as conjunctions, determin­ers, pronouns, and prepositions, but also suf­fixes such as plural markers and tense markers). Closed-class elements (CCEs) provide a scaffold­ing, across which concepts associated with the lex­ical subsystem (i.e. nouns, verbs, adjectives and adverbs) can be draped (Evans and Pourcel, 2009). Spurk (2006) developed a nominal group (NG) chunker that makes use of this grammatical sub­system. Using a finite list of CCEs and learned word class models for identifying verbs and ad­verbs, a small set of linguistically motivated ex­traction patterns is stated to extract NGs. The rules are based on the following four types of occur­rences of NGs in English: 1. at the sentence be­ginning, 2. within a determiner phrase, 3. follow­ing a preposition and 4. following a verb. Not being trained on a particular corpus, the chunker works in a domain-independent way. In addition, it scales well to large amounts of textual data. In order to use the chunker for keyphrase extrac­tion, we manually analysed annotated keyphrases in scientific texts, and, based on the outcome of the evaluation, made some adaptations to the chun­ker, which take care of the fact that the boundaries of a keyphrase do not always coincide with the boundaries of a NG. In particular, we remove de­terminers, split NGs on conjunctions, and process text within parentheses separately from the main text. An evaluation on the provided training data showed that the adapted chunker extracts 80% of the reader-annotated keyphrases found in the text.

The problem of ranking keyphrase candidates can be formalized as follows: For a document d and a collection of n keyword candidates C = c\...cn, the goal is to compute a ranking r that orders the candidates in C according to their degree of keyphraseness in d.

The problem can be transformed into an ordinal regression problem. In ordinal regression, the la­bel assigned to an example indicates a rank (rather than a nominal class, as in classification prob­lems). The ranking algorithm we use is SVMrank, developed by loachims (2006). This algorithm learns a linear ranking function and has shown to outperform classification algorithms in keyphrase extraction (liang et al., 2009). The target (i.e. rank) value defines the order of the examples (i.e. keyphrase candidates). Dur­ing training, the target values are used to gener­ate pairwise preference constraints. A preference constraint is included for all pairs of examples in the training file, for which the target value differs. Two examples are considered for a pairwise pref­erence constraint only if they appear within the same document.

The model that is learned from the training data is then used to make predictions on the test ex­amples. For each line in the test data, the model predicts a ranking score, from which the ranking of the test examples can be recovered via sorting. For ranking the candidates, they are transformed into vectors based on the features described in sec­tion 5.

During training, the set of candidates is made up of the annotated reader and author keywords as well as all NG chunks extracted from the text. These candidates are mapped to three different ranking values: All annotated keywords are given a ranking value of 2; all extracted NG chunks that were annotated somewhere else in the train­ing data are given a ranking value of 1 ; all other NG chunks are assigned a ranking value of 0. Giving a special ranking value to chunks an­notated somewhere else in the corpus is a way of exploiting domain-specific information about keyphrases. Even though not annotated in this par­ticular document, a candidate that has been anno­tated in some other document of the domain, is more likely to be a keyphrase than a candidate that has never been annotated before (cf. Frank et al. (1999)).

We used two types of features: term-specific features and document-specific features. Term-specific features cover properties of the candidate term itself (e.g. term length). Document-specific features relate properties of the candidate to the text, in which it appears (e.g. frequency of the term in the document). Our term-specific features concern the following properties:

• Term length refers to the length of a can­didate in number of tokens. We express this property in terms of five boolean fea­tures: hasltoken, has2tokens, has3tokens, has4tokens, has5orMoreTokens. The advan­tage over expressing term length as a nu­meric value is that using binary features, we allow the algorithm to learn that candidates of medium lengths are more likely to be keyphrases than very short or very long can­didates.

• The MSN score of a candidate refers to the number of results retrieved when querying the candidate string using the MSN search engine. The usefulness of MSN scores for technical term extraction has been shown by Eichler et al. (2009). We normalize the MSN scores based on the number of digits of the score and store the normalized value in the feature normalizedMsn. We also use a binary feature isZeroMsn expressing whether query­ing the candidate returns no results at all.

• Special characters can indicate whether a candidate is (un)likely to be a keyphrase. We use two features concerning special charac­ters: containsDigit and containsHyphen.

• Wikipedia has shown to be a valuable source for extracting keywords (Medelyan et al., 2009). We use a feature isWikipediaTerm, expressing whether the term candidate corre­sponds to an entry in Wikipedia.

In addition, we use the following document-specific features:

• TFIDF, a commonly used feature introduced by Salton and McGill (1983), relates the fre­quency of a candidate in a document to its frequency in other documents of the corpus.

• Term position relates the position of the first appearance of the candidate in the document to the length of the document. In addition, our feature appearsInTitle covers the fact that candidates appearing in the document title are very likely to be keyphrases.

• Average token count measures the average occurrence of the individual (lemmatized) to­kens of the term in the document. Our assumption is that candidates with a high average token count are more likely to be keyphrases.

• Point-wise mutual information (PMI, Church and Hanks (1989)) is used to capture the semantic relatedness of the candidate to the topic of the document. A similar feature is introduced by Turney (2003), who, in a first pass, ranks the candidates based on a base feature set, and then reranks them by calculating the statistical association between the given candidate and the top K candidates from the first pass. To avoid the two-pass method, rather than calculating inter-candidate association, we calculate the association of each candidate to the terms specified in the General Terms section of the paper. Like Turney, we calculate PMI based on web search results (in our case, using MSN). The feature maxPmi captures the maximum PMI score achieved with the lemmatized candidate and any of the general terms.

Table 1 presents the results achieved by applying the KeyWE system on the data set of scientific articles provided by the organizers of the shared task along with two sets of manually assigned keyphrases for each article (reader-assigned and author-assigned keyphrases). Our model was trained on the trial and training data (144 articles) and evaluated on the test data set (100 articles). The evaluation is based on stemmed keyphrases, where stemming is performed using the Porter stemmer (Porter, 1980).

Since SVMrank learns a linear function, one can analyze the individual features by studying the learned weights. Roughly speaking, a high pos­itive (negative) weight indicates that candidates with this feature should be higher (lower) in the

Table 1: Results on the two keyword sets: reader (reader-assigned keyphrases) and combined (reader- and author-assigned keyphrases) ranking. In our learned model, the four most im­portant features (i.e. those with the highest ab­solute weight) were containsDigit (-1.17), isZe-roMsn (-1.12), normalizedMsn (-1.00), and avgTo-kenCount (+0.97). This result confirms that web frequencies can be used as a valuable source for ranking keyphrases. It also validates our assump­tion that a high average token count indicates a good keyphrase candidate. The maxPMI feature turned out to be of minor importance (-0.16). This may be due to the fact that we used the terms from the General Terms section of the paper to calculate the association scores, which may be too general for this purpose.

We thank Angela Schneider for her adaptations to the chunker and helpful evaluations. The research project DiLiA is co-funded by the European Re­gional Development Fund (ERDF) in the context of Investitionsbank Berlins ProFIT program under grant number 10140159. We gratefully acknowl­edge this support.