Keyphrases are expressions, either single words or phrases, describing the most important con­cepts of a document. As such, a list of key-phrases provides an approximate but useful char­acterization of the content of a text and can be used in a number of interesting ways both for human and automatic processing. For example, keyphrases provide a sort of quick summary of a document. This can be exploited not only in automatic summarization tasks, but also to en­able quick topic search over a number of docu­ments indexed according to their keywords, which is more precise and efficient than full-text search. Once the keywords of a document collec­tion are known, they can also be used to calculate semantic similarity between documents and to cluster the texts according to such similarity (Ricca et al, 2004). Also, keyword extraction can be used as an intermediate step for automatic sense extraction (Jones et al, 2002).

For these reasons, the keyphrase extraction task proposed at SemEval 2010 raised much at­tention among NLP researchers, with 20 groups participating to the competition. In this frame­work, we presented the KX system, specifically tuned to identify keyphrases in scientific articles. In particular, the challenge comprised two sub-tasks: the extraction of reader-assigned and of author-assigned keyphrases in scientific articles from the ACM digital library. The former are assigned to the articles by annotators, who can choose only keyphrases that occur in the docu­ment, while author-assigned keyphrases are not necessarily included in the text.

A previous version of the KX system, named KXPat (Pianta, 2009), was developed to extract keyphrases from patent documents in the PatEx­pert project (www.patexpert.org). The sys­tem employed in the SemEval task has additional parameters and has been tailored to identify key-phrases in scientific articles.

With KX, the identification of keyphrases can be accomplished with or without the help of a reference corpus, from which some statistical measures are computed in an unsupervised way. We present here the general KX architecture, including the corpus-based pre-processing, even if in the SemEval task the information extracted from the corpus did not contribute as expected (see Section 3).

KX keyphrase extraction combines linguistic and statistical information, similar to (Frantzi et al., 2000) and is based on 4 steps. The first three steps are carried out at corpus level, whereas the fourth one extracts information specific to each single document to be processed. This means that the first three steps require a corpus C, preferably sharing the same domain of the document d from which the keyphrases should be extracted. The fourth step, instead, is focused only on the document d. The steps can be summarized as follows:

Step 1: Extract from C the list NG-c of corpus n-grams, where an n-gram is any sequence of tokens in the text, for instance "the sys­tem", "of the", "specifically built". Step 2: Select from the list NG-c a sub-list of multiword terms MW-c, that is combina­tions of words expressing a unitary con­cept, for instance "light beam" or "access control"

Step 3: For each document in C, recognize and mark the multiword terms. Calculate the inverse document frequency (IDF) for all words and multiword terms in the cor­pus.

Step 4: Given a document d from which a set of relevant keyphrases should be ex­tracted, count all words and multiword terms and rank them.

Step 1 is aimed at building a list of all possible n-grams in C. The maximum length of the selected n-grams can be set by the user. For SemEval, beside one-token n-grams, we select 2-, 3- and 4-grams. Since n-grams occurring only a few times are very unlikely to be useful for keyphrase recognition, they are cut off from the extracted list and excluded for further processing. The fre­quency threshold can be set according to the ref­erence corpus dimensions. For SemEval, we fixed the frequency threshold to 4. In this step, a black-list was also used in order to exclude n-grams containing any of the stopwords in the list. Such stopwords include for example "every­thing", "exemplary", "preceding", etc.

In Step 2, we select as multiword terms those n-grams that match certain lexical patterns. To this purpose, we first analyze all n-grams with the MorphoPro morphological analyzer of the TextPro toolsuite (Pianta et al., 2006). Then, we filter out the n-grams whose analysis does not correspond to a predefined set of lexical patterns. For example, one of the patterns admitted for 4-grams is the following: [N]~[O]~[ASPGLU]~[NU]. This means that a 4-gram is a candidate multiword term if it is composed by a Noun, fol­lowed by "of" or "for" (defined as O), followed by either an Adjective, Singular noun, Past parti­ciple, Gerund, punctuation (L) or Unknown word, followed by either a Noun or Unknown word. This is matched for example by the 4-gram "subset [S] of [O] parent [S] peers [N]".

Both the lexical categories (e.g. S for singular noun) and the admissible lexical patterns can be defined by the user.

In Step 3, multiword terms are recognized by combining local (document) and global (corpus) evidence. To this purpose, we do not exploit as­sociation measures such as Log-Likelihood, or Mutual Information, but a simple frequency based criterion. Two thresholds are defined: MinCorpus, which corresponds to the minimum number of occurrences of an n-gram in a refer­ence corpus, and MinDoc, i.e. the minimum number of occurrences in the current document. KX marks an n-gram in a document as a multiword term if it occurs at least MinCorpus times in the corpus or at least MinDoc times in the document. The two parameters depend on the size of the corpus and the document respectively. In SemEval, we found that the best thresholds are MinDoc=4 and MinCorpus=8. A similar, fre­quency-based, strategy is used to solve ambigui­ties in how sequences of contiguous multiwords should be segmented. For instance, given the sequence "combined storage capability of sen­sors" we need to decide whether we recognize "combined storage capability" or "storage capa­bility of sensors". To this purpose, we calculate the strength of each alternative collocation as docFrequency * corpusFrequency, and then choose the stronger one. To calculate IDF for each word and multiword term, we use the usual formula: log( TotDocs / DocsContaningTerm ).

In step 4, we take into account a new docu­ment d, possibly not included in C, from which the keyphrases should be extracted. First we rec­ognize and mark multiword terms, through the same algorithm used in Step 3. Note that KX can recognize multiwords also in isolated documents, independently of any reference corpus, by acti­vating only the MinDoc parameter (see above). Then, we count the frequency of words and multiword terms in d, obtaining a first list of keyphrases, ranked according to frequency. Thus, frequency is the baseline ranking parame­ter, based on the assumption that important con­cepts are mentioned more frequently than less important ones.

After the creation of a frequency-based list of keyphrases, various techniques are used to re­rank it according to relevance. In order to find the best ranking mechanism for the type of key-phrases we want to extract, different parameters can be set:

• Inverse document frequency (IDF): this parameter takes into account the fact that a

In the SemEval task, 144 training files were made available before the test data release. We split them into a training/development set of 100 documents and a test set of 44 documents, in or­der to find the best parameter combination. Key­phrase assignment is a subjective task and crite­ria for keyphrase identification depend on the domain and on the goal for which the keyphrases are needed. For example in scientific articles longer keyphrases are often more informative than shorter ones, so the parameters for boosting longer concepts are particularly relevant.

We first tested all parameters in isolation to compute the improvement over the frequency-based baseline. Results are reported in Table 1. F1 is computed as the harmonic mean of preci­sion and recall over the 15 top-ranked key-phrases after stemming. We report the combined F1, as computed by the task scorer in order to combine reader-assigned and author-assigned keyword sets.

In the SemEval task, the system was run on the test set (100 articles) with the best performing parameter combination described in the previous section. The results obtained over the 15 top-ranked keyphrases are reported in Table 2.

In the competition, the F1 score over reader-assigned keyphrases was ranked 3rd out of 20 participants, while the combined measure achieved the 7th best result out of 20.

In this work we have described KX, a flexible system for keyphrase extraction, which achieved promising results in the SemEval task 5. The good KX performance is due to its adaptable ar­chitecture, based on a set of parameters that can be tailored to the document type, the preferred keyphrase length, etc. The system can also ex­ploit multiword lists (with frequency) extracted from a reference corpus, even if this feature did not improve KX performance in this specific task. However, this proved to be relevant when applied to keyphrase extraction in the patent do­main, using a large domain-specific corpus of 10.000 very long documents (Pianta, 2009).

A limitation of KX in the task was that it ex­tracts only keyphrases already present in a given document, while the author-assigned subtask in the SemEval competition included also key-phrases that do not occur in the text. Another improvement, which is now being implemented, is the extraction of the best parameter combina­tion using machine-learning techniques.