Keyphrase extraction is a natural language pro­cessing task for collecting the main topics of a document into a list of phrases. Keyphrases are supposed to be available in the processed docu­ments themselves, and the aim is to extract these most meaningful words and phrases from the doc­uments. Keyphrase extraction summarises the content of a document as few phrases and thus provides a quick way to find out what the docu­ment is about. Keyphrase extraction is a basic text mining procedure that can be used as a ground for other, more sophisticated text analysis meth­ods. Automatically extracted keyphrases may be used to improve the performance of information retrieval, automatic user model generation, docu­ment collection clustering and visualisation, sum­marisation and question-answering, among others.

This article describes the participation of the Likey method in the Task 5 of the SemEval 2010 challenge, automatic keyphrase extraction from scientific articles (Kim et al., 2010).

Likey keyphrase extraction approach comes from the tradition of statistical machine learn­ing (Paukkeri et al, 2008). The method has been developed to be as language-independent as possible. The only language-specific component needed is a corpus in each language. This kind of data is readily available online or from other sources.

Likey selects the words and phrases that best crystallize the meaning of the documents by com­paring ranks of frequencies in the documents to those in the reference corpus. The Likey ra­tio (Paukkeri et al, 2008) for each phrase is de­fined as rankd(p) where rankd{p) is the rank value of phrase p in document d and rankr{p) is the rank value of phrase p in the reference corpus. The rank val­ues are calculated according to the frequencies of phrases of the same length n. If the phrase p does not exist in the reference corpus, the value of the maximum rank for phrases of length n is used: rankr{p) = max-rankr{n) + 1. The Likey ra­tio orders the phrases in a document in such a way that the phrases that have the smallest ratio are the best candidates for being a keyphrase.

As a post-processing step, the phrases of length n > 1 face an extra removal process: if one of the words composing the phrase has a rank of less than a threshold £ in the reference corpus, the phrase is removed from the keyphrase list. This procedure excludes phrases that contain function words such as "of or "the". As another post­processing step, phrases that are subphrases of those that have occurred earlier on the keyphrase list are removed, excluding e.g. "language model" if "unigram language model" has been already ac­cepted as a keyphrase.

We participated the challenge with Likey results of three different parameter settings. The settings are given in Table 3. Likey-1 has phrases up to 3 words and Likey-2 and Likey-3 up to 4 words. The thresh­old value for postprocessing was selected against the trial set, with £ = 100 performing best. It is used for Likey-1 and Likey-2. Also a bit larger threshold £ = 130 was tried for Likey-3 to exclude more function words.

Table 3 : Different parametrizations for Likey: n-gram length and threshold value £.

An example of the resulting keyphrases ex­tracted by Likey-1 from the first scientific arti­cle in the test set (article C-l) is given in Ta­ble 4. Also the corresponding "correct" answers in reader-assigned and author-provided answer sets are shown. The keyphrases are given in stemmed versions. Likey keyphrases that can be found in the reader or author answer sets are emphasized.

Likey-1 uddi registri, proxi registri, servie discoveri, grid servie discoveri, uddi kei, uniqu uddi kei, servie discoveri mechan, distribut hash tabl, web servie, dht, servie name, web servie discoveri, local proxi registri, local uddi registri, queri multipl registri Reader grid servie discoveri, uddi, distribut web-servic discoveri architectur, dht base uddi registri hierarchi, deploy issu, bamboo dht code, case-insensit search, queri, longest avail prefix, qo-base servie discoveri, autonom control, uddi registri, scalabl issu, soft state Author uddi, dht, web servie, grid comput, md, discoveri

Table 4: Extracted keyphrases by Likey-1 from ar­ticle C-l and the corresponding correct answers in reader and author answer sets.

The example shows clearly that many of the ex­tracted keyphrases contain the same words that can be found in the correct answer sets but the length of the phrases vary and thus they cannot be counted as successfully extracted keyphrases.

The results for the three different Likey parametrizations and the three baselines are given in Table 5 for reader-assigned keyphrases and Ta­ble 6 for the combined set of reader and author-assigned keyphrases. The evaluation is conducted by calculating precision (P), recall (R) and F-measure (F) for top 5,10, and 15 keyphrase candi­dates for each method, using the reader-assigned and author-provided lists as correct answers. The baseline methods are unsupervised tf-idf and su­pervised Naïve Bayes (NB) and Maximum Entropy (ME).

Likey-1 performed best in the competition and is thus selected as the official result of Likey in the task. Anyway, all Likey parametrizations outper­form the baselines, Likey-1 having the best pre­cision 24.60% for top-5 candidates in the reader data set and 29.20% for top-5 candidates in the combined data set. The best F-measure is obtained with Likey-1 for top-10 candidates for both reader and combined data set: 16.24% and 17.11%, respectively. Likey seems to produce the best keyphrases in the beginning of the keyphrase list: for reader-assigned keyphrases the top 5 keyphrase precision for Likey-1 is 6.8 points better than the best-performing baseline tf-idf and the cor­responding F-measure is 4.0 points better. For the combined set, the numbers are 7.2 and 3.7 points, respectively. The difference decreases for the larger keyphrase sets.

This article describes our submission to SemEval 2010 Task 5, keyphrase extraction from scien­tific articles. Our unsupervised and language-independent method Likey uses reference corpus and is able to outperform both the unsupervised and supervised baseline methods. The best results are obtained with the top-5 keyphrases: precision of 24.60% with reader-assigned keyphrases and 29.20% with the combination of reader-assigned and author-provided keyphrases.

There are some keyphrases in the answer sets that our method does not find: due to the com­paratively large threshold value £ many phrases that contain function words, e.g. "of, cannot be found. We also extract keyphrases of maximum length of three or four words and thus cannot find keyphrases longer than that. The next step of this research would be to take these problems into ac­count.

This work was supported by the Finnish Graduate School in Language Studies (Langnet) funded by Ministry of Education of Finland.