<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="710"/><title>Multiple Evidence for Term Extraction in Broad Domains</title><pubinfo>Proceedings of Recent Advances in Natural Language Processing,pages 710-715, Hissar, Bulgaria, 12-14 September 2011.</pubinfo><author surname="Dobrov" givenname="Boris"><org  name="Moscow State University" city="Moscow Russia"/></author><author surname="Loukachevitch" givenname="Natalia"><org  name="Bath University" country="United Kingdom"/></author></firstpageheader><frontmatter><p><b>Multiple Evidence for Term Extraction in Broad Domains</b></p><p><b>Boris Dobrov</b></p><p>Research Computing Center of Lomonosov Moscow State University, Moscow, Russia</p><p>dobroff@mail.cir.ru</p><p><b>Natalia Loukachevitch</b></p><p>louk nat@mail.cir.ru</p></frontmatter><abstract>The paper describes the method of extraction of two-word domain terms combining their features. The features are computed from three sources: the occurrence statistics in a domain-specific text collection, the statistics of global search engines, and a domain-specific thesaurus. The evaluation of the approach is based on manually created thesauri. We show that the use of multiple features considerably improves the automatic extraction of domain-specific terms. We compare the quality of the proposed method in two different domains. </abstract></header><body><section number="1" title="Introduction"><p>The important stage of domain specific knowledge acquisition is recognition of terms, representing domain concepts in documents. Automatic extraction of domain terms from texts is a subject of constant interest in automatic document processing. The special difficulty is the automatic extraction of multiword terms (Zhang et. al. 2008; Wong et. al. 2008).</p><p>Contemporary information systems usually contain documents related to broad domains, which requires development of large terminological resources. Term extraction to develop such resources should be based on processing of large amount of documents. Besides, existing terminological resources need periodic updates.</p><p>For many years, researchers tried to find the best statistical feature for term extraction. Now machine learning methods allow for the combination of many features (Vivaldi et.al, 2001, Pecina and Schlesinger, 2008, Foo and</p><p>Merkel, 2010).</p><p>In (Vivaldi et. al., 2001) features for extraction of medical terms are combined using boosting algorithm. The features include information from EuroWordNet, Greek and Latin word forms, statistical measures. Some of the features are rather domain-dependent. (Foo and Merkel, 2010) study applicacability of rule-based machine-learning algorithm Ripper for term extraction from patent texts.</p><p>In (Pecina and Schlesinger, 2008) the combi­nation of statistical characteristics of phrases, based on the Czech text collection, is used to ex­tract several types of collocations (such as phras­al verbs, idioms, terms). The authors used over 80 features and obtained 20% improvement compared with the best individual feature. But the authors of this paper indicate that efficiency of different features is very variable and depends on a collection, types of expressions and so on.</p><p>In this paper we describe an experiment to extract two-word terms (noun groups) based on a combination of three types of features: features based on a domain-specific text collection, features obtained from an Internet search engine, features obtained from a domain-specific thesau­rus.</p><p>Working with a thesaurus, we simulate the situation when the thesaurus partially exists. We want to study its potential to recognise new terms. The important point of our research is to study the stability of the term extraction model among different domains.</p></section><section number="2" title="Description of Experiment: Data and Evaluation"><p>We conduct our study in two domains. The first domain is the very broad domain of natural sciences and technologies. The second one is domain of banking and bank regulation. For both domains we have Russian thesauri, developed manually, which we use as a basis for evaluation of term extraction methods (see section 2.1).</p><page local="2" global="711"/><p>Besides, there are Russian domain-specific text collections used for development of these thesauri. From the text collections, we have ex­tracted single words and multiword expressions. Two-word expressions belong to two types of noun groups: <i>Adjective+Noun </i>and <i>Noun+Noun_in_Genitive</i>.</p><p>The extracted expressions were initially or­dered in descending order of their frequencies. Terminologists usually work with these term candidate lists paying more attention to expres­sions with high frequencies. However it was noted that the important terms could have me­dium or low frequencies because of the unbal­ance of text collections. So the aim of our new term extraction method is to reorder the extracted expressions to get more approved terms in the top of the candidate list. We experimented with five thousands of the most frequent two-word expressions from these lists.</p><subsection number="2.1" title="Terminological   Resources   Used for Evaluation"><p>Ontology on Natural Sciences and Technologies comprises Russian terminology in a very broad domain of natural sciences including mathemat­ics, physics, chemistry, geology and elementary biology. It was created for automatic text processing of scientific documents such as auto­matic conceptual indexing, search results visuali­zation, search query expansion, automatic text categorization, text summarization etc. The wide scope of the ontology is intended to support in­terdisciplinary research, to serve as a general source of terminology described in a formalized way. The current volume of Ontology on Natural Sciences is more than 140 thousand terms (Do-brov and Loukachevitch, 2006).</p><p>Banking thesaurus was created during a state contract with the Central Bank of the Russian Federation. It comprises the terminology related to activity of the Central Bank, including such issues as banking activity, banking regulation, monetary politics, macroeconomics. Now it in­cludes about 15 thousand terms.</p><p>In structure, both terminological resources are similar to classical information-retrieval thesauri (ISO 2788), having descriptors, corresponding to concepts of the domain; synonyms and term va­riants attached to the descriptors; relations be­tween the descriptors.</p><p>At the same time, the resources are intended to be used in automatic text processing (in contrast to classical information-retrieval thesauri for manual indexing) and therefore they have considerable coverage of their domains, in particular, including a lot of term variants, occurred in real texts of the domain. This feature of our resources facilitates evaluation of term extraction methods (Nazarenko and Zargayouna, 2009). So we suppose that all term variants have been already described in our gold standards.</p></subsection><subsection number="2.2" title="Measure for Evaluation of Term Ex­traction Performance"><p>The evaluation of term candidates extracted from texts is a complicated procedure, because of, for example, subjectivity of domain experts, varia-tivity of terms (Nazarenko and Zargayouna, 2009).</p><p>We suppose that term extraction is needed for a broad domain with thousands of terms and term variants. A term extraction procedure is based on processing of large domain-specific text collec­tions consisting of hundreds and thousands me­gabytes of texts. From these texts a ranked list of term candidates is generated. The real domain terms should be situated mainly in the top of the list to facilitate expert work or automatic exploi­tation of such a list. So we want to evaluate reor­dering performance of various methods of term recognition</p><p>To evaluate the reordering performance of me­thods we use the measure of average precision adopted from information retrieval (Manning et. al., 2009). Average precision AvP in the task of term extraction is calculated as follows.</p><p>Suppose that in an ordered list of expressions there are k terms, and pos (i) - the position of the i-th term from the beginning of the list. Then the precision on the level of the i-th terminological expression PrecTermi in an ordered list is PrecTerm (pos (i)), that is the value of precision PrecTermi is calculated at the time of inclusion to the list of i-th term and is equal to the percentage of terms in the list from 1 to pos (i) positions.</p><p>Average precision for the given ordered list is equal to the average value of PrecTermi:</p><doubt alpha="50.0" length="2" tooSmall="False" monospace="0.0">1k</doubt><p>AvP= <b>— </b><b>y </b>Pr <i>ecTerm.</i></p></subsection></section><section number="3" title="Features for Term Candidate Reor­dering"><p>For extracted phrases we compute features of three types:</p><p>• features based on a domain-specific text collection,</p><page local="3" global="712"/><p>• features obtained from an Internet search engine,</p><p>• features obtained from a domain-specific thesaurus.</p><p>Each type of features allows us to model differ­ent aspects of domain terms.</p><subsection number="3.1" title="Features Based on Domain Specific Col­lection"><p>We use several features calculated on the basis of a domain-specific text collection. The chosen features reveal different properties of domain terms.</p><p><b>Frequency in the collection (Freq). </b>This feature is often used in term extraction methods because it is known that terms have to be frequent in domain-specific texts and the most frequent phrases of a domain include large share of domain terms.</p><p><b>Mutual information (MI). </b>The feature is also very popular in extraction of terms and is calcu­lated as follows:</p><doubt alpha="58.3" length="12" tooSmall="False" monospace="0.0">N ■ freq(ab)</doubt><doubt alpha="52.6" length="19" tooSmall="False" monospace="0.0">MI(ab) = log(-JHy;)</doubt><doubt alpha="62.5" length="16" tooSmall="False" monospace="0.0">freq(a)■ jreq(b)</doubt><p>where <i>ab - </i>is a two-word phrase, <i>freq () </i>is the frequency of phrases or words in the collection, <i>N - </i>number of words in the collection. The fea­ture indicates difference between real co­occurrences of a phrase and independent occur­rences of phrase components.</p><p><b>Cubical Mutual Information (MI3). </b>This fea­ture is a modification of MI feature. In corpora research it was shown that this feature better or­ders low frequent phrases (Daille et. al., 1998):</p><doubt alpha="53.8" length="13" tooSmall="False" monospace="0.0">N ■ freq3(ab)</doubt><doubt alpha="47.6" length="21" tooSmall="False" monospace="0.0">MI3(ab) = log (-JHv;)</doubt><doubt alpha="66.7" length="15" tooSmall="False" monospace="0.0">freq(a)^freq(b)</doubt><p><b>Insideness. </b>Insideness is calculated as the in­verse ratio between the phrase frequency and the maximal frequency of a three-word expression comprising the given phrase.</p><doubt alpha="60.0" length="10" tooSmall="False" monospace="0.0">freq(*ab*)</doubt><doubt alpha="50.0" length="16" tooSmall="False" monospace="0.0">Inside (ab)=——--</doubt><p><i>freq </i><i>(ab)</i></p><p>This feature is intended to reveal truncated word sequences <i>- </i>parts of real terms. The similar phe­nomenon is modeled by C-value feature, de­scribed in (Maynard and Ananiadou, 2000).</p></subsection><subsection number="3.2" title="Features Based on Internet Search"><p>An important characteristic of a domain term is "termhood" that is relevance to the domain (Ka-geura and Umino, <i>1996). </i>The known way to es­timate "termhood" is comparative analysis of a given text collection with a contrast text collec­tion. The huge collection of Internet texts can serve as such a contrast collection.</p><p>In previous research the Web was used for de­veloping domain specific corpora (Penas et.al., 2001; Baroni and Bernardini, 2004). (Turney, 2003) exploits the Web to obtain the most im­portant domain terms using so called coherence feature, ranking higher term candidates that co-occur with other candidates in Web documents.</p><p>In our study we extract several phrase fea­tures from the Web and combine them with other types of features (collection-based and thesaurus-based). We obtain Internet-based features using xml-interface of Russian Search Engine Yandex on the basis of specially formulated queries. For our experiments we utilised so-called search snippets - short fragments of texts explaining search results.</p><p>Use of Internet search is important for the fol­lowing reasons. First, a text collection of a broad domain is often not sufficient because a lot of fairly significant terms may have relatively low frequencies in it. Involvement of the Internet helps us get additional information on such terms. Secondly, the use of information from the Internet allows us to find out if a given phrase is rigidly connected with the domain.</p><p>To calculate the Internet-based features, 100 snippets from search results were utilised. The snippets from the same query were merged into one document and processed by a morphological processor. As a result, for each set of snippets, lemmas (words in a dictionary form) were extracted and their frequencies of occurrence were calculated.</p><p>So, for every query we obtain a vector of lemmas with corresponding frequencies. Snippets were generated for the whole phrases and their constituent words. We denote Sab <i>- </i>a vector of lemma frequencies derived from phrase snippets, Sa, Sb - vectors of lemmas from constituent word snippets. Using such vectors, the following types of features were calculated.</p><p><b>Scalar Features: Scalar1, Scalar2, Boolean1, Boolean2. </b>The first group of Internet-based features are scalar products of snippet vectors: &lt;Sab, Sa&gt; <b>(Scalar^, </b>&lt;Sab, Sb&gt; <b>(Scala</b>r). Many domain-specific terms have specificity of their meanings, which can not be deduced from their components (so-called non-compositionality).<page local="4" global="713"/> This specificity usually can be revealed using comparison of contexts of a phrase and its component words. The usual way to do this is to find scalar products between vectors of contexts. Also we calculated scalar products of boolean variants of snippet vectors (vector elements are</p><doubt alpha="52.4" length="42" tooSmall="False" monospace="0.0">from {0, 1}) : &lt;Sbab,Sba&gt;(Boolean1),&lt;Sbab,</doubt><p>Sbb&gt; <b>(Boolean2).</b></p><p><b>Features of semantically specific context (SnipFreq0, SnipFreq15 SnipFreq2). </b>Another way to find specificity of a phrase is to find a single lemma that is very frequent in phrase snippets and absent (or rarely mentioned) in component snippets.</p><p>Let lemma L occur fab times in phrase snippets and occur fa, fb times in snippets of components. Then we calculate SnipFreq0 feature as follows:</p><p><i>SnipFreq0= </i>max 1 where <i>f</i><i>ab-a-b= </i><i>max (f</i><i>ab- </i><i>f</i><i>a </i><i>- f</i><i>b, </i><i>0), dlcol </i>is the lemma frequency in documents of a contrast collection, N - is the number of documents in the <i>„   .</i><i> ,   </i><b>( </b><i>N - dlcol </i><b>^ </b><i>.</i></p><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">f</doubt><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">ab-a-b</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">log</doubt><doubt alpha="66.7" length="9" tooSmall="False" monospace="0.0">N - dlcol</doubt><doubt alpha="71.4" length="7" tooSmall="False" monospace="0.0">\ dlcol</doubt><p>contrast collection. Factor log<b>l- I </b>is</p><p><b>^   </b><i>dlcol J</i></p><p>so-called idf-factor known from information retrieval research (Manning et. al., 2009); it helps to diminish influence of frequent general words. The contrast collection is the collection of Belorussian Internet documents distributed in the framework of Russian Information Retrieval Evaluatopn Seminar (www.romip.ru/ en/index.html).</p><p>Features <b>SnipFreq1 </b>and <b>SnipFreq2 </b>are calculated in a similar way excluding words in a window of 1 (2) words near every occurrence of phrase <i>ab. </i>These variants of SnipFreq feature are intended to remove partial fragments of longer terms from consideration. For example, for such macroeconomic terms as <i>negative cash flow </i>and <i>negative cash balance </i>lemmas <i>flow </i>and <i>balance </i>will be very frequent in snippets of phrase <i>negative cash </i>and will be situated immediately after phrase <i>negative cash, </i>but this phrase is not a real term.</p><p><b>The frequency of a phrase in its own snippets (FreqBySnip). </b>We supposed that if the value of this feature is significantly greater than 100 (sometimes this feature reached 250-300 occurrences in 100 snippets), it means that there are many contexts in which this phrase is explained in detail, is the theme of the fragment, and, most likely, this phrase denotes an important concept or a specific entity, as, for example, phrase <i>internal debt </i>in the following snippet: <i>The first distinction to be made is be­tween an <u>internal debt</u> and an external debt. </i><i>An</i><i> <u>internal debt</u> is owed by a nation.</i></p><p><b>Number of definitional words in snippets (NearDefWords). </b>This feature calculates overall frequency of so called definitional words in phrase snippets. These words (as <i>type, class, de­fine </i>etc.) are often used in dictionary definitions. Therefore their presence in snippets can mean that a snippet contains a definition of this phrase or the phrase is used in definition of other term. <b>NearDefWords </b>feature is equal to the number of these definitional words that appeared immediately adjacent (left or right) with the original phrase in snippets.</p><p><b>Number of marker words in snippets (Markers). </b>This feature denotes number of five-ten the most important words of the domain in snippets of the phrase. For the natural science domain these words were as follows: <i>mathematics, mathematical, physics, physical, chemisry, chemical, geology, geological, biology, biological.</i></p><p><b>Number of Internet page titles (SnipTitle).</b></p><p>We calculated number of Internet page titles coinciding with a given phrase, because we supposed that the use of the phrase as the title of an Internet page stresses significance of the phrase.</p></subsection><subsection number="3.3" title="Features Based on Terms of Domain-Specific Thesaurus"><p>In many domains there are well-known terms and even information-retrieval thesauri. The third type of our features is based on the assumption that the known terms can help to predict un­known terms. For the experiments in two domains, we used the relevant thesauri. If a phrase was a thesaurus term, then it was excluded from the terminological basis for feature generation. We considered the following features obtained from a domain-specific thesaurus.</p><p><b>Synonym to Thesaurus Term (SynTerm).</b></p><p>Domain documents can contain a lot of variants of the same term (Nenadic et. al., 2004). Therefore we can suppose that a phrase similar to a thesaurus term is also a term. Let <i>a </i>and <i>b </i>be components of phrase <i>ab.</i><page local="5" global="714"/><i> </i>We consider phrase <i>cd </i>as a synonym of phrase <i>ab </i>if every component word of phrase <i>cd </i>is either equal to a component word of <i>ab </i>either is a synonym of a component word of <i>ab. </i>The order of components in the phrases is unimportant.</p><p><b>Synonym to Non-Term (SynNotTerm). </b>We also fix a feature of similarity to a phrase not included to the thesaurus.</p><p><b>Completeness of Description</b> <b>(Completeness).</b><b> </b>It is possible that component words <i>a </i>and/or <i>b </i>of phrase <i>ab </i>have been already described in a domain thesaurus. For example, <i>a </i>is related to thesaurus descriptor <i>Da </i>and <i>b </i>is related to thesaurus descriptor <i>Db. </i>Descriptor <i>Da</i>has <i>sa </i>synonyms and <i>ra </i>relations to other descriptors. Descriptor Db has <i>sb </i>synonyms and <i>rb</i>relations to other descriptors. <b>Completeness </b>feature is a sum of thesaurus relations of component terms that is:</p><p><i>Completeness = sa + sb + ra+ rb</i></p><p>If a component of a phrase is not included to the thesaurus then its <i>sa </i>and <i>ra </i>are equal to 0.</p></subsection></section><section number="4" title="Results of Experiments"><p>We experimented in two domains: the banking domain and the domain of natural sciences. In all experiments 5 thousand most frequent two-word expressions extracted from the correspond­ing text collections were used. For these expres­sions, all above-mentioned features were calcu­lated. To obtain the best combination of features for term extraction, we used machine learning methods implemented in programming package RapidMiner (www.rapidminer.com). The quality of reordering was evaluated with AvP measure. The training set was three-quarters of the phrase list, the testing set was a remaining part. As basic minimal levels of AvP we used the alphabet or­der and the decreasing frequency order.</p><p>To find the best combination of features for phrase reordering we tested various machine learning methods from RapidMiner package. Every time logistic regression achieved maximal level of AvP. Therefore we took this method as a basic machine learning method for our experi­ments on term extraction.</p><p>Table 1 shows AvP values for single features and their combination obtained with logistic re­gression. SynTerm and SynNotTerm features are Boolean and can not be evaluated with AvP. We concluded that SynTerm feature is highly informative: if <i>SynTerm (ab) </i>=1 then phrase <i>ab </i>is a domain term with probability more than 80%.</p><p><b>Table 1. </b>Average Precision (AvP) for single features and logistic regression. Feature SnipTitle was not extracted for phrases in science domain</p><p>From the table we can see that in both cases the same set of features and using of machine learning methods lead to much higher values of average precision. However there are significant distinctions in ratios between AvP of features between domains. For example, in the banking domain AvP of the frequency feature has the highest value, features with high average preci­sion in the science domain have relatively low values in the banking domain.</p><p>We explain this phenomenon with relative narrowness of the banking domain. Banking documents contain a lot of terminology of neigh­bour domains such as economy or politics. So among extracted expressions, there are many real terms having all specific qualities of "unithood", but not related to the banking activity. In the scientific text collection the share of terms from other domains is much lower.</p><p>Also we can see relative failure of SnipFreqi features in banking domain. The reason of this phenomenon, in our opinion, is as follows: the banking domain is subject to legal regulation, therefore documents of the domain contain a lot of citations from legal acts which leads to false large values of SnipFreqi.</p><p>To evaluate the significance of proposed fea­tures we fulfilled a feature selection procedure. For science domain the selected features were Boolean^ <b>Completeness, FreqBySnip, Inside,</b> <b>MI, Neardefwords, SynTerm </b>(AvP - 82%).<page local="6" global="715"/> For banking domain the selected features were <b>Completeness,  FreqBySnip,  MI, NearDef-</b> <b>Words, </b>Scalar1, SnipFreq0, <b>SynTerm </b>(AvP -78%). Selected features repeated for both do­mains are highlighted. We can see that in both cases all three types of features are represented in the short list of features.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Feature</p></td><td class="cell"><p>AvP (Banking)</p></td><td class="cell"><p>AvP (Natural</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>%</p></td><td class="cell"><p>Sciences)%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Alphabet</p></td><td class="cell"><p>40%</p></td><td class="cell"><p>57%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Frequency</p></td><td class="cell"><p>57%</p></td><td class="cell"><p>66%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MI</p></td><td class="cell"><p>43%</p></td><td class="cell"><p>64%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>MI3</p></td><td class="cell"><p>45%</p></td><td class="cell"><p>67%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Inside</p></td><td class="cell"><p>55%</p></td><td class="cell"><p>75%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FreqBySnip</p></td><td class="cell"><p>53%</p></td><td class="cell"><p>69%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NearDefWords</p></td><td class="cell"><p>49%</p></td><td class="cell"><p>73%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Scalar;</p></td><td class="cell"><p>42%</p></td><td class="cell"><p>61%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Scalar2</p></td><td class="cell"><p>45%</p></td><td class="cell"><p>60%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Boolean!</p></td><td class="cell"><p>49%</p></td><td class="cell"><p>64%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Boolean2</p></td><td class="cell"><p>48%</p></td><td class="cell"><p>62%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SnipFreq0</p></td><td class="cell"><p>34%</p></td><td class="cell"><p>66%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SnipFreqj</p></td><td class="cell"><p>38%</p></td><td class="cell"><p>67%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SnipFreq2</p></td><td class="cell"><p>38%</p></td><td class="cell"><p>67%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Markers</p></td><td class="cell"><p>40%</p></td><td class="cell"><p>65%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Completeness</p></td><td class="cell"><p>52%</p></td><td class="cell"><p>69%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SnipTitle</p></td><td class="cell"><p>50%</p></td><td class="cell"><p>-</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Logistic Regres-</p></td><td class="cell"><p><b>79% (+38.6%</b></p></td><td class="cell"><p><b>83% (+25.8%</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>sion</p></td><td class="cell"><p><b>from Freq)</b></p></td><td class="cell"><p><b>from Freq)</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><section number="5" title="Conclusion"><p>In this paper we proposed to use three types of features for extraction of two-word terms and showed that all these types of features are useful for term extraction. The set of features includes new features such as features extracted from the existing domain-specific thesauri and features based on Internet search results.</p><p>We showed that the combination of several types of features considerably enhances the qual­ity of the term extraction procedure. The devel­oped system of term extraction reorders terms in a list of candidates much better than the basic­line ordering by decreasing frequency.</p><p>We studied the set of features for term extrac­tion in two different domains. We found that for developing term extraction models in a specific domain, it is important to take into account such properties of the domain as broad scope or nar­row scope (science vs. banking) and connection with the socio-political domain, which is regu­lated with legal acts. We suppose that it is possi­ble to find the main types of domains for term extraction, to select the best feature sets and spe­cial machine learning models for every type of domains.</p></section><references><p>M. Baroni, S. Bernardini. 2004. BootCaT: Bootstrap­ping Corpora and Terms from the Web. In <i>Proc. of LREC- 2004: </i>1313-1316.</p><p>B. Daille, E. Gaussier, J. Lang. 1998. An evaluation of statistics scores for word association. In <i>Proc. of Tbilisi Symposium on Logic, Language and Com­putation. </i>CSLI Publications: 177-188.</p><p>B. Dobrov and N. Loukachevitch. 2006. Development of Linguistic Ontology on Natural Sciences and Technology. In <i>Proc. of LREC-2006.</i></p><p>J. Foo and M. Merkel. 2010. Using machine learning to perform automatic term recognition. In <i>Proc. of LREC2010 Aquisition Workshop.</i></p><p>ISO-2788. 1986. Documentation -- Guidelines for the establishment and development of monolingual thesauri.</p><p>K. Kageura and B. Umino. 1996. Methods of auto­matic term recognition: a review. <i>Terminology, 3(2):259-289.</i></p><p>Ch. Manning, P. Raghavan and H. Shutze. 2008. In­troduction to Information Retrieval. Cambridge <i>University Press.</i><i></i></p><p>D. Maynard and S. Ananiadou. 2000. Identifying Terms by their Family and Friends. In <i>Proc. of 18th International Conference on Computational</i> <i>Linguistics COLING-2000.</i><i></i></p><p>A. Nazarenko and H. Zargayouna. 2009. Evaluation</p><p>Term Extraction. In <i>Proc. of RANLP-2009.</i></p><p>G. Nenadic, S. Ananiadou, J. McNaught. 2004. En­hancing automatic term recognition through recog­nition of variation. In <i>Proc. of International Confe­rence on Computational Linguistics COLING-2004: </i>604-610.</p><p>P. Pecina and P. Schlesinger. 2006. Combining asso­ciation measures for collocation extraction. In <i>Proc.</i><i> of Annual Meeting of the Association for Computational Linguistics ACL-2006.</i></p><p>A. Penas, F. Verdejo and J. Gonzalo. 2001. Corpus-Based Terminology Extraction Applied to Informa­tion Access. In <i>Proc. of Corpus Linguistics-2001, </i>Lancaster University.</p><p>S. Sato and Y. Sasaki. 2003. Automatic Collection of Related Terms from the Web. <i>The Companion Vo­lume to the Proceedings of 41st Annual Meeting of the ACL, </i>Sapporo, Japan, 2003: 121-124.</p><p>P. D. Turney. 2003. Coherent Keyphrase Extraction via Web Mining. In <i>Proc. the 18th International Joint Conference on Artificial Intelligence IJCAI-</i> <i>03:</i><i> </i>434-439.</p><p>J.Vivaldi, L. Marquez and H. Rodriguez. 2001. Im­proving Term Extraction by System Combination Using Boosting. In <i>Proc. of ICML </i>2001, LNCS, V2167: 515-526.</p><p>W. Wong, W. Liu. and M. Mennamoun. 2008. De­termination of Unithood and Termhood for Term Recognition. In <i>Proc.: M.Song and Y.Wu. (eds) Handbook of Research on Text and Web Mining Technologies, </i>IGI Global.</p><p>Z. Zhang, J. Iria, Ch. Brewster and F. Ciravegna. 2008. A Comparative Evaluation of Term Recogni­tion Algorithms. In <i>Proc. Language Resources and Evaluation Conference of LREC-2008.</i></p></references></body></article>