<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="57"/><title>Classifying Domain-Specific Terms Using a Dictionary</title><author surname="Kim" givenname="Su Nam"><org  name="University of Melbourne" country="Australia" city="Melbourne"/></author><author surname="Cavedon" givenname="Lawrence"><org  name="RMIT University" country="Australia" city="Melbourne"/></author></firstpageheader><frontmatter><p><b>Classifying Domain-Specific Terms Using a Dictionary</b></p><p><b>Su Nam Kim</b></p><p>Dept. ofCSSE University of Melbourne Melbourne, Australia</p><p>snkim@csse.unimelb.edu.au</p><p><b>Lawrence Cavedon</b></p><p>School of CS IT</p><p>RMIT University Melbourne, Australia</p><p>lawrence.cavedon@rmit.edu.au</p></frontmatter><abstract>Automatically building domain-specific on­tologies is a highly challenging task as it re­quires extracting domain-specific terms from a corpus and assigning them relevant domain concept labels. In this paper, we focus on the second task: i.e., assigning domain concepts to domain-specific terms. Motivated by pre­vious approaches in related research (such as word sense disambiguation (WSD) and named entity recognition (NER)) that use semantic similarity among domain concepts, we ex­plore three types of features — contextual, domain concepts, topics — to measure the semantic similarity of terms; we then assign the domain concepts from the best matching terms. As evaluation, we collected domain-specific terms from FOLDOC, a freely avail­able on-line dictionary for the the <i>Comput­ing </i>domain, and defined 9 domain concepts for this domain. Our results show that be­yond contextual features, using domain con­cepts and topics derived from domain-specific terms helps to improve assigning domain con­cepts to the terms. </abstract></header><body><section number="1" title="Introduction"><p><b>Domain-specific terms </b>are terms that have signif­icant meaning(s) in a specific domain. For exam­ple, terms such as <i>Gulf </i>and <i>Kuwait </i>are associated with the domain of oil due to their frequent appear­ances in contexts related to oil although they indi­cate geographical areas. In some resources, domain-specific terms are further categorized in terms of their domain concepts (i.e., semantic labels/classes).</p><p>For example, <i>Firefox </i>belongs to the domain con­cept Software, while <i>Prolog </i>is associated with the domain concept Programming. In this paper, we use the term <b>domain concept </b>for consistency. Note that in previous work, the meaning of the domain-specificity is associated with either word senses (e.g. (Magnini et al., 2002; Rigutini et al., 2005)) or the statistical use of terms in context (e.g. (Drouin, 2004; Milne et al., 2006; Kida et al., 2007; Park et al., 2008; Kim et al., 2009; Vivaldi and Rodr-guez, 2010)). In WordNet, the domain concept is assigned based on the word senses. Similarly, WordNet Domain has terms with domain con­cepts per sense. However, most work previously conducted work used domain-specificity is based on statistical use. In this paper, we follow the latter def­inition, i.e., domain-specificity associated with the statistical use of the term.</p><p>Domain-specificity of terms has been leveraged in various natural language processing (NLP) and related tasks, such as word sense disambiguation (WSD) (Magnini et al., 2002), named entity recog­nition (NER) (Kazama and Torisawa, 2007), and query expansion (Rigutini et al., 2005). Resources containing domain information fall into two groups: the list of domain-specific terms without domain concepts (e.g. Agrivoc, EUROVOC, ASFA The­saurus); and with domain concepts (e.g. WordNet, WordNet Domain, Unified Medical Language Sys­tem (UMLS)). Although there have been efforts de­veloping such knowledge resources, the task has been generally carried out by hand, requiring high cost and time. Further, even hand-crafted resources are often limited in terms of quality and quantity.</p><p>Su Nam Kim and Lawrence Cavedon. 2011. Classifying Domain-Specific Terms Using a Dictionary. In <i>Proceedings of Australasian Language Technology Association Workshop, </i>pages 57—65<page local="2" global="58"/></p><p>Moreover, the content needs to be constantly up­dated/maintained as new words are added. As a re­sult, there has been recent work on automatic on­tology builders (e.g. OntoLearn, Text2Onto) that work by extracting domain-specific terms and tagging domain-concepts to build such resources.</p><p>Building a domain-specific ontology requires two main tasks — extracting domain-specific terms and assigning the domain concept(s). There have been several methods proposed for each task and also as a complete ontology builder—we describe such work in Section 2. In this paper, our interest lies on assign­ing domain concepts to the existing domain-specific resources. In one sense our task can be viewed as building a taxonomy from dictionaries (Rigau et al., 1998) and/or a semantic class labelling task (Punuru and Chen, 2007). Since some resources are already publicly available (despite shortcomings), utilizing these resources reduces the time for manually devel­oping training data, and should lead to robust sys­tems due to consistent labeling. In addition, such resources are reusable for enlarging the existing re­sources or creating new semantic resources.</p><p>Our basic approach is to use semantic similar­ity between domain-specific terms. Contextual fea­tures have often been employed for semantic sim­ilarity in various tasks, such as text categoriza­tion (Joachims, 1998) and dialogue act classification (Ivanovic, 2005). Thus, we also explore using con­text as base features. Furthermore, we explore the use of rich semantic features. That is, we employ the domain concepts and topics derived from known domain-specific terms over the same resource as ad­ditional features. We detail our rich semantic fea­tures in Section 4.2 and 4.3. In evaluation, we ap­plied our approaches to the domain Computing, as the interest in this domain is growing due to the large volume of web corpora, including social media such as web forums and blogs.</p><p>In the following sections, we describe related work and the existing resources in Section 2 and 3. We then describe our features in Section 4, and eval­uate our methods in Section 5. We summarize our work in Section 6.</p></section><section number="2" title="Related Work"><p>There are two individual sub-tasks that deal with domain-specific terms — extraction/identification and labeling domains/concepts. Further, extract­ing domain-specific terms is combined with tech­nical term extraction in order to extract candidates, while identification of domain-specific terms is a bi­nary decision (i.e., with a given term, determining whether it is domain-specific to the target domain).</p><p>A number of extraction methods have been pro­posed (Drouin, 2004; Rigutini et al., 2005; Milne et al., 2006; Kida et al., 2007; Park et al., 2008; Kim et al., 2009; Vivaldi and Rodrguez, 2010).</p><p>Most used supervised approaches while (Park et guez, 2010) undertook the task in an unsupervised manner. In addition, (Rigutini et al., 2005) used sense-based domain-specificity, while others used statistical use-based measures to determine domain-specificity of the candidate terms. (Rigutini et al., 2005) is motivated by the intuition that, similar to word sense disambiguation, domain-specificity can be identified using contextual semantic similarity in which those terms occur, since domain-specificity is associated with the word senses. (Milne et al., 2006) studied Wikipedia entries as domain-specific terms and crosschecked the terms in Agrivoc with Wikipedia entries to verify the domain-specificity of Wikipedia entries. The basic idea in (Kida et al., 2007) is that a domain can be identified via a list of known technical domain terms. As unsupervised ap­proaches, (Park et al., 2008) introduced a probabil­ity based weighting in order to measure the domain-specificity of the term over a large corpus. Similarly, (Kim et al., 2009) used term frequencies across the documents using modified TF IDF, which replace a document with a domain. (Vivaldi and Rodrguez, 2010) made use of Wikipedia categories and page structures. The intuition is that the Wikipedia cat­egories are domain-specific, thus, by retrieving the Wikipedia entries through the category trees starting with a target domain, the domain-specific terms un­der the target domain can be automatically retrieved.</p><doubt alpha="50.0" length="46" tooSmall="False" monospace="0.0">al., 2008; Kim et al., 2009; Vivaldi and Rodr-</doubt><p>The task of domain assignment has some rela­tionship to the word sense disambiguation (WSD) and named entity recognition (NER) tasks. While WSD attempts to assign the correct sense of terms from some given repository of senses (typically, WordNet), assigning domains to domain-specific terms in our work first requires that we construct the repository.<page local="3" global="59"/> NER is a subtask of information extrac­tion that involves finding named entities and assign­ing each a tag from a predefined set of categories, such as LOCATION or PERSON. The difference with our task in this paper is that in both WSD and NER, the target terms are generally in some use con­text (i.e., the correct word sense of target term de­pends on that context), while our targets are isolated, i.e., appear out of context. In this paper, our ap­proach is closer to corpus-based WSD which nor­mally uses co-occurrence of terms between two cor­pora.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">58</doubt><p>In recent years, there have been systems proposed to extract terms and to assign semantic labels to them (Navigli and Velard, 2004; Cimiano and Vlker, 2005; Nicola et al., 2009). OntoLearn (Navigli and Velard, 2004) has three components. First, it ex­tracts a domain terminology from Web sites. It then assigns the domain concepts in order to build a hierarchical structure of ontologies. The system uses semantic similarity between WordNet concepts for component words in a candidate and concep­tual relations among the concept components based on word senses. Finally, ontologies in WordNet are trimmed and enriched with the extracted domain concepts. Text2Onto (Cimiano and Vlker, 2005) is another ontology builder and includes three compo­nents. First, the system represents the knowledge as metadata in the form of instantiated modeling prim­itives called Probabilistic Ontology Model, which is language-independent. Second, the system uses the user interaction in order to measure the domain-specificity for candidates. Finally, it accumulates the ontologies based on previously added ontolo­gies to overcome computational redundancy over time when corpus/documents are changed. More recently, (Nicola et al., 2009) developed an auto­matic ontology builder by combining Unified Soft­ware Development Processing (UP) and UML. It bases its characteristics on UP and uses UML to sup­port the preparation of the blueprints of the ontology development.</p><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">3 Data</doubt><p>Multiple taxonomy resources such as WordNet and Wikipedia are available for identifying terms in the Computing domain. However, not all of these systematically assign semantic labels to terms. To determine suitability of popularly used resources for our task, we first investigated their utility.</p><p>WordNet3.0 (Fellbaum, 1998) includes domain information but the number of terms with domain information is very limited. Moreover, it does not include many terms related to the Computing do­main, and those terms it does include are often proper nouns (e.g. <i>Firefox) </i>or compound nouns (e.g. <i>wireless connection). </i>For WordNet Domain, (Magnini et al., 2002) developed a domain sys­tem and semi-manually assigned domains to Word­Net terms in terms of their word senses. Although the size of the resource with domain information is larger than WordNet3.0 (i.e., 6,050 nouns with one sense), it is still a relatively small resource. More­over, the domain called <i>factotum </i>(i.e., "undecided") is used for many terms, which makes it less usable.</p><p>Wikipedia<footnote anchor="1"/> is the largest folksonomy taxon­omy. It contains terms (hereafter, <i>entries) </i>and cat­egories per entry. It also provides hyperlinks be­tween entries and entry pages. Despite its vast size, Wikipedia is not designed to be a dictionary, thus it does not contain definitions entries. Moreover, the categories are not systematically organized and of­ten contain noise (which are not relevant to the target domains).</p><p>Wiktionary<footnote anchor="2"/>, on the other hand, is a grow­ing online dictionary for all domains. It has char­acteristics similar to WordNet, such as definitions and relations (e.g. hypernym, synonym). It also par­tially contains domain information per word sense. In addition, it is linked to Wikipedia. However, since one term could have multiple senses and not all senses are tagged with a specific domain, it re­quires a preprocessing step to discover terms re­lated to the specific domain (for this paper, Com­puting domain only). The number of unique terms in Wiktionary is 10,586 without counting indi­vidual word senses.</p><footnote label="1">http://en.wikipedia.org/wiki/Mai^_Page  2  http://en.wiktionary .org/wiki/dictionary</footnote><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">59</doubt><page local="4" global="60"/><p>The final resource we consider is FOLDOC<footnote anchor="3"/>. FOLDOC is the (so far) largest handcrafted on-line dictionary for the Computing domain. It also con­tains definitions and sub-domains such as <i>hardware </i>and <i>operating system, </i>and provides hyperlinks to other dictionary terms in the definition and links to Wikipedia and OneLook dictionary search<footnote anchor="4"/>. Unlike Wiktionary, only word senses of terms related to Computing are listed, thus, all terms in the dictionary are relevant to the Computing do­main. For these reasons, as well as the size of the resource being sufficient to evaluate our method, we decided to use FOLDOC for our purposes. To un­derstand FOLDOC better, we checked the overlaps between it and other resources in Table 1. Note that all resources were retrieved in March 2011.</p><p>FOLDOC contains 14,826 unique terms with mul­tiple senses, resulting in 16,450 terms in total. 13,072 and 3,378 terms are <i>direct </i>and <i>redirect, </i>re­spectively (the concept of <i>direct </i>and <i>redirect </i>is the same as that in Wikipedia). Among direct terms, 8,621 terms have manually assigned domain con-cept(s). The total number of domain concepts in the dictionary is 188. Finally, we manually mapped 188 onto 9 domain concepts which are super-labels. For example, labels in foldoc, security, specifi­cation, Unix are mapped on to Networking, Doc­umentation, OS, respectively. Note that, based on our observations, we found many of the labels de­fined in FOLDOC are too fine-grained, and some are used only for 1 or 2 terms. Furthermore, the labels are not hierarchically structured. In addition, simi­lar to the trade-off between fine-grained vs. coarse­grained word senses, we believe coarse-grained la­bels would be more usable (e.g. document classifi­cation using coarse-grained labels of terms), thus, we used 9 super-labels in this work.</p><table caption="Table 2 shows the final domain concepts we used"></table><footnote label="3">http://foldoc.org/  4 http://www.onelook.com/</footnote><p>and the number of instances in each domain concept. Note that since one term can have multiple semantic labels, the total number of instances with one label is 10,147. Table 3 shows the number of terms with multiple senses and multiple domain concepts.</p><p><i>computer science, hardware, operating system, </i>and <i>soft­ware, </i>respectively.</p></section><section number="4" title="Methodology"><subsection number="4.1" title="Feature Set I: Bag-of-Words"><p>n-gram-based bag-of-words (BoW) features are one of the most broadly applied features to measure the semantic similarity between two terms/texts. This has been used in various tasks such as document classification (Joachims, 1998), dialogue act clas­sification (Ivanovic, 2005) and term classification (Lesk, 1986; Baldwin et al., 2008).</p><p>As shown in (Hulth and Megyesi, 2006), key­words along the contextual features (i.e., simple 1-grams) are useful in identifying semantic similar­ity. However, keywords are often multi-grams such as 2-grams (e.g. <i>Fast Ethernet, optical mouse) </i>and 3-grams (e.g. <i>0/1 knapsack problem, Accelerated Graphics Port). </i>Sharing the same intuition, some previous work (Ivanovic, 2005) employed not only 1-grams but also 2-grams for the classification task. Similarly, we also observed that terms are often multi-grams. Thus, in this work, we also explored various n-grams. In evaluation, we tested 1- and 2-grams individually as well as the combination of 1and 2-grams together (i.e., 1+2-grams).<page local="5" global="61"/> Note that since previous work has shown that the use of lem­mas performed better than raw words, we chose to use lemmas as features. We also tested BoWs from nouns and verbs only. As feature weights, we tested simple Boolean and TFIDF. In evaluation, the fea­tures are filtered with respect to the frequency of in­dexing words. That is, we tested three different term frequencies (i.e., frequency &gt; 1,2, and 3) in order to select the indexing terms as BoW features.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">60</doubt><table caption="Table 2: Data Size per Domain Concept.8,621is the number of word types, whereCS, HW, OS, SWindicate" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Domain (Terms)</p></td><td class="cell"><p>Domain (Terms)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CS(755)</p></td><td class="cell"><p>Documentation (1,906)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>HW (1,490)</p></td><td class="cell"><p>Jargon (298)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Networking (1,220)</p></td><td class="cell"><p>OS (363)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Programming (3,042)</p></td><td class="cell"><p>SW (144)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Other(929)</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Total</p></td><td class="cell"><p>10,147(8,621)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 1: Overlap between the FOLDOC dictionary and other resources." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Source</p></td><td class="cell"><p>WikiDic</p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p>WordNet</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Instance</p></td><td class="cell"><p>10,586</p></td><td class="cell"><p>10,863,326</p></td><td class="cell"><p>117,798</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Overlap</p></td><td class="cell"><p>1,682</p></td><td class="cell"><p>9,917</p></td><td class="cell"><p>2,756</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 3: Terms with multiple labels and senses." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Info.</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>3</p></td><td class="cell"><p>4</p></td><td class="cell"><p>5</p></td><td class="cell"><p>6</p></td><td class="cell"><p>7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Label</p></td><td class="cell"><p>7172</p></td><td class="cell"><p>1353</p></td><td class="cell"><p>79</p></td><td class="cell"><p>8</p></td><td class="cell"><p>—</p></td><td class="cell"><p>—</p></td><td class="cell"><p>—</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sense</p></td><td class="cell"><p>7957</p></td><td class="cell"><p>475</p></td><td class="cell"><p>124</p></td><td class="cell"><p>41</p></td><td class="cell"><p>11</p></td><td class="cell"><p>2</p></td><td class="cell"><p>2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></subsection><subsection number="4.2" title="Feature (II): Domain Concepts of Domain-Specific Terms"><p>BoW features are useful for measuring the seman­tic similarity between two targets. However, we ob­served that since the number of terms in the dic­tionary definition is small, there will be a lack of context (similar to shortcomings reported in (Lesk, 1986) for WSD using dictionary definition). On the other hand, we noticed that a term's definition of­ten contains terms which belong to the same domain concepts. For example, the target term <i>Ethernet </i>be­longs to the domain concept Networking, and its definition is "A local area network first described by <i>..</i>.". <i>Local area network </i>in the definition also belongs to the same domain concept Networking. Hence, we use the domain concept(s) of dictionary terms found in the definition of the target term as a feature.</p><p>We also extend the target's definition with its dic­tionary terms. To overcome the pitfall of the algo­rithm in (Lesk, 1986) due to lack of terms in the def­inition, (Baldwin et al., 2008) utilized the extended definition from the dictionary terms found in the tar­get's definition. Similarly, instead of the definition of dictionary terms in the target's definition (i.e., ex­tended definition), we used the domain concept(s) of dictionary terms in the extended definition. We hy­pothesised that using these domain concepts would provide more direct information about domain con­cepts of the target term.</p><p>In Table 4, we demonstrate how and what to ex­tract as domain concepts for the target <i>database. </i>The definition of the target <i>database </i>contains dic­tionary terms <i>database, table, flat file, comma-separated values. </i>Since <i>database </i>is the target term itself and <i>comma-separated values </i>is not found in the dictionary, we use the domain concepts from <i>ta-</i> <i>ble </i>and <i>flat file </i>only, which include CS, and OS, HW. Note that some terms have multiple labels as described in Section 3. We also extend <i>table </i>and <i>flat file </i>to obtain the extended domain concepts from the extended definition. Finally, we accumulated do­main concepts, CS, Programming, Documenta­tion from <i>records, </i>CS from <i>relational database </i>and Documentation from <i>flat ASCII </i>from the extended definitions.</p></subsection><subsection number="4.3" title="Feature (III): Topics of Domain-Specific Terms"><p>Topic modeling (Blei et al., 2003) is an unsuper-vised method to cluster documents based on con­text information. As it is not a classification method, the topics produced by the topic modeling algorithm are abstract, thus not directly associated with prede­fined semantic labels. However, we observed that topic terms for each topic are generally associated with the domain concepts. From our observation, we hypothesized that (ideally) one topic is associated with one domain concept (although some may con­tain multiple domain concepts same as multi-sensed words). As such, we assigned topic ID(s) per dic­tionary terms using topic modeling software<footnote anchor="5"/>, then used this as an additional feature with BoWs. Like­wise, we also obtained topic ID(s) for dictionary terms found in the definition of the target term. Note that depending on the features used to obtain topics, the association between topic IDs and our 9 domain concepts would change. Table 5 demonstrates the topics and how we extract the <i>Topic </i>and <i>extended Topic </i>features using the same example use above. <i>database </i>is tagged with topic ID = 1,2,4 while <i>table </i>has topic ID = 4. We represented features by accu­mulating the topic IDs over 9 topic IDs which are the same number of our domain concepts.</p></subsection></section><section number="5" title="Experiments"><p>For our evaluation, we first replaced the email ad­dresses, numbers, urls with their category <i>EMAIL, NUMBER, URL, </i>respectively. We then per­formed POS tagging and lemmatization using Lingua::EN::Tagger and morph tools (Min­nen et al., 2001), respectively.  For learning, we<page local="6" global="62"/></p><footnote label="5">The topic modeling tool we used can be downloaded from http://www.ics.uci.edu/ newman/code/</footnote><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">61</doubt><p>Term     <b>I   </b>Label   <b>I </b>Definition <b>Target Term, "database" and its Domain Concept and Definition</b> database</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">CS</doubt><p>A <i>database </i>containing a single <i><u>table</u></i><u>c</u>g, stored in a single <i>flatfile</i><b><i>Qg </i></b>|_|W' often in a human-readable format such as <i>comma separated values </i>or fixed-width columns.</p><p><b>Extending Definitions of Dictionary Terms found in Target's Definition</b></p><p>A collection of recordscs!Programmlng,Documentatlon   in a <i><u>relational database</u></i><u>ç</u>g.</p><p>A single file containing <i>flat ASCII </i>Documentation representing or encoding some structure, e.g. a <i>database, </i>tree or network.</p><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">table</doubt><p>flat file simulated our method by both supervised and semi-supervised approaches. We used SVM (Joachims, 1998)<footnote anchor="6"/> for supervised learning and SVMlin (Sind-hwani and Keerthi, 2006)<footnote anchor="7"/> for semi-supervised learning. For supervised learning, we performed 10­fold cross-validation over 8,621 terms which have manually assigned labels in FOLDOC. For semi-supervised learning, we used the same data for test and training and 4,451 unlabeled terms in FOLDOC as unlabeled data. As the baseline system, we used the feature TF IDF valued <i>1-gram </i>from all terms with frequency <i>&gt; </i>1. The performances are com­pared using micro-averaged F-score <i>Fß.</i></p><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">OS, HW</doubt><table caption="Table 4: Extracting domain concepts of dictionary terms in definitions."></table><subsection number="5.1" title="Supervised Learning"><p>Table 6 shows performance by supervised learners with various BoWs. Note that we only report per­formance using TF IDF since those using Boolean weights performed poorly. We also ran the experi­ments over different frequency which leads to vari­ous numbers of indexing terms. Finally, we tested noun- and verb-only features.</p><footnote label="6">http://www.cs.cornell.edu/People/tj/sv^_light/svmhmm.html  7  http://vikas.sindhwani.org/svmlin.html</footnote><p>Overall, the best performance is produced by us­ing both 1- and 2-grams with frequency <i>&gt; </i>1, as this configuration contains the largest amount of fea­tures. However, the improvement by adding 2-grams is not significant (i.e., <i>52.01% </i>vs. <i>52.47% </i>in <i>Fß</i>). Between using all terms vs. nouns and verbs only, using all terms performed slightly better. Despite dominant information derived from nouns and verbs, other POS tagged words also contributed to distin­guishing the domain concepts. Likewise, the per­formances using nouns and verbs only are generally better when using features with frequency <i>&gt; </i>1.</p><p>Table 7 shows performance using both n-grams and semantic features. At first, adding rich seman­tic features (i.e., <i>Domain Concept, Topic) </i>signifi­cantly improved performance <i>(52.01% </i>vs. <i>60.62%). </i>In particular, <i>Topic </i>features helped to improve per­formance. As we hypothesised, topics are likely associated with domain concepts which resulted in performance improvements. <i>Domain concept </i>fea­tures also helped to gain higher performance, as they provide more direct semantic information than n-gram features. Between direct and extended (i.e., indirect) semantic features, we noticed that <i>extended</i><page local="7" global="63"/></p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">62</doubt><table caption="Table 5: Extracting topics for target terms and dictionary terms." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Type</p></td><td class="cell"><p>Value</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>Topics</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>T1</p></td><td class="cell"><p>T2</p></td><td class="cell"><p>T3</p></td><td class="cell"><p>T4</p></td><td class="cell"><p>T5</p></td><td class="cell"><p>T6</p></td><td class="cell"><p>T7</p></td><td class="cell"><p>T8</p></td><td class="cell"><p>T9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Target Term</p></td><td class="cell"><p><i>database</i></p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dictionary Term</p></td><td class="cell"><p><i>table</i></p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>in Target Definition</p></td><td class="cell"><p><i>flat file</i></p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Feature</p></td><td class="cell"><p>Direct</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>representation</p></td><td class="cell"><p>Extended</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"><p>0</p></td><td class="cell"><p>2</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Table 6: Performances with BoW Features: Performance of the baseline system is marked with ]. The best perfor­mance is bold-faced. <i>Indexing </i>means n-grams. Indexing value is TF IDF.</p><p>Table 7: Performances with Rich Semantic Features in FM: The best performances in each group are bold-faced.</p><p>features decreased performance as they tend to intro­duce more erroneous instances. Likewise, using all words as well as 1-grams performed better among all various <i>n</i>-grams with few exceptions.</p><p>Table 8 and Figure 1 show performance over indi­vidual classes and detail of predicted labels. This is the system using TF IDF valued <i>1-grams </i>from all terms with frequency &gt; 2, as this was our best-performing system. Overall, we found that many do­main concepts are mislabeled with Programming and Documentation since they are most often used concepts and could be a border concept for terms la­beled with other domain concepts. For example, CS and Documentation are often labeled as Program­ming, while Networking is mislabeled as Docu­mentation.</p><p>Finally, we used randomized estimation to calcu­late whether any performance differences between methods are statistically significant (Yeh, 2000) and found all systems exceeding the baseline system had p-value <i>&lt; </i>0.05, which indicates significant improve­ment.</p></subsection><subsection number="5.2" title="Semi-supervised Learning"><p>For semi-supervised learning, we evaluated the im­pact of the size of training data. We observed that despite increasing training data, performance does not significantly improve. However, to com­pare the performance between supervised and semi-supervised systems, we simulated semi-supervised system with unused training data from FOLDOC. Table 9 shows the performance of semi-supervised learning using two groups of features: <i>1-gram with frequency &gt; 1, </i>and <i>1-gram with frequency &gt; 1 + Domain Concept. </i>Note that we did not test with <i>Topic </i>as topic IDs change according to the data.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">63</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Feature</p></td><td class="cell"><p>Indexing</p></td><td class="cell"><p>Frequency &gt;1 (F1) <i>Vß</i><i> </i><i>R</i><i> Fß</i></p></td><td class="cell"><p>Frequency&gt;2 (F2) <i>Vß</i><i> </i><i>Kß</i><i> Fß</i></p></td><td class="cell"><p>Frequency&gt;3 (F3) <i>Vß</i><i> </i><i>Kß</i><i> Fß</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>All</p><p>Terms</p></td><td class="cell"><p>1</p><p>2</p><p>1+2</p></td><td class="cell"><p>56.64   48.07 <i>52.01] </i>54.42   46.19 49.97 57.14   48.50 <b>52.47</b></p></td><td class="cell"><p>56.64   48.07 52.01 54.42   46.19 49.97 57.14   48.50 52.47</p></td><td class="cell"><p>56.56   48.00 51.93 51.68   43.87 47.45 57.05   48.42 52.38</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Noun</p><p>+ Verb</p></td><td class="cell"><p>1</p><p>2</p><p>1+2</p></td><td class="cell"><p>55.56   47.16 51.02 52.39   44.47 48.10 56.37   47.85 51.76</p></td><td class="cell"><p>55.56   47.16 51.02 52.39   44.47 48.10 56.37   47.85 51.76</p></td><td class="cell"><p>55.49   47.10 50.95 49.30   41.84 45.27 56.19   47.69 51.59</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p>All Words</p></td><td class="cell"><p>Noun+Verb</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Feature</p></td><td class="cell"><p>Index</p></td><td class="cell"><p>F1</p></td><td class="cell"><p>F2</p></td><td class="cell"><p>F3</p></td><td class="cell"><p>F1</p></td><td class="cell"><p>F2</p></td><td class="cell"><p>F3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Domain</p></td><td class="cell"><p>1</p></td><td class="cell"><p>57.30</p></td><td class="cell"><p>57.02</p></td><td class="cell"><p>57.20</p></td><td class="cell"><p>56.90</p></td><td class="cell"><p>56.96</p></td><td class="cell"><p><b>57.51</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Concept</p></td><td class="cell"><p>2</p></td><td class="cell"><p>55.61</p></td><td class="cell"><p>55.78</p></td><td class="cell"><p>55.22</p></td><td class="cell"><p>55.70</p></td><td class="cell"><p>55.77</p></td><td class="cell"><p>55.54</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>1+2</p></td><td class="cell"><p>56.84</p></td><td class="cell"><p>57.16</p></td><td class="cell"><p>57.07</p></td><td class="cell"><p>56.09</p></td><td class="cell"><p>56.57</p></td><td class="cell"><p>56.41</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Extended</p></td><td class="cell"><p>1</p></td><td class="cell"><p><b>55.97</b></p></td><td class="cell"><p>55.86</p></td><td class="cell"><p>55.74</p></td><td class="cell"><p>55.63</p></td><td class="cell"><p>55.13</p></td><td class="cell"><p>55.88</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Domain</p></td><td class="cell"><p>2</p></td><td class="cell"><p>53.44</p></td><td class="cell"><p>53.98</p></td><td class="cell"><p>53.31</p></td><td class="cell"><p>53.79</p></td><td class="cell"><p>53.82</p></td><td class="cell"><p>53.43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Concept</p></td><td class="cell"><p>1+2</p></td><td class="cell"><p>55.91</p></td><td class="cell"><p>55.82</p></td><td class="cell"><p>55.50</p></td><td class="cell"><p>55.54</p></td><td class="cell"><p>55.49</p></td><td class="cell"><p>55.30</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Topic</p></td><td class="cell"><p>1</p></td><td class="cell"><p>60.58</p></td><td class="cell"><p><b>60.62</b></p></td><td class="cell"><p>60.50</p></td><td class="cell"><p>59.65</p></td><td class="cell"><p>59.99</p></td><td class="cell"><p>59.75</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>2</p></td><td class="cell"><p>50.94</p></td><td class="cell"><p>54.27</p></td><td class="cell"><p>53.30</p></td><td class="cell"><p>50.99</p></td><td class="cell"><p>54.03</p></td><td class="cell"><p>52.88</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>1+2</p></td><td class="cell"><p>59.75</p></td><td class="cell"><p>60.11</p></td><td class="cell"><p>59.82</p></td><td class="cell"><p>58.37</p></td><td class="cell"><p>59.48</p></td><td class="cell"><p>59.20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Extended</p></td><td class="cell"><p>1</p></td><td class="cell"><p>59.18</p></td><td class="cell"><p>59.09</p></td><td class="cell"><p>59.10</p></td><td class="cell"><p>59.47</p></td><td class="cell"><p><b>59.60</b></p></td><td class="cell"><p>59.40</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Topic</p></td><td class="cell"><p>2</p></td><td class="cell"><p>52.77</p></td><td class="cell"><p>53.54</p></td><td class="cell"><p>52.11</p></td><td class="cell"><p>50.44</p></td><td class="cell"><p>51.51</p></td><td class="cell"><p>50.03</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>1+2</p></td><td class="cell"><p>58.73</p></td><td class="cell"><p>58.98</p></td><td class="cell"><p>58.56</p></td><td class="cell"><p>56.52</p></td><td class="cell"><p>56.85</p></td><td class="cell"><p>56.58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="8" global="64"/><table caption="Table 8: Confusion Matrix withNoun+Verb:1-gram+F2+Topicwhere the proportion is presented in()"></table><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(%)</doubt><doubt alpha="22.2" length="9" tooSmall="False" monospace="0.0">S3œ*£°»«£</doubt><doubt alpha="18.8" length="16" tooSmall="False" monospace="0.0">§ ^        I nS°</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">a</doubt><p>Figure 1: Performances over individual classes with <i>Noun+Verb:1-gram+F2+Topic: Proportion </i>means the amount of instances per domain concept in %.</p><p>The results show that the use of simple BoWs improves performance but does not exceed the best performance produced using <i>BoW+Domain Con­cept/Topic. </i>On the other hand, adding <i>Domain Con­cept </i>actually decreases performance when adding more unlabeled data, except when adding a small amount (i.e., 1000). We found that <i>Domain Con­cept </i>is sensitive, thus this decreased the overall per­formance when it includes more noise by semi-supervised learning.  Previously, the outcomes of semi-supervised learning have shown that its effec­tiveness is somewhat dependent on the nature of the task and aspects of features. There have been mixed reports on improvement by semi-supervised learn­ing; some work reported significant improvement while other showed little or no impact on the task. In this paper, we observed that semi-supervised learn­ing would not help improve the performance on clas­sifying domain concepts. We expect that since the best performance by the supervised system is not high enough, adding automatically assigned data as training data introduces further error.</p></subsection></section><section number="6" title="Conclusion"><p>We have proposed an automatic method to assign domain concepts to terms in FOLDOC using various contextual features as well as semantic features — <i>Domain Concept </i>and <i>Topic. </i>We demonstrated that the system performed best when using rich semantic features directly derived from dictionary terms. We also showed that for the target task, semi-supervised learning did not significantly improve performance, unlike for other tasks. As future work, we are in­terested in applying the proposed method to other existing resources in order to build a larger domain-specific ontology resource.</p><table caption="Table 8: Confusion Matrix withNoun+Verb:1-gram+F2+Topicwhere the proportion is presented in ()" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>G/P</p></td><td class="cell"><p>CS</p></td><td class="cell"><p>Document</p></td><td class="cell"><p>HW</p></td><td class="cell"><p>Jargon</p></td><td class="cell"><p>Network</p></td><td class="cell"><p>OS</p></td><td class="cell"><p>Program</p></td><td class="cell"><p>SW</p></td><td class="cell"><p>Other</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>CS</p></td><td class="cell"><p>435(40.6)</p></td><td class="cell"><p>95(8.9)</p></td><td class="cell"><p>140(13.1)</p></td><td class="cell"><p>9(0.8)</p></td><td class="cell"><p>29(2.7)</p></td><td class="cell"><p>23(2.1)</p></td><td class="cell"><p>297(27.7)</p></td><td class="cell"><p>12(1.1)</p></td><td class="cell"><p>31(2.9)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Document</p></td><td class="cell"><p>74(3.8)</p></td><td class="cell"><p>1101(56.3)</p></td><td class="cell"><p>151(7.7)</p></td><td class="cell"><p>43(2.2)</p></td><td class="cell"><p>174(8.9)</p></td><td class="cell"><p>33(1.7)</p></td><td class="cell"><p>252(12.9)</p></td><td class="cell"><p>17(0.9)</p></td><td class="cell"><p>111(5.7)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>HW</p></td><td class="cell"><p>35(3.0)</p></td><td class="cell"><p>86(7.4)</p></td><td class="cell"><p>850(72.7)</p></td><td class="cell"><p>10(0.9)</p></td><td class="cell"><p>41(3.5)</p></td><td class="cell"><p>18(1.5)</p></td><td class="cell"><p>93(8.0)</p></td><td class="cell"><p>2(0.2)</p></td><td class="cell"><p>34(2.9)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Jargon</p></td><td class="cell"><p>23(4.8)</p></td><td class="cell"><p>77(16.1)</p></td><td class="cell"><p>62(13.0)</p></td><td class="cell"><p>145(30.4)</p></td><td class="cell"><p>29(6.1)</p></td><td class="cell"><p>16(3.4)</p></td><td class="cell"><p>74(15.5)</p></td><td class="cell"><p>7(1.5)</p></td><td class="cell"><p>44(9.2)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Network</p></td><td class="cell"><p>19(1.7)</p></td><td class="cell"><p>205(18.0)</p></td><td class="cell"><p>22(1.9)</p></td><td class="cell"><p>12(1.1)</p></td><td class="cell"><p>766(67.4)</p></td><td class="cell"><p>12(1.1)</p></td><td class="cell"><p>58(5.1)</p></td><td class="cell"><p>4(0.4)</p></td><td class="cell"><p>39(3.4)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>14(2.7)</p></td><td class="cell"><p>42(8.2)</p></td><td class="cell"><p>60(11.7)</p></td><td class="cell"><p>11(2.2)</p></td><td class="cell"><p>31(6.1)</p></td><td class="cell"><p>198(38.7)</p></td><td class="cell"><p>130(25.4)</p></td><td class="cell"><p>6(1.2)</p></td><td class="cell"><p>19(3.7)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Program</p></td><td class="cell"><p>51(2.7)</p></td><td class="cell"><p>86(4.5)</p></td><td class="cell"><p>48(2.5)</p></td><td class="cell"><p>24(1.3)</p></td><td class="cell"><p>38(2.0)</p></td><td class="cell"><p>21(1.1)</p></td><td class="cell"><p>1576(83.3)</p></td><td class="cell"><p>7(0.4)</p></td><td class="cell"><p>42(2.2)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>SW</p></td><td class="cell"><p>32(4.6)</p></td><td class="cell"><p>94(13.4)</p></td><td class="cell"><p>48(6.8)</p></td><td class="cell"><p>7(1.0)</p></td><td class="cell"><p>52(7.4)</p></td><td class="cell"><p>27(3.8)</p></td><td class="cell"><p>302(43.0)</p></td><td class="cell"><p>73(10.4)</p></td><td class="cell"><p>67(9.5)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Other</p></td><td class="cell"><p>72(5.8)</p></td><td class="cell"><p>120(9.7)</p></td><td class="cell"><p>109(8.9)</p></td><td class="cell"><p>37(3.0)</p></td><td class="cell"><p>60(4.9)</p></td><td class="cell"><p>15(1.2)</p></td><td class="cell"><p>260(21.1)</p></td><td class="cell"><p>16(1.3)</p></td><td class="cell"><p>542(44.0)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p><b>a     </b>proportion —b— /\         'each.P' —■—"</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>/ \        'each.R' —e—</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>: y\</i></p></td><td class="cell"><p>/   \        'each.F' —•—-</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>i</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 9: Performances by semi-supervised learning in %.1is unigram andDmeans domain concept." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>S</p></td><td class="cell"><p>Semi-S (Unlabeled Term #)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Fea.</p></td><td class="cell"><p>0</p></td><td class="cell"><p>1000    2000    3000 4415</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1</p></td><td class="cell"><p>52.01</p></td><td class="cell"><p>56.94   58.05   57.97 58.86</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1+D</p></td><td class="cell"><p>57.30</p></td><td class="cell"><p><b>59.33   </b>58.02   57.83 57.17</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Timothy Baldwin, Su Nam Kim, Francis Bond, Sanae Fujita, David Martinez, and Takaaki Tanaka. 2008. Mrd-based word sense disambiguation: Further ex­tending lesk. In <i>Proceedings of 3rd International Joint Conference on Natual Language Processing, </i>pages 775-780, Hyderabad, India.</p><p>David Blei, Andrew Ng, and Michael Jordan. 2003. Latent dirichlet allocation.<page local="9" global="65"/> <i>Journal of Machine Learning Research, </i>3:993-1022.</p><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">O</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">64</doubt><p>Philipp Cimiano and Johanna Vlker. 2005. Text2onto - a framework for ontology learning and data-driven change discovery. In <i>10th International Conference on Applications of Natural Language to Information Systems (NLDB), </i>pages 227-238.</p><p>Patrick Drouin. 2004. Detection of domain specific ter­minology using corpora comparison. In <i>Proceedings ofthe fourth international Conference on Language Resources and Evaluation, </i>pages 79-82, Lisbon, Por­tugal.</p><p>Christiane Fellbaum, editor. 1998. <i>WordNet, An Elec­tronic Lexical Database. </i>MIT Press, Cambridge, Mas­sachusetts, USA.</p><p>Annette Hulth and Beata B. Megyesi. 2006. A study on automatically extracted keywords in text categoriza­tion. In <i>Proceedings of 21st International Conference on Computational Linguistics and 44th Annual Meet­ing ofthe Association for Computational Linguistics, </i>pages 537-544, Sydney, Australia.</p><p>Edward Ivanovic. 2005. Dialogue act tagging for instant messaging chat sessions. In <i>Proceedings ofthe ACL Student Research Workshop, </i>pages 79-84, Ann Arbor,</p><p>USA.</p><p>Thorsten Joachims. 1998. Text categorization with sup­port vector machines: Learning with many relevant features. In <i>Proceedings ofEuropean Conference on Machine Learning, </i>pages 137-142.</p><p>Jun'ichi Kazama and Kentaro Torisawa. 2007. Exploit­ing Wikipedia as external knowledge for named entity recognition. In <i>Proceedings of the 2007 Joint Confer­ence on Empirical Methods in Natural Language Pro­cessing and Computational Natural Language Learn­ing, </i>pages 698-707.</p><p>Mitsuhiro Kida, Masatsugu Tonoike, Takehito Utsuro, and Satoshi Sato. 2007. Domain classification of tech­nical terms using the web. <i>Systems and Computers, </i>38(14):2470-2482.</p><p>Su Nam Kim, Timothy Baldwin, and Min-Yen Kan. 2009. An unsupervised approach to domain-specific term extraction. In <i>Australasian Language Technol­ogy Association Workshop, </i>pages 94-98.</p><p>Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. In <i>Proceedings of the 1986 SIGDOC Conference, </i>pages 24-26, Ontario, Canada.</p><p>Bernardo Magnini, Carlo Strapparava, Giovanni Pezzulo, and Alfio Gliozzo. 2002. Using domain information for word sense disambiguation. <i>Natural Langauge En­gineering, </i>8(4):359-373.</p><p>David Milne, Olena Medelyan, and Ian H. Wit­ten.   2006.   Mining domain-specific thesauri from wikipedia : A case study. In <i>Proceedings ofthe 2006 IEEE/WIC/ACM International Conference on Web In­telligence, </i>pages 442-448, Washington, USA.</p><p>Guido Minnen, John Carroll, and Darren Pearce. 2001. Applied morphological processing of English. <i>Natu­ral Language Engineering, </i>7(3):207-223.</p><p>Robert Navigli and Paola Velard. 2004. Learning domain ontologies from document warehouses and dedicated websites. <i>Computational Linguistics, </i>30(2):151-179.</p><p>Antonio De Nicola, Michele Missikoff, and Roberto Nav-igli. 2009. A software engineering approach to ontol­ogy building. <i>Information Systems, </i>34(2):258-275.</p><p>Youngja Park, Siddharth Patwardhan, Karhik Visweswariah, and Stephen C. Gates. 2008. An empirical analysis of word error rate and keyword error rate. In <i>Proceedings ofInternational Conference on Spoken Language Processing, </i>Brisbane, Australia.</p><p>Janardhana Punuru and Jianhua Chen. 2007. Learning for semantic classification of conceptual terms. <i>Gran­ular Computing, IEEE International Conference on,</i> 0:253.</p><p>German Rigau, Horacio Rodriguez, and Eneko Agirre. 1998. Building accurate semantic taxonomies from monolingual mrds. In <i>Proceedings of the 36th Annual Meeting ofthe Association for Computational Linguis­tics and 17th International Conference on Computa­tional Linguistics, Volume 2, </i>pages 1103-1109.</p><p>Leonardo Rigutini, B. Liu, and Marco Magnini. 2005. An em based training algorithm for cross-language text categorization. In <i>Proceedings ofthe Web Intel­ligence Conference (WI), </i>pages 529-535, Compiegne, France.</p><p>Vikas Sindhwani and S. Sathiya Keerthi. 2006. Large scale semi-supervised linear svms. In <i>Annual ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR).</i></p><p>Jorge Vivaldi and Horacio Rodrguez. 2010. Finding domain terms using wikipedia. In <i>Proceedings of the Seventh conference on International Language Re­sources and Evaluation (LREC2010).</i></p><p>Alexander Yeh. 2000. More accurate tests for the statisti­cal significance of result differences. In <i>International Conference on Computational Linguistics (COLING),</i> pages 947-953.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">65</doubt></references></body></article>