<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1"/><title>Two-Phase Biomedical Named Entity Recognition Using A Hybrid Method</title><pubinfo>R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 646-657,2005. © Springer-Verlag Berlin Heidelberg 2005</pubinfo><author surname="Kim" givenname="Seonho"><org  name="Korea University" country="Korea" city="Seoul"/></author><author surname="Yoon" givenname="Juntae"><org  name="Frei University" country="Germany" city="Berlin"/></author><author surname="Park" givenname="Kyung-Mi"><org  name="Korea University" country="Korea" city="Seoul"/></author><author surname="Rim" givenname="Hae-Chang"><org  name="Korea University" country="Korea" city="Seoul"/></author></firstpageheader><frontmatter><p><b>Two-Phase Biomedical Named Entity Recognition Using A Hybrid Method</b></p><p>Seonho Kim,JuntaeYoon<footnote anchor="2"/>, Kyung-Mi Parkland Hae-Chang Rim</p><p><b><footnote anchor="1"/> Dept. of Computer Science and Engineering, Korea University, Seoul, Korea <footnote anchor="2"/> NLP Lab. Daumsoft Inc. Seoul, Korea</b></p></frontmatter><abstract><b>Abstract. Biomedical named entity recognition (NER) is a difficult problem in biomedical information processing due to the widespread am­biguity of terms out of context and extensive lexical variations. This pa­per presents a two-phase biomedical NER consisting of term boundary detection and semantic labeling. By dividing the problem, we can adopt an effective model for each process. In our study, we use two exponential models, conditional random fields and maximum entropy, at each phase. Moreover, results by this machine learning based model are refined by rule-based postprocessing implemented using a finite state method. Ex­periments show it achieves the performance of F-score 71.19% on the JNLPBA 2004 shared task of identifying 5 classes of biomedical NEs.</b> </abstract></header><body><section number="1" title="Introduction"><p>Due to dynamic progress in biomedical literature, a vast amount of new infor­mation and research results have been published and many of them are available in the electronic form - for example, like the PubMed MedLine database. Thus, automatic knowledge discovery and efficient information access are strongly de­manded to curate domain databases, to find out relevant information, and to integrate/update new information across an increasingly large body of scien­tific articles. In particular, since most biomedical texts introduce specific no­tations, acronyms, and innovative names to represent new concepts, relations, processes, functions, locations, and events, automatic extraction of biomedical terminologies and mining of their diverse usage are major challenges in biomed­ical information processing system. In these processes, biomedical named entity recognition (NER) is the core step to access the higher level of information.</p><p>In fact, there has been a wide range of research on NER like the NER task on the standard newswire domain in the Message Understanding Conference (MUC-6). In this task, the best system reported 95% accuracy in identifying seven types of named entities (person, organization, location, time, date, money, and per­cent). While the performance in the standard domain turned out to be quite good as shown in the papers, that in the biomedical domain is not still satisfactory, which is mainly due to the following characteristics of biomedical terminologies: First, NEs have various naming conventions. For instance, some entities have descriptive and expanded forms such as <i>"activated B cell lines, 47 kDa sterol</i> <i>regulatory element binding factor", </i>whereas some entities appear in shortened or abbreviated forms like "EGFR"and <i>"EGF receptor</i>" representing epidermal growth factor receptor.<page local="2"/> Second, biomedical NEs have the widespread ambiguity out of context. For instance, <i>"IL-2</i>" can be doubly classified as "protein" and "DNA" according to its context. Third, biomedical NEs often comprise a nested structure, for example <i>"{DNA){protein)TNF alpha{/protein)gene(/DNA)". </i>Ac­cording to [13], 16.57% of biomedical terms in GENIA have cascaded construc­tions. In the case, recognition of the longest terms is the main target in general. However, in our evaluation task, when the embedded part of a term is regarded as the meaningful or important class in the context, the term is labeled only with the class of embedded one. Thus, identification of internal structures of NEs is helpful to recognize correct NEs. In addition, more than one NE often share the same head noun with a conjunction/disjunction or enumeration structure, for instance, <i>"IFN-gamma and GM-CSF mRNA", "CD33+, CD56+, CD16- acute leukemia </i>"or <i>"antigen- or cAMP-activated Th2 cell </i>". Last, there is a lot of inter-annotator disagreement. [7] reported that the inter-annotator agreement rate of human experts was just 77.6% when performing gene/protein/mRNA classifica­tion task manually.</p><p>Thus, a lot of term occurrences in real text would not be identified with sim­ple dictionary look-up, despite the availability of many terminological databases, as claimed in [12]. That is one of the reasons why machine learning approaches are more dominant in biomedical NER than rule-based or dictionary-based ap­proaches [5], even though existence of reliable training resources is very critical.</p><p>Accordingly, much work has been done on biomedical NER, based on ma­chine learning techniques. [3] and [13] have used hidden Markov Model (HMM) for biomedical NER where state transitions are made by semantic trigger fea­tures. [4] and [11] have applied maximum entropy plus Markovian sequence based models such as maximum entropy markov model (MEMM) and conditional ran­dom fields (CRFs), which present a way for integrating different features such as internal word spellings and morphological clues within an NE string and con­textual clues surrounding the string in the sentence.</p><p>These works took an one-phase based approach where boundary detection of named entities and semantic labeling come together. On the other hand, [9]proposed a two-phase model in which the biomedical named entity recognition process is divided into two processes of distinguishing biomedical named entities from general terms and labeling the named entities with semantic classes that they belong to. They use support vector machines (SVM) for each phase. How­ever, the SVM does not provide an easy way for labeling Markov sequence data like B following O and I following B in named entities. Furthermore, since this system is tested on the GENIA corpus rather than JNLPBA 2004 shared task, we cannot confirm the effectiveness of this approach on the ground of experiments for common resources.</p><p>In this paper, we present a two-phase named entity recognition model: (1) boundary detection for NEs and (2) term classification by semantic labeling. The advantage of dividing the recognition process into two phase is that we can select separately a discriminative feature set for each subtask, and moreover can measure effectiveness of models at each phase.<page local="3"/> We use two exponential models for this work, namely conditional random fields for boundary detection having Markov sequence, and the maximum entropy model for semantic labeling. In ad­dition, results from the machine learning based model are refined by a rule-based postprocessing, which is implemented using a finite state transducer (FST). The FST is constructed with the GENIA 3.02 corpus. We here focus on identification of five classes of NEs, i.e. "protein", "RNA", "DNA", "cell line", and "cell type" and experiments are conducted on the training and evaluation set provided by the shared task in COLING 2004 JNLPBA.</p></section><section number="2" title="Training"><subsection number="2.1" title="Maximum Entropy and Conditional Random Fields"><p>Before we describe the features used in our model, we briefly introduce the ME and CRF model which we make use of. In the ME framework, the conditional probability of predicting an outcome <i>o </i>given a history <i>h </i>is defined as follows:</p><doubt alpha="46.4" length="28" tooSmall="False" monospace="0.0">P\(o\h) = exp\yXifi(h,o)](1)</doubt><doubt alpha="40.0" length="5" tooSmall="False" monospace="0.0">Z\(h)</doubt><p>where <i>fi(h, o) </i>is a binary-valued feature function, <i>Ai </i>is the weighting parameter of <i>fi(h,o), k </i>is the number of features, and <i>Z\(h) </i>is a normalization factor for <i>£op\(o\h)=l. </i>That is, the probability <i>p\(o\h) </i>is calculated by the weighted sum of active features. Given an exponential model with k features and a set of training data, empirical distribution, weights of the k features are trained to maximize the model's log-likelihood:</p><doubt alpha="38.1" length="21" tooSmall="False" monospace="0.0">L(P)=EP(h,o)%(o\h)(2)</doubt><p>Although the maximum entropy model above provides a powerful tool for classification by integrating different features, it is not easy to model the Markov sequence data. In this case, the CRF is used for a task of assigning label sequences to a set of observation sequences. Based on the principle of maximum entropy, a CRF has a single exponential model for the joint probability of the entire sequence of labels given the observation sequence. The CRF is a special case of the linear chain that corresponds to conditionally trained finite-state machine and define conditional probability distributions of a particular label sequence <b>s </b>given observation sequence <b>o</b> where <b>s </b>= si .<page local="4" global="649"/> ..sn,and <b>o </b>= <i>o\ ...on, Z</i><b>(o) </b>is a normalization factor, and each feature is a transition function [8]. For example, we can think of the following feature function.</p><doubt alpha="56.5" length="62" tooSmall="False" monospace="0.0">Pa(s|o) =zJÖ)exp{YJkj=lAjFj(s,o)) ^Fj(s&gt;o)=E™=1fj(si-l,si,o,i)</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">648</doubt><p>{
1if Sj_i=B and Sj=I, and the observation word at position i is <i>"gene" </i>(4) 0 <i>otherwise</i></p><p>Our CRFs for term boundary detection have a first-order Markov dependency between output tags. The label at position i, is one of B, I and O. In contrast to the ME model, since B is the beginning of a term, the transition from O to I is not possible. CRFs constrain results to consider only reasonable paths. Thus, total 8 combinations are possible for <i>(si-1,si) </i>and the most likely <b>s </b>can be found with the Viterbi algorithm. The weights are set to maximize the conditional log likelihood of labeled sequences in the training set using a quasi-Newton method calledL-BFGS[2].</p></subsection><subsection number="2.2" title="Features for Term Boundary Detection"><p>Table 1shows features for the step of finding the boundary of biomedical terms. Here, we give a supplementary description of a part of the features.</p><table caption="Table 1. Feature set for boundary detection (+:conjunction)"></table><p><b>— word and POS: </b>5 words(target word(wj), left two words, and right two words) and three <i>POS(POSWi1 </i>, <i>POSWi</i>, <i>POSWi+1 </i>) are considered.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>M<b>odel</b></p></td><td class="cell"><p><b>Feature</b></p></td><td class="cell"><p><b>Description</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Word</b></p></td><td class="cell"><p><b><i>Wi-i,Wi-2,Wi,Wi+i,Wi+2</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Word Normalization</b></p></td><td class="cell"><p><b>normalization forms of the 5 words</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>POS</b></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Word Construction form</b></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Word Characteristics</b></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Contextual Bigrams</b></p></td><td class="cell"><p><b><i>Wi-i + Wi</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b><i>Wi </i></b><i>+ wi+1</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><b><i>Wi+1 + Wi+2</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Contextual Trigrams</b></p></td><td class="cell"><p><b><i>Wi-i + Wi +</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Bigram POS</b></p></td><td class="cell"><p><i>POSWi_, + POSw%</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"><p><i>POSVH + POSVH+l</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Trigram POS</b></p></td><td class="cell"><p><i>POSw%_t + POSw% + POSw%+1</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Modifier</b></p></td><td class="cell"><p><b>MODI</b>(wi)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Header</b></p></td><td class="cell"><p><b>HEAD</b>(wi)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>SUFFIX</b></p></td><td class="cell"><p><b>SUFFIX</b>(wi)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Chunk Type</b></p></td><td class="cell"><p><i>C'TypeWi</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>CRF, </b><i>MEmarkov</i></p></td><td class="cell"><p><b>Chunk Type + Pre POS</b></p></td><td class="cell"><p><i>CType^ + POS^</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>M^</i><i> Emarkov</i></p></td><td class="cell"><p><b>Pre label</b></p></td><td class="cell"><p><i>labelWil</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>M^</i><i> Emarkov</i></p></td><td class="cell"><p><b>Pre label + Cur Word</b></p></td><td class="cell"><p><i>label </i><i>Wil</i><i> </i><i>+</i><i> </i><b><i>Wi</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="5" global="650"/><p><b>— word normalization: </b>This feature contributes to word normalization. We attempt to reduce a word to its stem or root form with a simple algorithm which has rules for words containing plural, hyphen, and alphanumeric let­ters. Specifically, the following patterns are considered.</p><doubt alpha="63.8" length="47" tooSmall="False" monospace="0.0">(1) "lymphocytes", "cells"—"lymphocyte", "cell"</doubt><doubt alpha="31.2" length="32" tooSmall="False" monospace="0.0">(2) "il-2", "il-2a", "il2a"—"il"</doubt><doubt alpha="59.3" length="54" tooSmall="False" monospace="0.0">(3) "5-lipoxygenase", "v-Abl"—s- "lipoxygenase", "abl"</doubt><p>(4) "peri-kappa" or"t-cell" has twonormalizationformsof"peri"and"kappa" and "t" and "cell" respectively.</p><p>(5) "Ca2+-independent" has two roots of "ca" and "independent". (ô) The root of digits is "D".</p><p><b>— informative suffix: </b>This feature appears if a target word has a salient suffix for boundary detection. The list of salient suffixes is obtained by relative entropy [10].</p><p><b>— word construction form: </b>This feature indicates how a target word is or-thographically constructed. Word shapes refer to a mapping of each word on equivalence classes that encodes with dashes, numerals, capitalizations, lower letters, symbols, and so on. All spellings are represented with combina­tions of the attributes1. For instance, the word construction form of <i>"IL-2</i>" would become "IDASH-ALPNUM".</p><p><b>— word characteristics: </b>This feature appears if a word represents a DNA sequence of "A","C","G","T" or Greek letter such as beta or alpha, ordinal index such as I, II or unit such as BU/ml, micron/mL. It is encoded with "ACGT", "GREEK", "INDEX", "UNIT".</p><p><b>— head/modifying information: </b>If a word prefers the rightmost position of terminologies, we regard it has the property of a head noun. On the other hand, if a word frequently occurs in other positions, we regard it has the property of a modifying noun. It can help to establish the beginning and ending point of multi-word entities. We automatically extract 4,382 head nouns and 7,072 modifying nouns from the training data as shown in</p><p>Table 2.</p><p><b>— chunk-type information: </b>This feature is also effective in determining the position of a word in NEs, "B", "I", "O" which means "begin chunk", "in chunk" and "others", respectively. We consider the chunk type of a target word and the conjunction of the current chunk type and the POS of the previous word to represent the structure of an NE.</p><p>We also tested an ME-based model for boundary detection. For this, we add two special features : previous state (label) and conjunction of previous label<page local="6" global="651"/></p><footnote label="1">"IDASH" (inter dash), "EDASH" (end dash), "SDASH" (start dash), "CAP"(capitalization), "LOW" (lowercase), "MIX"(lowercase and capitaliza­tion letters), "NUM"(digit), "ALPNUM"(alpha-numeric), "SYM"(symbol), "PUNC" (punctuation),and "COMMA" (comma)</footnote><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">650</doubt><p>T<b>wo-Phase Biomedical Named Entity Recognition 651 Table 2. Examples of Head/Modifying Nouns</b> and current word to consider <b>state transition.</b><b> </b>That is, a previous label can be represented as a feature function in our model as follows:</p><p><b>,      s _ </b>J 1 <i>if </i>pre_label+tw=B+gene,o=I , .</p><p>MM) = <i>\o </i><i>otherwise (5j</i></p><p>It means that the target word is likely to be inside a term (I), when the word is "gene" and the previous label is "B". In our model, the current label is de-terministically assigned to the target word with considering the previous state with the highest probability.</p></subsection><subsection number="2.3" title="Features for Semantic Labeling"><p>Table3showsfeaturesforsemanticlabeling with respect to recognized NEs.</p><p><b>— word contextual feature: </b>We make use of three kinds of internal and ex­ternal contextual features: words within identified NEs, their word normal­ization forms, and words surrounding the NEs. In Table 3, <i>NEW0 </i>denotes the rightmost word in an identified NE region. Moreover, the presence of specific head nouns acting as functional words takes precedence when de­termining the term class, even though many terms do not contain explicit term category information. For example, functional words, such as <i>"factor</i>", <i>"receptor</i>", and <i>"protein" </i>are very useful in determining <i>protein </i>class, and <i>"gene</i>", <i>"promoter</i>", and <i>"motif </i>" are clues for classifying <i>DNA </i>[5]. In gen­eral, such functional words are often the last word of an entity. This is the reason we consider the position where a word occurs in NEs along with the word. For inside context features, we use non-positional word features as well. As non-positional features, all words inside NEs are used.</p><p><b>— internal bigrams and trigrams: </b>We consider the rightmost bigrams/ trigrams inside identified NEs and the normalized bigrams/trigrams.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Modifying Nouns</b></p></td><td class="cell"><p><b>Head Nouns</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>nf-kappa</b></p></td><td class="cell"><p><b>cytokines</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>nuclear</b></p></td><td class="cell"><p><b>elements</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>activated</b></p></td><td class="cell"><p><b>assays</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>normal</b></p></td><td class="cell"><p><b>complexes</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>phorbol</b></p></td><td class="cell"><p><b>macrophages</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>viral</b></p></td><td class="cell"><p><b>molecules</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>inflammatory</b></p></td><td class="cell"><p><b>pathways</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>murine</b></p></td><td class="cell"><p><b>extracts</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>electrophoretic</b></p></td><td class="cell"><p><b>glucocorticoids</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>acute</b></p></td><td class="cell"><p><b>levels</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>intracellular</b></p></td><td class="cell"><p><b>responses</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>epstein-barr</b></p></td><td class="cell"><p><b>clones</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>cytoplasmic</b></p></td><td class="cell"><p><b>motifs</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="7" global="652"/><table caption="Table 3. Feature Set for Semantic Classification"></table><p><b>— IDASH-word related bigrams/trigrams: </b>This feature appears if <i>NEW0</i>or <i>NEW-1 </i>contains dash characters. In this case, the bigram/trigram are additionally formed by removing all dashes from the spelling. It is useful to deal with lexical variants.</p><p><b>— keywords: </b>This feature appears if the identified NE is informative key­word with respect to a specific class. The keywords set comprises terms obtained by the relative entropy between general and biomedical domain corpora.</p></subsection></section><section number="3" title="Rule-Based Postprocessing"><p>A rule-based method can be used to correct errors by NER based on machine learning. For example, the CRFs tag <i>"IL-2 receptor expression"as </i>"B I I", since the NEs ended with <i>"receptor expression</i>" in training data almost belong to <i>"other-name" </i>class even if the NEs ended with <i>"receptor" </i>belong to <i>"pro­tein" </i>class. It should be actually tagged as "B I O". That kind of errors is caused mainly by the cascaded phenomenon in biomedical names. Since our sys­tem considers all NEs belonging to other classes in the recognition phase, it tends to recognize the longest ones. That is, in the term classification phase, such NEs are classified as "other" class and are ignored. Thus, the system losts embedded NEs although the training and evaluation set in fact tends to consider only the embedded NE when the embedded one is more meaningful or important.</p><p>This error correction is conducted by the rule-based method, i.e. <b>If </b><i>condi­tion </i><b>THEN </b><i>action. </i>For example, the rule <b>'IF </b><i>wi-2 </i>=IL-2, <i>w— </i>1=receptor and Wj=expression <b>THEN </b>replace the tag of <i>wi </i>with O' canbeappliedfor theabove case. We use a finite state transducer for this rule-based transformation, which is easy to understand with given lexical rules, and very efficient. Rules used for the FST are acquired from the GENIA corpus. We first retrieved all NEs in­cluding embedded NEs and longest NEs from GENIA 3.02 corpus and change<page local="8" global="653"/></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">652</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F<b>eature</b></p></td><td class="cell"><p><b>description</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Word Features (positional)</b></p><p><b>Word Features (non-positional)</b></p><p><b>Word Normalization (positional)</b></p><p><b>Left Context (Words Surrounding an NE)</b></p><p><b>Right Context</b></p><p><b>Internal Bigrams</b></p><p><b>Internal Trigrams</b></p><p><b>Normalized Internal Bigrams</b></p><p><b>Normalized Internal Trigrams</b></p><p><b>IDASH-word related Bigrams/Trigrams</b></p><p><b>Keyword</b></p></td><td class="cell"><p>Affi»„tters, <i>NEW_3, NEW_2, NEW_1, NEWQ</i></p><p><i>WFNEm_3, WFNEm_2, WFNEvJl , WFNEmo</i></p><p><i>LC'W</i><b><i>-2, </i></b><i>LCW</i><b><i>-x</i></b></p><p><i>RCW+1, RC'W</i><b><i>+2</i></b></p><p><i>NEW_, + NEW0</i></p><p><i>NEW_2 + NEW_1 + NEW0</i></p><p><i>WFNEm_1 +WFNEmo</i></p><p><i>NEW_2 + NEW_, + NEW0</i></p><p><i>KEYWORD(NEi)</i></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">IL-2/O</doubt><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">IL-2/B</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">o</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">gene/I</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">gene/O</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">O</doubt><p>F<b>ig. 1. Non-Deterministic FST</b> <b>Fig.</b><b> 2. Deterministic FST</b> the outputs of all other classes except the target 5 classes to O. That is, the input of FST is a sequence of words in a sentence and the output is categories corresponding to the words.</p><doubt alpha="50.0" length="6" tooSmall="False" monospace="0.0">IL-2/E</doubt><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">gene/e</doubt><p>Then, we removed the rules in conflict with NE information from the training corpus. These rules are non-deterministic (Figure 1), and we can change it to the deterministic FST (Figure 2) since the lengths of NEs are finite. The deter­ministic FST is made by defining the final output function for the deterministic behavior of the transducer, delaying the output. The deterministic FST is de­fined as follows: <i>(S\,S2,Q, i, F, </i><i>&lt;S&gt;, </i><i>*,p), </i>where <i>S\ </i>is a finite input alphabet; <i>S2</i>is a finite output alphabet; <i>Q </i>is a finite set of states or vertices; <i>i </i><i>G </i><i>Q </i>is the initial state; <i>F </i><i>Ç </i><i>Q </i>is the set of final states; <i>(g&gt; </i>is the deterministic state transi­tion function that maps <i>Q </i><i>x </i><i>S\ </i>on Q; <i>* </i>is the deterministic emission function that maps <i>Q </i><i>x </i><i>S\ </i>on <i>£2 </i>and <i>p </i>: <i>F </i><i>— </i><i>£2 </i>is the final output function for the deterministic behavior of the transducer.</p></section><section number="4" title="Evaluation"><subsection number="4.1" title="Experimental Environments"><p>In the shared task, only biomedical named entities which belong to 5 specific classes are annotated in the given training data. That is, terms belonging to other classes in GENIA are excluded from the recognition target. However, we consider all NEs in the boundary detection step since we separate the NER task into two phases. Thus, in order to utilize other class terms, we additionally annotated "O" class words in the training data where they corresponds to other classes such as <i>other_organic_compound, lipid, </i>and <i>multi_cell </i>in GENIA 3.02p version corpus. During the annotation, we only consider the longest NEs on<page local="9" global="654"/></p><table caption="Table 4. Number of training examples"></table><p>GENIA. As a consequence, we find all biomedical named entities in text at the term detection phase. Then, biomedical NEs classified as <i>other </i>class are changed to <i>O </i>at the semantic labeling phase. The total words that belong to <i>other </i>class turned out to be 25,987. Table 4 shows the number of NEs with respect to each class on the training data. In our experiments, a quasi-Newton method called the L-BFGS with Gaussian Prior smoothing is applied for parameter estimation [2].</p></subsection><subsection number="4.2" title="Experimental Results"><p>Table 5 shows the overall performance on the evaluation data. Our system achieves an F-score of 71.19%. As shown in the table, the performance of NER for celLline class was not good, because its boundary recognition is not so good as other classes. Also, Table 6shows the results of semantic classification. In par­ticular, the system often confuses <i>protein </i>with <i>DNA, </i>and <i>celLline </i>with <i>celLtype. </i>Among the correctly identified 7,093 terms, 790 terms were misclassified.</p><p>Table 7 shows the performance of each phase. Our system obtains 76.88% F-score in the boundary detection task and, using 100% correctly recognized terms from annotated test data, 90.54% F-score in the semantic classification task. Currently, since we cannot directly assess the accuracy of the term detection process on the evaluation set because of <i>other </i>class words, the 75% of the training data were used for training and the rest for testing.</p><table caption="Table 5. Overall performance on the evaluation data"></table><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">654</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>R<b>NA</b></p></td><td class="cell"><p><b>DNA</b></p></td><td class="cell"><p><b>celLline</b></p></td><td class="cell"><p><b>celLtype</b></p></td><td class="cell"><p><b>protein</b></p></td><td class="cell"><p><b>other</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>472</b></p></td><td class="cell"><p><b>5,370</b></p></td><td class="cell"><p><b>2,236</b></p></td><td class="cell"><p><b>2,084</b></p></td><td class="cell"><p><b>16,042</b></p></td><td class="cell"><p><b>11,475</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 6. Confusion matrix over evaluation data" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Fully Correct</b></p></td><td class="cell"><p><b>Left Correct</b></p></td><td class="cell"><p><b>Right Correct</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>C<b>lass</b></p></td><td class="cell"><p><b>Recall Precision F-score</b></p></td><td class="cell"><p><b>F-score</b></p></td><td class="cell"><p><b>F-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>protein</b></p></td><td class="cell"><p><b>76.30</b></p></td><td class="cell"><p><b>69.71</b></p></td><td class="cell"><p><b>72.85</b></p></td><td class="cell"><p><b>77.60</b></p></td><td class="cell"><p><b>79.15</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>DNA</b></p></td><td class="cell"><p><b>67.80</b></p></td><td class="cell"><p><b>64.91</b></p></td><td class="cell"><p><b>66.33</b></p></td><td class="cell"><p><b>68.36</b></p></td><td class="cell"><p><b>74.57</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>RNA</b></p></td><td class="cell"><p><b>73.73</b></p></td><td class="cell"><p><b>63.04</b></p></td><td class="cell"><p><b>67.97</b></p></td><td class="cell"><p><b>71.09</b></p></td><td class="cell"><p><b>74.22</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>celLline</b></p></td><td class="cell"><p><b>57.40</b></p></td><td class="cell"><p><b>54.88</b></p></td><td class="cell"><p><b>56.11</b></p></td><td class="cell"><p><b>59.04</b></p></td><td class="cell"><p><b>65.69</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>celLtype</b></p></td><td class="cell"><p><b>70.12</b></p></td><td class="cell"><p><b>77.64</b></p></td><td class="cell"><p><b>73.69</b></p></td><td class="cell"><p><b>74.89</b></p></td><td class="cell"><p><b>81.51</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>overall</b></p></td><td class="cell"><p><b>72.77</b></p></td><td class="cell"><p><b>69.68</b></p></td><td class="cell"><p><b>71.19</b></p></td><td class="cell"><p><b>74.75</b></p></td><td class="cell"><p><b>78.23</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>g<b>old/sys</b></p></td><td class="cell"><p><b>protein</b></p></td><td class="cell"><p><b>DNA</b></p></td><td class="cell"><p><b>RNA</b></p></td><td class="cell"><p><b>celLline</b></p></td><td class="cell"><p><b>celLtype</b></p></td><td class="cell"><p><b>other</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>protein</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>72</b></p></td><td class="cell"><p><b>3</b></p></td><td class="cell"><p><b>1</b></p></td><td class="cell"><p><b>4</b></p></td><td class="cell"><p><b>267</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>DNA</b></p></td><td class="cell"><p><b>97</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>49</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>RNA</b></p></td><td class="cell"><p><b>11</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>celLline</b></p></td><td class="cell"><p><b>10</b></p></td><td class="cell"><p><b>1</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>63</b></p></td><td class="cell"><p><b>37</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>celLtype</b></p></td><td class="cell"><p><b>21</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>92</b></p></td><td class="cell"><p><b>0</b></p></td><td class="cell"><p><b>57</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="10" global="655"/><p>T<b>wo-Phase Biomedical Named Entity Recognition 655 Table 7. Performance of term detection and semantic classification</b></p><p>Also, we compared our model with the one-phase model. The detailed results are presented in Table 8. Both of them have pros and cons. The best-reported system presented by [13] uses one-phase strategy. In our evaluation, the two-phase method shows a better result than the one-phase method, although direct comparison is not possible since we tested with a maximum entropy based expo­nential models in all cases. The features for one-phase method are identical with the recognition features except that the local context of a word is extended as previous 4 words and next 4 words. In addition, we investigate whether the con­sideration of "other" class words is helpful in the recognition performance. Table 8 shows explicit annotations of other NE classes much improve the performance of existing entity types.</p><p>In the next experiment, we test how individual methods have an effect on the performance in the term detection step. Table9shows the results obtained by com­bining different methods in the NER process. At the semantic labeling phase, all methods employed the ME model using the features described in 2.3. Baseline1 is the two-phase ME model which restrict the inspection of NE candidates to the NPs which include at least one biomedical salient word. Baseline2 is the two-phase ME model considering all words. In order to retrieve domain salient words, we utilized a relative frequency ratio of word distribution in the domain corpus and that in the general corpus [10]. We used the Penn II raw corpus as out-of-domain corpus. Both models do not use the features related to previous labels. As a re­sult, usage of salient words decrease the performance and it only speeds up the training process. Baseline2+FST indicates boundary extension/contraction using FST are applied as postprocessing step in baseline2 recognition. In addition, we compared use of CRFs and ME with Markov process features. For this, we added features of previous labels to the feature set for ME. Baseline2+MEMarkov is the two-phase ME model considering all features including previous label related fea­tures. Baseline2+CRF is a model exploiting CRFs and baseline2+CRF+FST is a model using CRF and FST as postprocessing. As shown in Table9, the CRFs based model outperforms the ME based model.<page local="11" global="656"/> Our system reached F-score 71.19% on the <i>baseline2 + CRF </i>+ <i>FST </i>model.</p><table caption="Table 8. Performance of NE recognition methods (one-phase vs. two-phase)" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Recall Precision F-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>term detection </b><i>(MEMarkov) </i><b>term detection (CRF) semantic classification</b></p></td><td class="cell"><p><b>74.03      75.31 74.67 76.14      77.64 76.88 87.50      93.81 90.54</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>overall NER</b></p></td><td class="cell"><p><b>72.77      69.68 71.19</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>method</b></p></td><td class="cell"><p><b>Recall Precision F-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>one-phase</b></p></td><td class="cell"><p><b>64.23</b></p></td><td class="cell"><p><b>63.13</b></p></td><td class="cell"><p><b>63.68</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>two-phase(6ase/me2)</b></p><p><b>(only 5 classes)</b></p></td><td class="cell"><p><b>66.24</b></p></td><td class="cell"><p><b>64.54</b></p></td><td class="cell"><p><b>65.38</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>two-phase(6ase/me2) (5 classes+other class)</b></p></td><td class="cell"><p><b>68.51</b></p></td><td class="cell"><p><b>67.58</b></p></td><td class="cell"><p><b>68.04</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 9. F-score for different methods"></table><p>Table 10 shows the comparison with top-ranked systems in JNLPBA 2004 shared task. The top-ranked systems made use of external knowledge from gazetteers and abbreviation handling routines, which were reported to be ef­fective. Zhou et. al reported the usage of gazetteers and abbreviation handling improves the performance of the NER system by 4.8% in F-score [13]. Finkel et. al made use of a number of external resources, including gazetteers, web-querying, use of the surrounding abstract, abbreviation handling, and frequency counts from BNC corpus [4]. Settles utilized semantic domain knowledge of 17 kinds of lexicons [11]. Although the performance of our system is a bit lower than the best system, the results are very promising since most systems use external gazetteers, and abbreviation and conjunction/disjunction handling scheme. This suggests areas for further work.</p></subsection></section><section number="5" title="Conclusion and Discussion"><p>We presented a two-phase biomedical NE recognition model, term boundary detection and semantic labeling. We proposed two exponential models for each phase. That is, CRFs are used for term detection phase including Markov process and ME is used for semantic labeling. The benefit of dividing the whole process into two processes is that, by separating the processes with different characteris­tics, we can select separately the discriminative feature set for each subtask, and moreover measure effectiveness of models at each phase. Furthermore, we use the rule-based method as postprocessing to refine the result. The rules are ex­tracted from the GENIA corpus, which is represented by the deterministic FST. The rule-based approach is effective to correct errors by cascading structures of biomedical NEs.<page local="12" global="657"/> The experimental results are quite promising. The system achieved 71.19% F-score without Gazetteers or abbreviation handling process. The performance could be improved by utilizing lexical database and testing various classification models.</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">656</doubt><table caption="Table 10. Comparisons with other systems" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>M<b>ethod</b></p></td><td class="cell"><p><b>Recall Precision F-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline! (salientNP)</i></p></td><td class="cell"><p><b>66.21</b></p></td><td class="cell"><p><b>66.34</b></p></td><td class="cell"><p><b>66.27</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline2(all)</i></p></td><td class="cell"><p><b>68.51</b></p></td><td class="cell"><p><b>67.58</b></p></td><td class="cell"><p><b>68.04</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline2 + FST</i></p></td><td class="cell"><p><b>68.89</b></p></td><td class="cell"><p><b>68.53</b></p></td><td class="cell"><p><b>68.71</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baselineJl + </i><i>MEMarkov</i></p></td><td class="cell"><p><b>70.30</b></p></td><td class="cell"><p><b>67.65</b></p></td><td class="cell"><p><b>68.95</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline2 </i><i>+ MEMarkov </i><i>+ FST</i></p></td><td class="cell"><p><b>70.61</b></p></td><td class="cell"><p><b>68.40</b></p></td><td class="cell"><p><b>69.49</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline2 + CRF</i></p></td><td class="cell"><p><b>72.44</b></p></td><td class="cell"><p><b>68.77</b></p></td><td class="cell"><p><b>70.56</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><i>baseline2 + CRF + FST</i></p></td><td class="cell"><p><b>72.77</b></p></td><td class="cell"><p><b>69.68</b></p></td><td class="cell"><p><b>71.19</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>S<b>ystem</b></p></td><td class="cell"><p><b>Precision Recall F-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Zhou et. al (2004) Our system Finkel et. al (2004) Settles (2004)</b></p></td><td class="cell"><p><b>69.42        75.99 72.55 72.77        69.68 71.19 71.62        68.56 70.06 70.0          69.0 69.5</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><section title="Acknowledgements"><p>This work was supported by Korea Research Foundation Grant, KRF-2004-037-</p><doubt alpha="64.7" length="17" tooSmall="False" monospace="0.0">D00017.References</doubt><p><b>1. Thorten Brants. TnT A Statistical Part-of-Speech Tagger. In </b><i>Proceedings of the 6th Applied Natural Language Processing.; </i><b>2000.</b></p><p>2<b>. Stanley F. Chen and Ronald Rosenfeld. A Gaussian prior for smoothing maximum entropy models. </b><i>Technical Report CMUCS-99-108, Carnegie Mellon University.</i></p><p>3<b>. Nigel Collier, Chikashi Nobata and Jun-ichi Tsujii. Extracting the Names of Genes and Gene Products with a Hidden Markov Model. In </b><i>Proceedings of COLING 2000;</i> <b>201-207.</b><b></b></p><p>4<b>. Jenny Finkel, Shipra Dingare, and Huy Nguyen. Exploiting Context for Biomedical Entity Recognition From Syntax to thw Web. In </b><i>Proceedings of JNLPBA/BioNLP 2004; </i><b>88-91.</b></p><p><b>5. K. Fukuda, T. Tsunoda, A. Tamura, and T. Takagi. Toward information extrac­tion: identifying protein names from biological papers. In </b><i>Proceedins of the Pacific Symposium on Biocomputing 98; </i><b>707-718.</b></p><p>6<b>. Junichi Kazama, Takaki Makino, Yoshihiro Ohta and Junichi Tsujii. Tuning Sup­port Vector Machines for Biomedical Named Entity Recognition, </b><i>Proceedings ofthe ACL Workshop on Natural Language Processing in the Biomedical Domain </i><b>2002;</b> <b>1-8.</b><b></b></p><p>7<b>. Michael Krauthammer and Goran Nenadic. Term Identification in the Biomedical literature. Journal of Biomedical Informatics. 2004; 37(6):512-526.</b></p><p>8<b>. John Lafferty, Andrew McCallum, and Fernando Pereira. Conditional Random Fields: probabilistic models for segmenting and labeling sequence data. In </b><i>Proceed­ings of ICML-01; </i><b>282-289.</b></p><p><b>9. Ki-Joong Lee, Young-Sook Hwang, Seonho Kim, Hae-Chang Rim. Biomedical named entity recognition using two-phase model based on SVMs. </b><i>Journal of</i> <i>Biomedical Informatics </i><b>2004; 37(6):</b><b>436-447.</b></p><p>10<b>. Kyung-Mi Park, Seonho Kim, Ki-Joong Lee, Do-Gil Lee, and Hae-Chang Rim. Incorportating Lexical Knowledge into Biomedical NE Recognition. In </b><i>Proceedings ofNatural Language Processing in Biomedicine and its Applications Post-COLING</i> <i>Workshop </i><b>2004; 76-79.</b><b></b></p><p>1<b>1. Burr Settles. Biomedical Named Entity Recognition Using Conditional Random Fields andRichFeature Sets.In </b><i>Proceedings of JNLPBA/BioNLP </i><b>2004; 104-107.</b></p><p>1<b>2. Olivia Tuason, Lifeng Chen, Hongfang Liu, Judith A. Blake, Carol Friedman. Bi­ological Nomenclatures: A Source of Lexical Knowledge and Ambiguity. In </b><i>Pacific Symposium on Biocomputing </i><b>2004; 238-249.</b></p><p><b>13. GuoDong Zhou, Jie Zhang, Jian Su, Chew-Lim Tan. Exploring Deep Knowledge Resources in Biomedical Name Recognition. In </b><i>Proceedings of JNLPBA/BioNLP</i> <b>2004; 99-102.</b><b></b></p></section></body></article>