<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="53"/><title>The Ins and Outs of Dutch noun countability classification</title><author surname="Baldwin" givenname="Timothy"><org  name="Stanford University" country="USA" city="Stanford"/></author><author surname="Beek" givenname="Leonoor van der"><org  name="York University" country="Canada" city="North York"/></author></firstpageheader><frontmatter><p><b>The Ins and Outs of Dutch Noun Countability Classification</b></p><p><b>Timothy Baldwin Leonoor van der Beek</b></p><p>CSLI University of Groningen</p><p>Stanford University Pb 716, 9700 AS, Groningen</p><p>Stanford, CA 94305 USA The Netherlands</p><p>tbaldwin@csli.stanford.edu vdbeek@let.rug.nl</p></frontmatter><abstract>This paper presents a range of methods for classifying Dutch noun countability based on either Dutch or English data. The classification is founded on transla-tional equivalences and the corpus anal­ysis of linguistic features which corre­late with particular countability classes. We show that crosslingual classification on the basis of word-to-word or feature-to-feature mappings between English and Dutch performs at least as well as in-language classification based on gold-standard Dutch countability data. </abstract></header><body><section number="1" title="Introduction"><p>The performance of supervised learning methods is conditioned on the quality of annotation and also volume of training data (Hastie et al., 2001). This effect is felt particularly keenly in tasks of high feature dimensionality or low feature-class correlation. In many cases, high-quality data is not available in large quantities, but a large volume of lower-quality data can be accessed (Mitchell, 1999; Banko and Brill, 2001). Al­ternatively, high-quality data may exist for some parallel task which can be adapted to the task at hand through a lossy feature mapping. This strat­egy has been adopted successfully in NLP appli­cations such as part-of-speech tagging involving languages with a relative paucity of language re­sources or annotated data (Yarowsky et al., 2001; Cucerzan and Yarowsky, 2002).</p><p>This paper takes a supervised learning task and contrasts the use of a restricted volume of in-language training data with the use of a larger vol­ume of out-of-language training data adapted to the task through a lossy mapping. Our aim in this is to determine the most effective fast-track solu­tion when faced with a novel task in a given lan­guage for which high-quality annotated data ex­ists in a closely-related language.</p><p>We illustrate this issue by way of a type-level noun countability classification task in Dutch for which we have moderate amounts of high-quality annotated data in English and large amounts of medium-quality annotated data in Dutch (see §2.4). For English, previous research has shown that corpus evidence can be applied successfully to classify unannotated noun types for count-ability (Baldwin and Bond, 2003a; Baldwin and Bond, 2003b). We extend this research to Dutch and address the question of which of high-quality out-of-language English data and lower-quality in-language Dutch data produces the best Dutch countability classification results, realising that the feature mapping from English-to-Dutch in the first case will necessarily be lossy.</p><p>We treat <b>noun countability </b>as a lexical prop­erty that determines determiner co-occurrence, the ability to pluralise, and enumeration effects. Each Dutch noun type is classified as being count­able and/or uncountable, noting that different senses/usages of a given word can occur with different countabilities, cf. <i>I want a <u>rabbit</u> ¥^ Ik will een konijn </i>[countable] vs. <i>I would like some more <u>rabbit,</u> please ¥^ Izou graag nog wat konijn willen </i>[uncountable]. Knowledge of countabil-ity is important both for analysis and generation. In analysis it helps to constrain the set of pos­sible parses and their interpretation. In genera­tion, countability information determines whether a noun can be pluralised and what determiners it can combine with.</p><p>The assumption underlying the crosslingual countability classification task is that Dutch and English are sufficiently close linguistically that there is a strong correlation between noun count-ability in the two languages. Both languages distinguish countable, uncountable and plural only nouns.<footnote anchor="1"/> Although mismatches exist—e.g. <i>brain </i>[countable] vs. <i>hersenen </i>[plural only], <i>thun-</i> <i>derstorm </i>[countable] vs.<page local="2" global="54"/> <i>onweer </i>[uncountable]— many Dutch words are in the same countabil-ity class as their English equivalents (e.g. <i>car ¥^ auto </i>[countable], <i>food ¥^ eten </i>[uncountable] and <i>goods ¥^ goederen </i>[plural only]). One obvious ap­proach, therefore, is to simply map the countabil-ities of English nouns onto their Dutch counter­parts.</p><footnote label="1">A fourth class of bipartite nouns (e.g. scissors, trousers) is generally recognised for English, but has no Dutch corre­late.</footnote><p>A less direct approach to crosslingual count-ability transfer is to base classification on cor­pus occurrence with linguistic predictors of the different countability classes, in the manner of Baldwin and Bond (2003a). Linguistic features that are associated with the countability classes often have direct translations in the other lan­guage (e.g. syntactic number, co-occurrence with denumerators) or can be mapped onto an equiva­lent feature (e.g. the English <i>N1 of N2 </i>construc­tion and Dutch measure noun construction—see §2.2). In some cases however, the mapping is imperfect (e.g. <i>much </i>occurs only with uncount­able nouns, but the Dutch translation <i>veel </i>is also the translation of <i>many, </i>and occurs with both un­countable singular and countable plural nouns) or no equivalent exists in one of the languages (e.g. the occurrence of a plural noun as a modifier is a weak indicator of plural only in English, but not in Dutch).</p><p>The remainder of this paper is structured as follows. §2 describes the countability classes, the nature and extraction of the features used in the corpus-based method, the feature abstraction method and the gold-standard data. §3 outlines the various classifiers tested in this research. §4 presents and discusses the experimental results. The conclusions of the paper are given in §5.</p></section><section number="2" title="Preliminaries"><subsection number="2.1" title="Countability classes"><p>Dutch and English nouns are generally consid­ered to belong to one or more of three possi­ble countability classes: countable, uncountable and plural only. <b>Countable </b>nouns can be mod­ified by denumerators, prototypically numbers, and have a morphologically marked plural form: <i>one dog ^ een <u>hond</u>, two dogs ^ twee <u>honden</u>. </i><b>Uncountable </b>nouns cannot be modified by de-numerators, but can be modified by unspecific quantifiers such as <i>much ^ veel, </i>and do not show any number distinction (prototypically being sin­gular): <i>*one <u>rice</u> ¥^ *een rijst, some <u>rice</u> ¥^ een beetje rijst, *two <u>rices</u> ¥^ *twee rijsten. </i>This class includes many abstract nouns, material-denoting nouns, generics and deverbalised nouns. <b>Plu­ral only </b>nouns only have a plural form, such as <i>goods ^ goederen </i>and cannot be denumerated. The plural only class is considered to be a closed class in Dutch, and is thus ignored in the classi­fication experiments below.<footnote anchor="2"/> Note that countabil-ity distinctions are in fact not categorical (Allan, 1980): prototypical countable nouns can be used in uncountable contexts, forcing a 'substance' in­terpretation (the <b>universal grinder, </b>e.g. <i>there was <u>deer</u> all over the road ^ over de hele straat lag <u>hert</u>) </i>and uncountable nouns can in certain con­texts be denumerated, resulting in a 'type' inter­pretation (the <b>universal packager, </b>e.g. <i>this shop sells three different <u>wines</u> ¥^ deze winkel verkoopt drie verschillende wijnen). </i>However, nouns are generally considered to have a basic classification as countable and/or uncountable.</p></subsection><subsection number="2.2" title="Feature space"><p>The feature space used in this research is made up of <b>feature clusters, </b>each of which is condi­tioned on the occurrence of a <b>target noun </b>in a given construction. Feature clusters are either one-dimensional (describe a single multivariate feature) or two-dimensional (describe the interac­tion between two multivariate features), with each dimension describing a lexical or syntactic prop­erty of the construction in question. Below, we provide a basic description of the 9 feature clus­ters used in this research and their dimensional­ity (xlL=1-dimensional feature cluster with <i>x </i>unit features for language <b>L, </b>[xx^L = 2-dimensional feature cluster with <i>x x y </i>unit features for lan­guage <b>L). </b>For further details and predicted cor­relations between feature values and particular countability classes for English, the reader is re­ferred to Baldwin and Bond (2003a).</p><p><b>Head noun number:[2]</b>e ^~ [21D the  number of the target noun when it heads an NP</p><p><b>Subject-verb agreement</b>:[2x2lE ^ [2x2lD the number of the target noun in a subject posi­tion vs. number agreement on the governing verb</p><p><b>Coordinate noun number</b>:[2x2lE ^ [2x2lD the number of the target noun vs. the number of the head nouns of conjuncts <b>N</b>i <b><i>of </i>N2/measure noun constructions:</b><b></b></p><p>[iix2lE- [11x2lD the type of the N1 vs.</p><footnote label="2">But see van der Beek and Baldwin (2003) for classifica­tion results over the plural only class.</footnote><page local="3" global="55"/><p>the number of the target noun (N2) in an English N1 <i>of </i>N2 construction or Dutch measure noun construction. N1 types in­clude collective (a <i>group of people ¥^ een groep mensen), </i>unit (a <i>kilo of sugar ¥^ een kilo suiker) </i>and temporal (a <i>minute of silence ¥^ een minuut stilte).</i></p><p><b>Occurrence in PPs:[52x2]E ^ </b>[84x2lD the prepo­sition type and presence or absence of a de­terminer when the target noun occurs in <b>sin­gular </b>form in a PP.</p><p><b>Pronoun co-occurrence: [12x2]e ^~ </b>[7x2lD what personal, reflexive and possessive pronouns occur in the same sentence as singular and plural instances of the target noun.</p><p><b>Singular determiners: [10]e ^~ [10]d </b>what singular-selecting determiners (e.g. <i>much) </i>occur in NPs headed by the target noun in <b>singular </b>form.</p><p><b>Plural determiners:[12]e ^ [13]d </b>what plural-selecting determiners (e.g. <i>many) </i>occur in NPs headed by the target noun in <b>plural </b>form.</p><p><b>Number-neutral determiners</b>:[11x2lE <b>^ </b>[13x2lDwhat number-neutral determiners (e.g. <i>less) </i>occur in NPs headed by the target noun, and what is the number of the target noun for each.</p><p>The Dutch and English feature clusters repre­sent the same linguistic structures, even if the individual features are not direct translations of each other. The only exception is the N1 <i>of </i>N2/measure noun construction where markedly different constructions in the two languages ex­press the same concept (a quantity of something) and bring about the same restrictions with respect to countability.</p></subsection><subsection number="2.3" title="Feature extraction"><p>We use a variety of pre-processors to map the raw data onto the types of constructions targeted in the feature clusters, namely a POS tagger and a full-text chunker for both English and Dutch, and additionally a dependency parser for En­glish. For Dutch, POS tags, lemmata and chunk data were extracted from automatically generated, fully parsed <b>Alpino </b>output (Bouma et al., 2000). For English, we used a custom-built fnTBL-based tagger (Ngai and Florian, 2001) with the Penn tagset, morph (Minnen et al., 2001) as our lem-matiser, an fnTBL-based chunker which runs over the output of the tagger, and RASP (Briscoe and Carroll, 2002) as the dependency parser.</p><p>These data sets are then used independently to test the efficacy of the different systems at captur­ing features used in the classification process, or in tandem to consolidate the strengths of the in­dividual methods and reduce system-specific id­iosyncrasies in the feature values. When combin­ing Dutch and English in classification, we invari­ably combine like systems (e.g. Dutch POS data with English POS data).</p><p>The English data was extracted from the writ­ten component of the British National Corpus (90m words: Burnard (2000)), and the Dutch data from the newspaper component of the Twente Nieuws Corpus (20m words).<footnote anchor="3"/></p><p>After generating the different feature vectors for each noun based on the above configurations, we filtered out all nouns which did not occur at least 10 times in NP head position according to the output of all pre-processors. This resulted in 20,530 English nouns and 12,734 Dutch nouns.</p></subsection><subsection number="2.4" title="Gold standard data"><p>Information about English noun countability was obtained from two sources: <b>comlex </b>3.0 (Gr-ishman et al., 1998) and the common noun part of <b>alt-j/e</b>'s Japanese-to-English semantic trans­fer dictionary. These two resources were com­bined in two ways: (1) by taking the intersec­tion of positive and negative exemplars for each countability class (the <b>binary </b>datasets); and (2) by taking the union of all countabilities for a given word in the two resources and represent­ing it as a single multiclass (i.e. countable, un­countable or countable+uncountable: the <b>multiclass </b>dataset). In each case, the total number oftraining instances is around 6,000 words. To determine the quality of annotation, we hand-annotated 100 unseen nouns and measured the agreement<footnote anchor="4"/> with the gold-standard datasets. The agreement for the binary dataset was 92.4%, and that for the multi-class dataset was 89.8%.<footnote anchor="5"/></p><p>In Dutch, there are two electronic dictionaries with countability information: <b>celex </b>(Baayen et al., 1993) and the <b>Alpino </b>lexicon (Bouma et al., 2000).<page local="4" global="56"/> The latter includes the former as well as the Parole lexicon (no countability informa­tion), and has been manually modified and ex­tended. We thus used the <b>Alpino </b>data to gener­ate a total of three sets of training data for the monolingual Dutch classifiers in the same man­ner as for English: separate binary datasets for each of the countable and uncountable classes, and a combined multiclass-based dataset. The to­tal number of training instances in each case is around 14,500, over twice the size of the English datasets.</p><footnote label="3">http://wwwhome.cs.utwente.nl/~druid/ TwNC/TwNC-main.html</footnote><footnote label="4">I.e. the proportion of word-level countability class as­signments over which the two systems agreed.</footnote><footnote label="5">The disparity here is due to the fact that the binary dataset is constructed more conservatively and does not con­tain any words where there is disagreement in countability between comlex and alt-J/e.</footnote><p>In order to both evaluate the various classifiers and gauge the reliability of the <b>Alpino </b>count-ability judgements, we manually annotated 196 unseen Dutch nouns, basing judgements on ac­tual usage in the Twente Nieuws Corpus. The agreement in countability judgements between the <b>Alpino </b>lexicon and hand-annotated data is 81.1%. This is markedly lower than the agree­ment for the English datasets, and supports our claims about the relatively low quality of the Dutch <b>Alpino </b>data as compared to the English data.</p></subsection></section><section number="3" title="Classifier design"><p>We propose a variety of both monolingual (Dutch-to-Dutch = <b>NN </b>and English-to-English = <b>EE) </b>and crosslingual (English-to-Dutch = <b>EN) </b>unsupervised and supervised classifier architec­tures for the task of learning countability. We employ two basic classifier architectures: (1) a separate binary classifier for each countability class <b>(BIN), </b>and (2) a single multiclass classifier <b>(MULTI). </b>In all cases, our supervised classifiers are built using TiMBL version 4.2 (Daelemans et al., 2002), a memory-based classification system based on the -nearest neighbour algorithm.</p><subsection number="3.1" title="Monolingual classifiers"><p><b>Evidence-based classifiers: NNBIN(evidence,*)</b></p><p>In an attempt to derive a baseline for each countability class/pre-processor system combi­nation, we built a (binary) monolingual unsu-pervised classifier based on diagnostic evidence. For each target noun, the unsupervised classi­fier simply checks for the existence of diagnos­tic data in the output of each of the POS tag­ger and chunker for the given countability class <b>(NN(evidence,POS) </b>and <b>NN(evidence,chunk), </b>respectively). Diagnostic data takes the form of unit features which are uniquely associated with a given countability class, e.g. the determiner <i>a ^ een </i>co-occurring with a given (singular) noun is a strong indicator of that noun being countable. We perform basic system combination by posi­tively classifying any noun for which either ofthe two pre-processors produces diagnostic data for the given countability class <b>(NN(evidence,all)).</b></p><p><b>Distribution-based classifiers: NNBIN(featALL)</b></p><p>Despite our reservations about the quality of countability annotation in the <b>Alpino </b>lexicon, we implemented a conventional monolingual classi­fier based on the full feature set given above (§2.2). In this, we take each target noun in turn and compare its amalgamated value for each unit feature with: (a) the values for other target nouns, and (b) the value of other unit features within that same feature cluster (Baldwin and Bond, 2003b).</p><p>In the case ofa one-dimensional feature cluster, each unit feature <i>fs </i>for target noun <i>w </i>is translated into 3 separate feature values:</p><doubt alpha="63.6" length="11" tooSmall="False" monospace="0.0">freq(fs\w )</doubt><doubt alpha="39.3" length="28" tooSmall="False" monospace="0.0">corpfreq{fs, w)=     —-——(1)</doubt><doubt alpha="57.1" length="7" tooSmall="False" monospace="0.0">freq(*)</doubt><doubt alpha="33.3" length="33" tooSmall="False" monospace="0.0">wordfreq(fs, w)     =     —-——(2)</doubt><doubt alpha="55.6" length="9" tooSmall="False" monospace="0.0">freq (w )</doubt><doubt alpha="45.8" length="24" tooSmall="False" monospace="0.0">featfreq(fs,w)=—-——-}(3)</doubt><doubt alpha="57.1" length="14" tooSmall="False" monospace="0.0">2_^ifreq(fi\w)</doubt><p>where <i>freq</i><i>(*)</i><i> </i>is the frequency of all words in the corpus. That is, for each unit feature we capture the relative corpus frequency, frequency relative to the target word frequency, and frequency rel­ative to other features in the same feature clus­ter. Thus, for an n-valued one-dimensional fea­ture cluster, we generate 3n independent feature values.</p><p>In addition to mapping individual unit features onto triples, we introduce a triple for each feature cluster representing the sum over all member val­ues.</p><p>In the case of a two-dimensional feature ma­trix (e.g. subject-position noun number vs. verb number agreement), each unit feature <i>f</i><i>sj</i><i> </i>for tar­get noun <i>w </i>is translated into <i>corpfreq</i><i>(f</i><i>sj,</i><i> </i><i>w</i>), <i>wordfreq</i><i>(f</i><i>sj,</i><i> </i><i>w) </i>and <i>featfreq</i><i>(fs</i><i> </i><i>tt,</i><i> </i><i>w)</i><i> </i>as above, and 2 additional feature values:</p><p><i>freq(fs t\w)</i></p><doubt alpha="48.4" length="31" tooSmall="False" monospace="0.0">featdtmfreq1(fs,t,w)=^---;—-(4)</doubt><doubt alpha="66.7" length="12" tooSmall="False" monospace="0.0">freq(fi,t\w)</doubt><doubt alpha="48.4" length="31" tooSmall="False" monospace="0.0">featdtmfreq2(fs,t,w)=^-;-;—-(5)</doubt><doubt alpha="66.7" length="12" tooSmall="False" monospace="0.0">freq(fs,j\w)</doubt><p>which represent the <i>featfreq </i>values calculated along each of the two feature dimensions. As for one-dimensional feature clusters, we intro­duce amalgamated features for each row <b>(f</b><i>* * </i>) and column <i>(f*t j</i>) of the feature matrix, and describe each in the form of 3 values.<page local="5" global="57"/> For further details, see the description of the monolingual English task in Baldwin and Bond (2003a). This abstrac­tion generates a total of 1,664 individual feature values for Dutch.</p><p>We learned individual countable and uncount­able classifiers from the binary <b>Alpino </b>data, av­eraging the feature values across those from the tagger and chunker in each case.<footnote anchor="6"/> <b>3.</b><b>2   Crosslingual classifiers Translation-based classifier: ENBIN(translate)</b></p><p>Translation-based classification applies the ob­servation that Dutch nouns often take the same countability as their English translation equiv­alents. First, we derive English countabilities from the binary gold-standard datasets supple­mented with data from the output of a mono­lingual supervised English countability classi­fier <b>(EEBIN(featALL)</b>—see below). We then ex­tract translation pairs from a bilingual dictionary (English-Dutch freedict version 1.1-1, containing 15,426 Dutch entries) and for each countability class, check for the existence of an English trans­lation in the given countability class. If none of the English translations are classified as belong­ing to that countability class, we negatively clas­sify the Dutch noun. In the event that no transla­tion data exists for the Dutch noun or no count-ability data exists for the English translation(s), we classify the Dutch noun countability as un­known. Note that we map English plural only and bipartite nouns onto the Dutch uncountable class.</p><p><b>Transliteration-based classifier: EN</b>bin<b>(transliterate)</b></p><p>Transliteration-based classification also applies the observation that countability is frequently pre­served under translation from English to Dutch, but does so in a resource-free manner. It takes a Dutch noun and simply determines if a countability-annotated word of the same spelling exists in English, and if so, transfers the count-ability directly across to Dutch. In all other re­spects, we implement the method identically to translation-based classification.</p><footnote label="6">We additionally built separate classifiers based on the outputs of the individual pre-processors, and also based on the multiclass data, but found their performance to be marginally inferior to that of NN B iN (feat A LL).</footnote><p><b>Cluster-to-cluster classifier: ENBIN(cluster)</b></p><p>As observed above (§2.2), there is a strong correlation between the feature clusters used for Dutch and English. For example, co-occurrence with plural determiners is a strong indicator that the given noun is countable in both English and Dutch. At the same time, there is generally low correlation between individual unit features. For example, the English plural determiner <i>many </i>has no direct Dutch equivalent, and conversely, the Dutch plural determiner <i>sommige </i>has no di­rect English equivalent. The most straightfor­ward way of aligning feature clusters, therefore, is through the (three) amalgamated totals for each one-dimensional feature cluster and some sub­set of the column and row totals for each two-dimensional feature cluster (e.g. for the PP fea­ture, we align the totals for the singular and plu­ral features but not the totals for each individual preposition independent of number). All values for the individual unit features are then ignored. In this way, it is possible to align 88 feature val­ues, based on the output of the English and Dutch POS taggers.<footnote anchor="7"/> Note that as part of the feature alignment, we take the negative log of all corpus frequency <i>(corpfreq) </i>values in an attempt to re­duce the effects of differing corpus sizes in En­glish and Dutch.</p><p><b>Feature-to-feature classifiers: EN*(feat*)</b></p><p>While we stated above that there is generally low correlation between individual unit features in English and Dutch, some unit features are highly correlated crosslingually. One example is the English singular determiner <i>a </i>which corre­lates highly with the Dutch een. Here, we can thus simply match the feature values onto one another directly. In other cases, a many-to-many mapping exists between proper subsets of a given feature cluster (e.g. the English determiner pair <i>each </i>and <i>every </i>correlates highly with the Dutch determiner pair <i>ieder </i>and <i>elk), </i>and alignment takes the form of feature value amalgamation in each language by averaging over the unit values and aligning the amalgamated values. A total of 466 unit fea­ture values are amalgamated into 351 feature val­ues, which are then combined with the 88 aligned total values from cluster-to-cluster classification for a total of 439 feature values.<page local="6" global="58"/> As for cluster-to-cluster classification, we evaluate feature-to-feature classification over the output of the En­glish and Dutch POS taggers.</p><footnote label="7">All crosslingual feature-based methods were tested over the output of the POS taggers, the chunkers and the com­bined outputs of the three English and two Dutch pre­processors. Overall, there was very little separating the re­sults, and the simple POS tagger generally produced the most consistent results.</footnote><p>We implemented a total of 5 feature-to-feature classifiers: (1) <b>ENBIN(featALL) </b>makes use of all aligned features in the form of separate binary classifiers; (2) <b>ENMULTI(featALL) </b>similarly uses all aligned features, but in a multiclass classi­fier architecture; (3) <b>ENBIN(featDET) </b>is based on only aligned determiner features, plus the aligned cluster totals; (4) <b>ENBIN(featPREP) </b>is based on only aligned preposition features, plus the aligned cluster totals; and (5) <b>ENBIN(featPRON) </b>is based on only aligned pronoun features, plus the aligned cluster totals.<footnote anchor="8"/></p></subsection><subsection number="3.3" title="System combination"><p>System combination takes the outputs of het­erogeneous classifiers and makes a consolidated classification based upon them. It has been shown to be effective in tasks ranging from word sense disambiguation to tagging in consolidat­ing the performance of component systems (Klein et al., 2002; van Halteren et al., 2001). In our case, we first take the outputs of all unsu-pervised (i.e. evidence-based) and crosslingual classifiers—a total of 12 classifiers—for each countability class <b>(EN</b>bin<b>(combined)). </b>We test the effects of system classification by way of 10-fold cross-validation over the 196 annotated Dutch nouns. This provides an estimate of the classification performance we could expect over unannotated Dutch noun data using the 196 an­notated nouns as our sole source of annotated Dutch data. We also test combining the outputs of the 12 unsupervised and crosslingual classifiers with that of the <b>Alpino</b>-trained Dutch classifier <b>(E/NN</b>bin<b>(combined)).</b></p></subsection></section><section number="4" title="Results and Discussion"><p>Classifier performance is rated according to clas­sification accuracy (the proportion of instances classified correctly: <b>Acc), </b>precision <b>(P), </b>recall <b>(R) </b>and F-score^=i <b>(F).</b></p><p>The baseline for each countability class is a majority-class binary classifier which simply classifies all instances according to the most commonly-attested class in the given dataset. Ir­respective of the majority class, we calculate the recall and F-score based on a positive-class classi­fier, i.e. a classifier which naively classifies each instance as belonging to the given class; in the case that the positive class is not the majority class (as occurs for uncountable nouns), the recall and F-score are given in parentheses.</p><footnote label="8">Results for the multiclass classifier over feature subsets were found to be markedly worse than for binary classifiers.</footnote><p>We also provide an upper bound estimate of precision, recall and F-score based on a monolin­gual English countability classification task, with classifiers designed similarly to the monolingual Dutch classifiers <b>(EEBIN(featALL)). </b>In the case of English, the total number of feature values is 3,852, based on the concatenation of feature val­ues from each of a POS tagger, chunker and de­pendency parser (Baldwin and Bond, 2003a). Our reason for choosing this as an upper bound is that it is based on moderate-volume, relatively noise-free training data and full feature correlation. The classifier results are presented in Tables 1 and 2, broken down into the countable and un­countable classes.<page local="7" global="59"/> In each case, the best sin­gle value for each of evaluation metrics (other than the baseline and upper bound) is presented in <b>boldface.</b></p><table caption="Table 2: Results for uncountable nouns" class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b><i>Method</i></b></p></td><td class="cell"><p><b><i>Acc</i></b></p></td><td class="cell"><p><b><i>P</i></b></p></td><td class="cell"><p><b><i>R</i></b></p></td><td class="cell"><p><b><i>F</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(majority)</p></td><td class="cell"><p>.847</p></td><td class="cell"><p>.847</p></td><td class="cell"><p>1.000</p></td><td class="cell"><p>.917</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,all)</p></td><td class="cell"><p>.551</p></td><td class="cell"><p>.964</p></td><td class="cell"><p>.488</p></td><td class="cell"><p>.648</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,chunk)</p></td><td class="cell"><p>.510</p></td><td class="cell"><p>.973</p></td><td class="cell"><p>.434</p></td><td class="cell"><p>.600</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,POS)</p></td><td class="cell"><p>.474</p></td><td class="cell"><p>.970</p></td><td class="cell"><p>.392</p></td><td class="cell"><p>.558</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBIN (translate)</p></td><td class="cell"><p>.948</p></td><td class="cell"><p>.948</p></td><td class="cell"><p>.331</p></td><td class="cell"><p>.491</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBIN (transliterate)</p></td><td class="cell"><p><b>1.000</b></p></td><td class="cell"><p><b>1.000</b></p></td><td class="cell"><p>.151</p></td><td class="cell"><p>.262</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBIN (cluster)</p></td><td class="cell"><p>.806</p></td><td class="cell"><p>.957</p></td><td class="cell"><p>.807</p></td><td class="cell"><p>.876</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENMULTi(cluster)</p></td><td class="cell"><p>(.704)</p></td><td class="cell"><p>.959</p></td><td class="cell"><p>.837</p></td><td class="cell"><p>.894</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featALL)</p></td><td class="cell"><p>.750</p></td><td class="cell"><p>.983</p></td><td class="cell"><p>.717</p></td><td class="cell"><p>.829</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENmulti (feat all)</p></td><td class="cell"><p>(.704)</p></td><td class="cell"><p>.959</p></td><td class="cell"><p>.855</p></td><td class="cell"><p>.904</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featoET)</p></td><td class="cell"><p>.719</p></td><td class="cell"><p>.966</p></td><td class="cell"><p>.693</p></td><td class="cell"><p>.807</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featpREp)</p></td><td class="cell"><p>.755</p></td><td class="cell"><p>.968</p></td><td class="cell"><p>.735</p></td><td class="cell"><p>.836</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featpRON)</p></td><td class="cell"><p>.735</p></td><td class="cell"><p>.952</p></td><td class="cell"><p>.723</p></td><td class="cell"><p>.822</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN (combined)</p></td><td class="cell"><p>.873</p></td><td class="cell"><p>.944</p></td><td class="cell"><p>.904</p></td><td class="cell"><p>.923</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNbin (feat all )</p></td><td class="cell"><p>.867</p></td><td class="cell"><p>.961</p></td><td class="cell"><p>.880</p></td><td class="cell"><p>.918</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>E/NNBiN(combined)</p></td><td class="cell"><p>.903</p></td><td class="cell"><p>.947</p></td><td class="cell"><p><b>.940</b></p></td><td class="cell"><p><b>.943</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EEBiN(featALL)</p></td><td class="cell"><p>—</p></td><td class="cell"><p>.948</p></td><td class="cell"><p>.972</p></td><td class="cell"><p>.960</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 1: Results for countable nouns</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b><i>Method</i></b></p></td><td class="cell"><p><b><i>Acc</i></b></p></td><td class="cell"><p><b><i>P</i></b></p></td><td class="cell"><p><b><i>R</i></b></p></td><td class="cell"><p><b><i>F</i></b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(majority)</p></td><td class="cell"><p>.638</p></td><td class="cell"><p>.362 (1.000) (.532)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,all)</p></td><td class="cell"><p>.515</p></td><td class="cell"><p>.423</p></td><td class="cell"><p><b>.930</b></p></td><td class="cell"><p>.581</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,chunk)</p></td><td class="cell"><p>.505</p></td><td class="cell"><p>.414</p></td><td class="cell"><p>.887</p></td><td class="cell"><p>.565</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(evidence,POS)</p></td><td class="cell"><p>.628</p></td><td class="cell"><p>.490</p></td><td class="cell"><p>.718</p></td><td class="cell"><p>.583</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(translate)</p></td><td class="cell"><p>.583</p></td><td class="cell"><p>.583</p></td><td class="cell"><p>.099</p></td><td class="cell"><p>.169</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(transliterate)</p></td><td class="cell"><p><b>.966</b></p></td><td class="cell"><p><b>1.000</b></p></td><td class="cell"><p>.056</p></td><td class="cell"><p>.107</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(cluster)</p></td><td class="cell"><p>.740</p></td><td class="cell"><p>.692</p></td><td class="cell"><p>.507</p></td><td class="cell"><p>.585</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENMULTi(cluster)</p></td><td class="cell"><p>(.704)</p></td><td class="cell"><p>.750</p></td><td class="cell"><p>.592</p></td><td class="cell"><p>.661</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featALL)</p></td><td class="cell"><p>.699</p></td><td class="cell"><p>.750</p></td><td class="cell"><p>.254</p></td><td class="cell"><p>.379</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENMULTi(featALL)</p></td><td class="cell"><p>(.704)</p></td><td class="cell"><p>.822</p></td><td class="cell"><p>.521</p></td><td class="cell"><p>.638</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featoET)</p></td><td class="cell"><p>.801</p></td><td class="cell"><p>.758</p></td><td class="cell"><p>.662</p></td><td class="cell"><p><b>.707</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featpREp)</p></td><td class="cell"><p>.776</p></td><td class="cell"><p>.755</p></td><td class="cell"><p>.563</p></td><td class="cell"><p>.645</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(featpRoN)</p></td><td class="cell"><p>.689</p></td><td class="cell"><p>.708</p></td><td class="cell"><p>.239</p></td><td class="cell"><p>.358</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ENBiN(combined)</p></td><td class="cell"><p>.791</p></td><td class="cell"><p>.776</p></td><td class="cell"><p>.593</p></td><td class="cell"><p>.672</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NNBiN(featALL)</p></td><td class="cell"><p>.770</p></td><td class="cell"><p>.783</p></td><td class="cell"><p>.507</p></td><td class="cell"><p>.615</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>E/NN(combined)</p></td><td class="cell"><p>.812</p></td><td class="cell"><p>.819</p></td><td class="cell"><p>.609</p></td><td class="cell"><p>.699</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>EEBiN(featALL)</p></td><td class="cell"><p>—</p></td><td class="cell"><p>.884</p></td><td class="cell"><p>.907</p></td><td class="cell"><p>.895</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>The first thing to notice is how much bet­ter the classifiers perform for countable than un­countable nouns. This is due to two factors: the relative occurrence of members of the two classes (as reflected in the majority class classifi­cation accuracies), and the relative volume of fea­tures correlated with each class. The relatively high baseline accuracy and F-score for countable nouns (.847 and .917) surpassed the performance of all classifiers other than the translation-based, transliteration-based, combined and monolingual classifiers. For uncountable nouns, on the other hand, appreciable gains over the baseline were observed for many of the systems. Results for countable nouns were relatively close to the upper bound results for the English monolingual classi­fier, whereas results for uncountable nouns were less competitive.</p><p>We have made the claim that, due to the lack of reliable training data in Dutch, crosslingual clas­sification using English data is a viable option. This is borne out by the finding that the com­bined crosslingual classifier (ENBIN(combined)) consistently outperforms the monolingual Dutch classifier in F-score, with the discrepancy be­ing particularly noticeable for uncountable nouns. This finding is particularly striking given that the volume of Dutch training data is more than twice the volume of English data. Additional support comes from the analysis of the agree­ment between the system outputs the 196 hand­annotated nouns, recalling from §2.4 that the benchmark agreement for the <b>Alpino </b>data is 81.1%. The agreement for NNBIN(featALL) is 82.1%, that for ENbin(combined) is 83.2%, and that for E/NNbin(combined) is a respectable 85.7%. That is, all three methods produce count-ability judgements that are more parsimonious with actual corpus occurrence than the <b>Alpino </b>data, and the combined crosslingual classifier (ENbin(combined)) is superior to the monolin­gual classifier (NNBIN(featALL)). Having said this, the combined crosslingual/monolingual classifier (E/NNbin(combined)) outperforms both the com­bined crosslingual classifier and the monolingual classifier, in which sense the <b>Alpino </b>data has some empirical utility. That is, we have shown that high-quality out-of-language English countability data is a stronger predictor of Dutch count-ability than medium-quality in-language Dutch countability data, but at the same time that the two are complementary.</p><p>There is very little separating the cluster-to-cluster and feature-to-feature classifiers. Given the high overhead in hand-aligning features in feature-to-feature classification, cluster-to-cluster classification would appear a low-cost, high­performance solution to the crosslingual count-ability task. Within the feature-to-feature clas­sifiers, the results for the feature subsets are in­triguing. We would expect that the determiner features should provide greater leverage than ei­ther the pronoun or preposition features, and this is indeed the case for uncountable nouns, where the determiner feature-based classifier returns the best F-score of all the classifiers. For countable nouns, however, the determiner features perform the worst of the three. Further research is required to determine the cause of this effect.</p><p>The results for the translation- and transliteration-based classifiers require qual­ification. Unlike the other classifiers, we do not get 100% coverage, as classification is possible only in the case that we have an English translation or transliteration with countability information. Strictly speaking, this dimin­ished coverage should not be reflected in any of our evaluation metrics. In order to bring out this effect in Tables 1 and 2, we chose to base recall on the ratio of correctly-classified test exemplars to the number of positive-class exemplars, irrespective of whether the method is able to classify them. The F-score is thus proportionately low. If we were to base recall on the number of <i>classified </i>positive-class exemplars, the recall for the translation-based classifiers would become a perfect 1.000 ( f<footnote anchor="5"/>) and 1.000 (<footnote anchor="7"/> ) for the countable and uncountable classes, respectively, and the corresponding numbers for the transliteration-based classifiers would be 1.000 (ff ) and 0.800 (f). That is, assuming we have English translation(s) for a Dutch noun or an English word of the same spelling, we get a very accurate estimate of the Dutch countability from the English countability data.</p><p>Finally, it is important to realise that these re­sults are based on a limited test dataset (196 nouns) and that fuller evaluation is required to validate our findings. Also, our method relies cru­cially on the assumption that English and Dutch are closely-related languages, and its scalability to alternate language pairs remains to be deter­mined.<page local="8" global="60"/></p></section><section number="5" title="Conclusion"><p>We have presented several methods for classify­ing Dutch nouns as countable and/or uncountable on the basis of Dutch and English data. The clas­sifiers depend on translation/transliteration data or linguistic features that were extracted from unannotated corpora. We compared a range of crosslingual English-to-Dutch classifiers with a monolingual Dutch-to-Dutch classifier, and found that the crosslingual classifiers outperformed the monolingual classifier to varying degrees. Based on this, we suggest that the optimal fast-track so­lution to Dutch countability classification is to use English data.</p><p>In future research, we are interested in the possibility of co-training via translation- and transliteration-based classification, as this seems to provide a means for automatically generating high-quality Dutch countability data to learn a monolingual classifier from.</p></section><section title="Acknowledgements"><p>The first author was supported by the National Science Foun­dation under Grant No. BCS-0094638 and also the Research Collaboration between NTT Communication Science Labo­ratories, Nippon Telegraph and Telephone Corporation and CSLI, Stanford University. The second author was sup­ported by the NWO Council for the Humanities, the Grad­uate School for Behavioral and Cognitive Neurosciences (BCN) and the Center for Language and Cognition Gronin­gen (CLCG). The research was carried out within the frame­work of the PIONIER Project <i>Algorithms for Linguistic Pro­cessing, </i>which is funded by NWO (Dutch Organization for Scientific Research) and the University of Groningen.</p><p>We would like to thank Francis Bond, Ann Copestake, Dan Flickinger, Gertjan van Noord, Ivan Sag and the anony­mous reviewers for their valuable input on this research, and John Carroll for allowing us to use RASP</p></section><references><p>Keith Allan. 1980. Nouns and countability. <i>Language, </i>56(3):541-67.</p><p>Harald R. Baayen, Richard Piepenbrock, and Hedderik van Rijn. 1993. <i>The CELEX Lexical Database (CD-ROM)</i>. Linguistic Data Consortium, University ofPennsylvania,</p><p>Philadelphia, PA.</p><p>Timothy Baldwin and Francis Bond. 2003a. Learning the countability ofEnglish nouns from corpus data. In <i>Proc. of the 41st Annual Meeting of the ACL, </i>pages 463-70, Sapporo, Japan.</p><p>Timothy Baldwin and Francis Bond. 2003b. A plethora of methods for learning English countability. In <i>Proc. ofthe 2003 Conference on Empirical Methods in Natural Lan­guage Processing(EMNLP2003), </i>pages 73-80, Sapporo, Japan.</p><p>Michele Banko and Eric Brill. 2001. Scaling to very very large corpora for natural language disambiguation. In <i>Proc. of the 39th Annual Meeting of the ACL and 10th</i> <i>Conference ofthe EACL (ACL-EACL 2001), </i>Toulouse,</p><p>France.</p><p>Gosse Bouma, Gertjan van Noord, and Rob Malouf. 2000. Alpino: Wide coverage computational analysis ofDutch. In <i>Computational Linguistics in the Netherlands (CLIN</i> <i>2000).</i><i></i></p><p>Ted Briscoe and John Carroll. 2002. Robust accurate statis­tical annotation ofgeneral text. In <i>Proc. ofthe 3rdInter-national Conference on Language Resources andEvalua-tion (LREC 2002), </i>pages 1499-1504, Las Palmas, Canary Islands.</p><p>Lou Burnard. 2000. <i>User Reference Guide for the British National Corpus. </i>Technical report, Oxford University Computing Services.</p><p>Silviu Cucerzan and David Yarowsky. 2002. Bootstrapping a multilingual part-of-speech tagger in one person-day. In <i>Proc. of the 6th Conference on Natural Language Learn­ing (CoNLL-2002), </i>Taipei, Taiwan.</p><p>Walter Daelemans, Jakub Zavrel, Ko van der Sloot, and An-tal van den Bosch. 2002. TiMBL: Tilburg memory based learner, version 4.2, reference guide. ILK technical report 02-01.</p><p>Ralph Grishman, Catherine Macleod, and Adam Myers, 1998. <i>COMLEX Syntax Reference Manual. </i>Pro­teus Project, NYU. (http://nlp.cs.nyu.edu/ comlex/refman.ps).</p><p>Hans van Halteren, Jakub Zavrel, and Walter Daelemans. 2001. Improving accuracy in word class tagging through combination of machine learning systems. <i>Computa­tional Linguistics, </i>27(2):199-230.</p><p>Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. <i>Elements of Statistical Learning. </i>Springer-Verlag,</p><p>New York, USA.</p><p>Dan Klein, Kristina Toutanova, H. Tolga Ilhan, Sepandar D. Kamvar, and Christopher D. Manning. 2002. Combining heterogeneous classifiers for word-sense disambiguation. In <i>Proc. ofthe ACL-02 Workshop on Word Sense Disam­biguation: Recent Successes and Future Directions, </i>Pitts­burgh, USA.</p><p>Guido Minnen, John Carroll, and Darren Pearce. 2001. Ap­plied morphological processing of English. <i>Natural Lan­guage Engineering, </i>7(3):207-23.</p><p>Tom M. Mitchell. 1999. The role ofunlabeled data in super­vised learning. In <i>Proc. ofthe Sixth International Collo­quium on Cognitive Science, </i>San Sebastian, Spain.</p><p>Grace Ngai and Radu Florian. 2001. Transformation-based learning in the fast lane. In <i>Proc. ofthe 2nd Annual Meeting ofthe North American Chapter ofAssociation for Computational Linguistics (NAACL2001), </i>pages 40­7, Pittsburgh, USA.</p><p>Leonoor van der Beek and Timothy Baldwin. 2003. Crosslingual countability classification: English meets Dutch. <i>LinGO Working Paper No. 2003-03.</i></p><p>David Yarowsky, Grace Ngai, and R Wicentowski. 2001. Inducing multilingual text analysis tools via robust pro­jection across aligned corpora. In <i>Proc. ofHuman Lan­guage Technology (HLT) 2001, </i>pages 161-8, San Diego,</p><p>USA.</p></references></body></article>