<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="1151"/><title>Word-Based Dialect Identification with Georeferenced Rules</title><pubinfo>Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,pages 1151-1161, MIT, Massachusetts, USA, 9-11 October 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Scherrer" givenname="Yves"><org  name="Columbia University" country="USA" city="New York"/></author><author surname="Rambow" givenname="Owen"><org  name="Columbia University" country="USA" city="New York"/></author></firstpageheader><frontmatter><p><b>Word-based dialect identification with georeferenced rules</b></p><p><b>Yves Scherrer Owen Rambow</b></p><p>LATL CCLS Université de Genève Columbia University</p><p>Genève, Switzerland New York, USA</p><p>yves.scherrer@unige.ch rambow@ccls.columbia.edu</p></frontmatter><abstract>We present a novel approach for (written) di­alect identification based on the discrimina­tive potential of entire words. We generate Swiss German dialect words from a Standard German lexicon with the help of hand-crafted phonetic/graphemic rules that are associated with occurrence maps extracted from a linguis­tic atlas created through extensive empirical fieldwork. In comparison with a character-n-gram approach to dialect identification, our model is more robust to individual spelling dif­ferences, which are frequently encountered in non-standardized dialect writing. Moreover, it covers the whole Swiss German dialect contin­uum, which trained models struggle to achieve due to sparsity of training data. </abstract></header><body><section number="1" title="Introduction"><p>Dialect identification (dialect ID) can be viewed as an instance of language identification (language ID) where the different languages are very closely re­lated. Written language ID has been a popular re­search object in the last few decades, and relatively simple algorithms have proved to be very successful. The central question of language ID is the following: given a segment of text, which one of a predefined set of languages is this segment written in? Language identification is thus a classification problem.</p><p>Dialect identification comes in two flavors: spoken dialect ID and written dialect ID. These two tasks are rather different. Spoken dialect ID relies on speech recognition techniques which may not cope well with dialectal diversity. However, the acoustic signal is also available as input. Written dialect ID has to deal with non-standardized spellings that may occult real dialectal differences. Moreover, some phonetic dis­tinctions cannot be expressed in orthographic writing systems and limit the input cues in comparison with spoken dialect ID.</p><p>This paper deals with written dialect ID, applied to the Swiss German dialect area. An important aspect of our model is its conception of the dialect area as a continuum without clear-cut borders. Our dialect ID model follows a bag-of-words approach based on the assumption that every dialectal word form is defined by a probability with which it may occur in each geographic area. By combining the cues of all words of a sentence, it should be possible to obtain a fairly reliable geographic localization of that sentence.</p><p>The main challenge is to create a lexicon of dialect word forms and their associated probability maps. We start with a Standard German word list and use a set of phonetic, morphological and lexical rules to obtain the Swiss German forms. These rules are manually extracted from a linguistic atlas. This lin­guistic atlas of Swiss German dialects is the result of decades-long empirical fieldwork.</p><p>This paper is organized as follows. We start with an overview of relevant research (Section 2) and present the characteristics of the Swiss German di­alect area (Section 3). Section 4 deals with the im­plementation of word transformation rules and the corresponding extraction of probability maps from the linguistic atlas of German-speaking Switzerland. We present our dialect ID model in Section 5 and discuss its performance in Section 6 by relating it to a baseline n-gram model.</p><page local="2" global="1152"/></section><section number="2" title="Related work"><p>Various language identification methods have been proposed in the last three decades. Hughes et al. (2006) and Rehurek and Kolkus (2009) provide re­cent overviews of different approaches. One of the simplest and most popular approaches is based on character n-gram sequences (Cavnar and Tren-kle, 1994). For each language, a character n-gram language model is learned, and test segments are scored by all available language models and labeled with the best scoring language model. Related ap­proaches involve more sophisticated learning tech­niques (feature-based models, SVM and other kernel­based methods).</p><p>A completely different approach relies on the iden­tification of entire high-frequency words in the test segment (Ingle, 1980). Other models have proposed to use morpho-syntactic information.</p><p>Dialect ID has usually been studied from a speech processing point of view. For instance, Biadsy et al. (2009) classify speech material from four Arabic dialects plus Modern Standard Arabic. They first run a phone recognizer on the speech input and use the resulting transcription to build a trigram language model. Classification is done by minimizing the per­plexity of the trigram models on the test segment.</p><p>An original approach to the identification of Swiss German dialects has been taken by the <i>Chochichästli-Orakel.<footnote anchor="1"/> </i>By specifying the pronunciation of ten pre­defined words, the web site creates a probability map that shows the likelihood of these pronunciations in the Swiss German dialect area. Our model is heavily inspired by this work, but extends the set of cues to the entire lexicon.</p><p>As mentioned, the ID model is based on a large Swiss German lexicon. Its derivation from a Standard German lexicon can be viewed as a case of lexicon induction. Lexicon induction methods for closely related languages using phonetic similarity have been proposed by Mann and Yarowsky (2001) and Schafer and Yarowsky (2002), and applied to Swiss German data by Scherrer (2007).</p><p>The extraction of digital data from hand-drawn di-alectological maps is a time-consuming task. There­fore, the data should be made available for differ­ent uses. Our Swiss German raw data is accessible on an interactive web page (Scherrer, 2010), and we have proposed ideas for reusing this data for machine translation and dialect parsing (Scherrer and Rambow, 2010). An overview of digital dialec-tological maps for other languages is available on http://www.ericwheeler.ca/atlaslist.</p><footnote label="1">http://dialects.from.ch</footnote></section><section number="3" title="Swiss German dialects"><p>The German-speaking area of Switzerland encom­passes the Northeastern two thirds of the Swiss ter­ritory, and about two thirds of the Swiss population define (any variety of) German as their first language.</p><p>In German-speaking Switzerland, dialects are used in speech, while Standard German is used nearly ex­clusively in written contexts (diglossia). It follows that all (adult) Swiss Germans are bidialectal: they master their local dialect and Standard German. In addition, they usually have no difficulties understand­ing Swiss German dialects other than their own.</p><p>Despite the preference for spoken dialect use, writ­ten dialect data has been produced in the form of dialect literature and transcriptions of speech record­ings made for scientific purposes. More recently, written dialect has been used in electronic media like blogs, SMS, e-mail and chatrooms. The Alemannic Wikipedia contains about 6000 articles, among which many are written in a Swiss German dialect.<footnote anchor="2"/> How­ever, all this data is very heterogeneous in terms of the dialects used, spelling conventions and genre.</p></section><section number="4" title="Georeferenced word transformation rules"><p>The key component of the proposed dialect ID model is an automatically generated list of Swiss German word forms, each of which is associated with a map that specifies its likelihood of occurrence over German-speaking Switzerland. This word list is gen­erated with the help of a set of transformation rules, taking a list of Standard German words as a start­ing point. In this section, we present the different types of rules and how they can be extracted from a dialectological atlas.</p><footnote label="2">http://als.wikipedia.org ; besides Swiss German, the Alemannic dialect group encompasses Alsatian, South-West Ger­man Alemannic and Vorarlberg dialects of Austria.</footnote><page local="3" global="1153"/><subsection number="4.1" title="Orthography"><p>Our system generates written dialect words according to the Dieth spelling conventions without diacritics (Dieth, 1986).<footnote anchor="3"/> These are characterized by a transpar­ent grapheme-phone correspondence and are widely used by dialect writers. However, they are by no means enforced or even taught.</p><p>This lack of standardization is problematic for di­alect ID. We have noted two major types of deviations from the Dieth spelling conventions in our data. First, Standard German orthography may unduly influence dialect spelling. For example, <i>spiele </i>is modelled af­ter Standard German <i>spielen </i>'to play', although the vowel is a short monophthong in Swiss German and should thus be written <i>spile (ie </i>represents a diph­thong in Dieth spelling). Second, dialect writers do not always distinguish short and long vowels, while the Dieth conventions always use letter doubling to indicate vowel lengthening. Future work will incor­porate these fluctuations directly into the dialect ID model.</p><p>Because of our focus on written dialect, the follow­ing discussion will be based on written representa­tions, but IPA equivalents are added for convenience.</p></subsection><subsection number="4.2" title="Phonetic rules"><p>Our work is based on the assumption that many words show predictable phonetic differences between Stan­dard German and the different Swiss German dialects. Hence, in many cases, it is not necessary to explicitly model word-to-word correspondences, but a set of phonetic rules suffices to correctly transform words.</p><p>For example, the word-final sequence nd [nd] (as in Standard German <i>Hund </i>'dog'<footnote anchor="4"/>) is maintained in most Swiss German dialects. However, it has to be transformed to <i>ng </i>[g] in Berne dialect, to nn [n] in Fribourg dialect, and to nt [nt] in Valais and Uri dialects.</p><p>This phenomenon is captured in our system by four transformation rules nd — nd, nd — ng, nd — nn and nd — nt. Each rule is <i>georeferenced, </i>i.e. linked to a probability map that specifies its validity in every geographic point. These four <i>rules </i>capture one single linguistic <i>phenomenon: </i>their left-hand side is the same, and they are geographically complementary.</p><footnote label="3">Of course, these spelling conventions make use of umlauts like in Standard German. There is another variant of the Di­eth conventions that uses additional diacritics for finer-grained phonetic distinctions.</footnote><footnote label="4">Standard German nd is always pronounced [nt] following a general final devoicing rule; we neglect that artifact as we rely only on graphemic representations.</footnote><p>Some rules apply uniformly to all Swiss Ger­man dialects (e.g. the transformation st [st] — scht [ft]). These rules do not immediately contribute to the dialect identification task, but they help to ob­tain correct Swiss German forms that contain other phonemes with better localization potential.</p><p>More information about the creation of the proba­bility maps is given in Sections 4.5 and 4.6.</p></subsection><subsection number="4.3" title="Lexical rules"><p>Some differences at the word level cannot be ac­counted for by pure phonetic alternations. One reason are idiosyncrasies in the phonetic evolution of high frequency words (e.g. Standard German und 'and' is reduced to u in Bern dialect, where the phonetic rules would rather suggest <i>*ung</i>). Another reason is the use of different lexemes altogether (e.g. Standard German <i>immer </i>'always' corresponds to geng, <i>immer, </i>or all, depending on the dialect). We currently use lexical rules mainly for function words and irregular verb stems.</p></subsection><subsection number="4.4" title="Morphological rules"><p>The transformation process from inflected Standard German word forms to inflected Swiss German word forms is done in two steps. First, the word stem is adapted with phonetic or lexical rules, and then, the affixes are generated according to the morphological features of the word.</p><p>Inflection markers also provide dialect discrimina­tion potential. For example, the verbal plural suffixes offer a surprisingly rich (and diachronically stable) interdialectal variation pattern.</p></subsection><subsection number="4.5" title="The linguistic atlas SDS"><p>One of the largest research projects in Swiss German dialectology has been the elaboration of the <i>Sprachat­las der deutschen Schweiz </i>(SDS), a linguistic atlas that covers phonetic, morphological and lexical dif­ferences of Swiss German dialects. Data collection and publication were carried out between 1939 and 1997 (Hotzenköcherle et al., 1962-1997). Linguis­tic data were collected in about 600 villages <i>(in­quiry points) </i>of German-speaking Switzerland, and resulted in about 1500 published maps (see Figure 1 for an example).<page local="4" global="1154"/></p><p>Each map represents a linguistic phenomenon that potentially yields a set of transformation rules. For our experiments, we selected a subset of the maps ac­cording to the perceived importance of the described phenomena. There is no one-to-one correspondence between maps and implemented phenomena, for sev­eral reasons. First, some SDS maps represent in­formation that is best analyzed as several distinct phenomena. Second, a set of maps may illustrate the same phenomenon with different words and slightly different geographic distributions. Third, some maps describe (especially lexical) phenomena that are be­coming obsolete and that we chose to omit.</p><p>As a result, our rule base contains about 300 pho­netic rules covering 130 phenomena, 540 lexical rules covering 250 phenomena and 130 morpholog­ical rules covering 60 phenomena. We believe this coverage to be sufficient for the dialect ID task.</p></subsection><subsection number="4.6" title="Map digitization and interpolation"><p>Recall the nd -example used to illustrate the phonetic rules above. Figure 1 shows a reproduction of the original, hand-drawn SDS map related to this phe­nomenon. Different symbols represent different pho­netic variants of the phenomenon.<footnote anchor="5"/> We will use this example in this section to explain the preprocessing steps involved in the creation of georeferenced rules.</p><p>In a first preprocessing step, the hand-drawn map is digitized manually with the help of a geographical information system. The result is shown in Figure 2. To speed up this process, variants that are used in less than ten inquiry points are omitted. (Many of these small-scale variants likely have disappeared since the data collection in the 1940s.) We also collapse minor phonetic variants which cannot be distinguished in the Dieth spelling system.</p><p>The SDS maps, hand-drawn or digitized, are point maps. They only cover the inquiry points, but do not provide information about the variants used in other locations. Therefore, a further preprocessing step in­terpolates the digitized point maps to obtain surface maps. We follow Rumpf et al. (2009) to create kernel density estimators for each variant. This method is</p><footnote label="5">We define a variant simply as a string that may occur on the right-hand side of a transformation rule.</footnote><p>Figure 1: Original SDS map for the transformation of word-final -nd. The map contains four major linguistic variants, symbolized by horizontal lines (-nd), vertical lines (-nt), circles (-ng), and triangles (-nn) respectively. Minor linguistic variants are symbolized by different types of circles and triangles.</p><figure caption="Figure 2: Digitized equivalent of the map in Figure 1."></figure><p>Figure 3: Interpolated surface maps for the variants -nn (upper left), -ng (upper right), -nt (lower left) and -nd (lower right). Black areas represent a probability of 1, white areas a probability of 0.</p><page local="5" global="1155"/><p>less sensitive to outliers than simpler linear interpola­tion methods.<footnote anchor="6"/> The resulting surface maps are then normalized such that at each point of the surface, the weights of all variants sum up to 1. These normalized weights can be interpreted as conditional probabili­ties of the corresponding transfer rule: <i>p(r</i><i> </i>1<i>1</i>), where <i>r </i>is the rule and <i>t </i>is the geographic location (repre-sentedas a pairoflongitude and latitude coordinates) situated in German-speaking Switzerland. (We call the set of all points in German-speaking Switzerland GSS.) Figure 3 shows the resulting surface maps for each variant. Surface maps are generated with a reso­lution of one point per square kilometer.</p><p>As mentioned above, rules with a common left-hand side are grouped into phenomena, such that at any given point <i>t </i>G GSS, the probabilities of all rules <i>r </i>describing a phenomenon <i>Ph </i>sum up to 1:</p><doubt alpha="41.7" length="12" tooSmall="False" monospace="0.0">VVp(r|t) = 1</doubt><p><i>tGGSS </i><i>rGPh</i></p></subsection></section><section number="5" title="The model"><p>The dialect ID system consists of a Swiss German lexicon that associates word forms with their geo­graphical extension (Section 5.1), and of a testing procedure that splits a sentence into words, looks up their geographical extensions in the lexicon, and condenses the word-level maps into a sentence-level map (Sections 5.2 to 5.4).</p><subsection number="5.1" title="Creating a Swiss German lexicon"><p>The Swiss German word form lexicon is created with the help of the georeferenced transfer rules pre­sented above. These rules require a lemmatized, POS-tagged and morphologically disambiguated Standard German word as an input and generate a set of di­alect word/map tuples: each resulting dialect word is associated with a probability map that specifies its likelihood in each geographic point.</p><p>To obtain a Standard German word list, we ex­tracted all leaf nodes of the TIGER treebank (Brants et al., 2002), which are lemmatized and morphologi­cally annotated. These data also allowed us to obtain word frequency counts. We discarded words with one single occurrence in the TIGER treebank, as well as forms that contained the genitive case or preterite tense attribute (the corresponding grammatical cate­gories do not exist in Swiss German dialects).</p><footnote label="6">A comparison of different interpolation methods will be the object of future work.</footnote><p>The transfer rules are then applied sequentially on each word of this list. The notation w0 <i>— </i><i>wn </i>repre­sents an iterative derivation leading from a Standard German word <i>w</i>0 to a dialectal word form <i>w</i><i>n </i>by the application of <i>n </i>transfer rules of the type <i>wi </i><i>— </i><i>wi+</i>1. The probability of a derivation corresponds to the joint probability of the rules it consists of. Hence, the probability map of a derivation is defined as the <b>pointwise product </b>of all rule maps it consists of:</p><doubt alpha="46.2" length="26" tooSmall="False" monospace="0.0">Vp(w0—wn|t) =Hp(wi —w+1|t)</doubt><p><i>t GGSS k</i><i>=0</i></p><p>Note that in dialectological transition zones, there may be several valid outcomes for a given <i>w</i>0.</p><p>The Standard German word list extracted from TIGER contains about 36,000 entries. The derived Swiss German word list contains 560,000 word forms, each of which is associated with a map that specifies its regional distribution.<footnote anchor="7"/> Note that proper nouns and words tagged as "foreign material" were not transformed. Derivations that did not obtain a probability higher than 0.1 anywhere (because of geographically incompatible transformations) were discarded.</p></subsection><subsection number="5.2" title="Word lookup and dialect identification"><p>At test time, the goal is to compute a probability map for a text segment of unknown origin.<footnote anchor="8"/> As a prepro­cessing step, the segment is tokenized, punctuation markers are removed and all words are converted to lower case.</p><p>The identification process can be broken down in three levels:</p><p>1. The probability map of a text segment depends on the probability maps of the words contained in the segment.</p><p>2. The probability map of a word depends on the probability maps of the derivations that yield the word.</p><footnote label="7">Technically, we do not store the probability map, but the sequence of rule variants involved in the derivation. The proba­bility map is restored from this rule sequence at test time.</footnote><footnote label="8">The model does not require the material to be syntactically well-formed. Although we use complete sentences to test the system, any sequence of words is accepted.</footnote><page local="6" global="1156"/><p>3. The probability map of a derivation depends on the probability maps of the rules it consists of.</p><p>In practice, every word of a given text segment is looked up in the lexicon. If this lookup does not suc­ceed (either because its Standard German equivalent did not appear in the TIGER treebank, or because the rule base lacked a relevant rule), the word is skipped. Otherwise, the lookup yields <i>m </i>derivations from <i>m </i>different Standard German words.<footnote anchor="9"/> The lexicon al­ready contains the probability maps of the derivations (see 5.1), so that the third level does not need to be discussed here. Let us thus explain the first two levels in more detail, in reverse order.</p></subsection><subsection number="5.3" title="Computing the probability map for a word"><p>A dialectal word form may originate in different Stan­dard German words. For example, the three deriva­tions sind [VAFIN] — si (valid only in Western di­alects), sein [PPOSAT] — si (in Western and Central dialects), and sie [PPER] — si (in the majority of Swiss German dialects) all lead to the same dialectal form si.</p><p>Our system does not take the syntactic context into account and therefore cannot determine which derivation is the correct one. We approximate by choosing the most probable one in each geographic location. The probability map of a Swiss German word <i>w </i>is thus defined as the <b>pointwise maximum<footnote anchor="10"/></b>of all derivations leading to w, starting with different Standard German words <i>w(0j):</i></p><doubt alpha="40.0" length="25" tooSmall="False" monospace="0.0">Vp(w11) = max p(w0j)—w11)</doubt><p><i>t<b>e</b>GSS j </i><footnote anchor="0"/></p><p>This formula does not take into account the relative frequency of the different derivations of a word. This may lead to unintuitive results. Consider the two derivations der [ART] — dr (valid only in Western dialects) and Dr. [NN] — dr (valid in all dialects). The occurrence of the article dr in a dialect text is a good indicator for Western Swiss dialects, but it is completely masked by the potential presence of the abreviation Dr. in all dialects. We can avoid this by weighting the derivations by the <b>word frequency </b>of <i>w</i>0: the article der is much more frequent than the abreviation Dr. and is thus given more weight in the identification task. This weighting can be justified on dialectological grounds: frequently used words tend to show higher interdialectal variation than rare words.</p><footnote label="9">Theoretically, two derivations can originate at the same Standard German word and yield the same Swiss German word, but nevertheless use different rules. Our system handles such cases as well, but we are not aware of such cases occurring with the current rule base.</footnote><footnote label="10">Note that these derivations are alternatives and not joint events. This is thus not a joint probability.</footnote><p>Another assumption in the above formula is that each derivation has the same <b>discriminative poten­tial. </b>Again, this is not true: a derivation that is valid in only 10% of the Swiss German dialect area is much more informative than a derivation that is valid in 95% of the dialect area. Therefore, we propose to weight each derivation by the proportional size of its validity area. The discriminative potential of a derivation <i>d </i>is defined as follows:<footnote anchor="11"/></p><doubt alpha="27.3" length="11" tooSmall="False" monospace="0.0">DP(d) = 1 -</doubt><doubt alpha="64.7" length="17" tooSmall="False" monospace="0.0">LteGSSp(d11)|GSS|</doubt><p>Theexperiments inSection6willshowtherelative impact of these two weighting techniques and of the combination of both with respect to the unweighted map computation.</p></subsection><subsection number="5.4" title="Computing the probability map for a segment"><p>The probability of a text segment <i>s </i>can be defined as the joint probability of all words <i>w </i>contained in the segment. Again, we compute the <b>pointwise product </b>of all word maps. In contrast to 5.1, we performed some smoothing in order to prevent erroneous word derivations from completely zeroing out the proba­bilities. We assumed a minimum word probability of <b>0 </b>= 0.1 for all words in all geographic points:</p><p><i>t <b>e</b>GSS w<b>e</b>s</i></p><p>Erroneous derivations were mainly due to non-implemented lexical exceptions.</p></subsection></section><section number="6" title="Experiments and results 6.1 Data"><p>In order to evaluate our model, we need texts an­notated with their gold dialect. We have chosen to use the Alemannic Wikipedia as a main data source.</p><footnote label="11">d is anotationalabreviationfor</footnote><p><i>w<b>0 </b>— w</i><i>n.</i></p><page local="7" global="1157"/><p>Table 1: The six dialect regions selected for our tests, with their annotation on Wikipedia and our abreviation. We also show the percentage of the German-speaking population living in the regions, and the percentage of the surface of the region relative to the entire country.</p><figure caption="Figure 4: The localization of the six dialect regions used in our study."></figure><p>The Alemannic Wikipedia allows authors to write articles in any dialect, and to annotate the articles with their dialect. Eight dialect categories contained more than 10 articles; we selected six dialects for our experiments (see Table 1 and Figure 4).</p><p>We compiled a test set consisting of 291 sentences, distributed across the six dialects according to their population size. The sentences were taken from dif­ferent articles. In addition, we created a development set consisting of 550 sentences (100 per dialect, ex­cept FR, where only 50 sentences were available). This development set was also used to train the base­line model discussed in section 6.2.</p><p>In order to test the robustness of our model, we collected a second set of texts from various web sites other than Wikipedia. The gold dialect of these texts could be identified through metadata.<footnote anchor="12"/> This informa­tion was checked for plausibility by the first author. The Web data set contains 144 sentences (again dis-</p><footnote label="12">We mainly chose websites of local sports and music clubs, whose localization allowed to determine the dialect of their con­tent.</footnote><p>Table 2: Performances of the 5-gram model on Wikipedia test data (left) and Web test data (right). The columns refer to precision, recall and F-measure respectively. The average is weighted by the relative population sizes of the dialect regions.</p><p>tributed according to population size) and is thus roughly half the size of the Wikipedia test set.</p><p>The Wikipedia data contains an average of 17.8 words per sentence, while the Web data shows 14.9 words per sentence on average.</p><subsection number="6.2" title="Baseline: N-gram model"><p>To compare our dialect ID model, we created a base­line system that uses a character-n-gram approach. This approach is fairly common for language ID and has also been successfully applied to dialect ID (Bi-adsy et al., 2009). However, it requires a certain amount of training data that may not be available for specific dialects, and it is uncertain how it performs with very similar dialects.</p><p>We trained 2-gram to 6-gram models for each di­alect with the SRILM toolkit (Stolcke, 2002), using the Wikipedia development corpus. We scored each sentence of the Wikipedia test set with each dialect model. The predicted dialect was the one which ob­tained the lowest perplexity.<footnote anchor="13"/></p><p>The 5-gram model obtained the best overall per­formance, and results on the Wikipedia test set were surprisingly good (see Table 2, leftmost columns).<footnote anchor="14"/>Note that in practice, 100% accuracy is not always achievable; a sentence may not contain a sufficient localization potential to assign it unambiguously to one dialect.</p><footnote label="13">We assume that all test sentences are written in one of the six dialects.</footnote><footnote label="14">All results represent percentage points. We omit decimal places as all values are based on 100 or less data points. We did not perform statistical significance tests on our data.</footnote><table class="main" frame="box" rules="all" border="0" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Wikipedia name</p></td><td class="cell"><p>Abbr.</p></td><td class="cell"><p>Pop.</p></td><td class="cell"><p>Surface</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Baseldytsch</p></td><td class="cell"><p>BA</p></td><td class="cell"><p>8%</p></td><td class="cell"><p>1%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Bärndütsch</p></td><td class="cell"><p>BE</p></td><td class="cell"><p>17%</p></td><td class="cell"><p>13%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Seislertütsch</p></td><td class="cell"><p>FR</p></td><td class="cell"><p>2%</p></td><td class="cell"><p>1%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Ostschwizertütsch</p></td><td class="cell"><p>OS</p></td><td class="cell"><p>14%</p></td><td class="cell"><p>8%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Wallisertiitsch</p></td><td class="cell"><p>WS</p></td><td class="cell"><p>2%</p></td><td class="cell"><p>7%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Züritüütsch</p></td><td class="cell"><p>ZH</p></td><td class="cell"><p>22%</p></td><td class="cell"><p>4%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p></p></td><td class="cell"><p>Web</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>34</p></td><td class="cell"><p>61</p></td><td class="cell"><p>44</p></td><td class="cell"><p>27</p></td><td class="cell"><p>61</p></td><td class="cell"><p>37</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>78</p></td><td class="cell"><p>51</p></td><td class="cell"><p>61</p></td><td class="cell"><p>51</p></td><td class="cell"><p>47</p></td><td class="cell"><p>49</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>28</p></td><td class="cell"><p>71</p></td><td class="cell"><p>40</p></td><td class="cell"><p>10</p></td><td class="cell"><p>33</p></td><td class="cell"><p>15</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>63</p></td><td class="cell"><p>64</p></td><td class="cell"><p>64</p></td><td class="cell"><p>50</p></td><td class="cell"><p>38</p></td><td class="cell"><p>43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>58</p></td><td class="cell"><p>100</p></td><td class="cell"><p>74</p></td><td class="cell"><p>14</p></td><td class="cell"><p>33</p></td><td class="cell"><p>20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>77</p></td><td class="cell"><p>62</p></td><td class="cell"><p>69</p></td><td class="cell"><p>77</p></td><td class="cell"><p>41</p></td><td class="cell"><p>53</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p>62</p></td><td class="cell"><p>46</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="8" global="1158"/><p>However, we suspect that these results are due to overfitting. It turns out that the number of Swiss German Wikipedia authors is very low (typically, one or two active writers per dialect), and that ev­ery author uses distinctive spelling conventions and writes about specific subjects. For instance, most ZH articles are about Swiss politicians, while many OS articles deal with religion and mysticism. Our hypothesis is thus that the n-gram model learns to recognize a specific author and/or topic rather than a dialect. This hypothesis is confirmed on the Web data set: the performances drop by 15 percentage points or more (same table, rightmost columns; the performance drops are similar for <i>n </i>= [2..6]).</p><p>In all our evaluations, the average F-measures for the different dialects are weighted according to the relative population sizes of the dialect regions be­cause the size of the test corpus is proportional to population size (see Section 6.1).<footnote anchor="15"/></p><p>We acknowledge that a training corpus of only 100 sentences per dialect provides limited insight into the performance of the n-gram approach. We were able to double the training corpus size with additional Wikipedia sentences. With this extended corpus, the 4-gram model performed better than the 5-gram model. It yielded a weighted average F-measure of 79% on Wikipedia test data, but only 43% on Web data. The additional increase on Wikipedia data (+17% absolute with respect to the small training set), together with the decrease on Web data (-3% absolute) confirms our hypothesis of overfitting. An ideal training corpus should thus contain data from several sources per dialect.</p><p>To sum up, n-gram models can yield good perfor­mance even with similar dialects, but require large amounts of training data from different sources to achieve robust results. For many small-scale dialects, such data may not be available.</p></subsection><subsection number="6.3" title="Our model"><p>The n-gram system presented above has no geo­graphic knowledge whatsoever; it just consists of six distinct language models that could be located anywhere. In contrast, our model yields probability maps of German-speaking Switzerland. In order to evaluate its performance, we thus had to determine the geographic localization of the six dialect regions defined by the Wikipedia authors (see Table 1). We defined the regions according to the respective can­ton boundaries and to the German-French language border in the case of bilingual cantons. The result of this mapping is shown in Figure 4.</p><footnote label="15">Roughly, this weighting can be viewed as a prior (the proba­bility of the text being constant):</footnote><doubt alpha="65.3" length="49" tooSmall="False" monospace="0.0">p(dialect | text) = p(text | dialect) *p(dialect)</doubt><p>The predicted dialect region of a sentence <i>s </i>is de­fined as the region in which the most probable point has a higher value than the most probable point in any other region:</p><doubt alpha="59.4" length="32" tooSmall="False" monospace="0.0">Region(s) =arg max I  maxp(s11))</doubt><p><i>Region </i>\<i> t<b>e</b>Region I</i></p><p>Experiments were carried out for the four combi­nations of the two derivation-weighting techniques presented in Section 5.3 and for the two test sets (Wikipedia and Web). Results are displayed in Ta­bles 3 to 6. The majority of FR sentences were mis-classified as BE, which reflects the geographic and linguistic proximity of these regions.</p><p>The tables show that frequency weighting helps on both corpora: the discriminative potential only slightly improves performance on the web corpus. Crucially, the two techniques are additive, so in combination, they yield the best overall results. In comparison with the baseline model, there is a per­formance drop of about 16 percent absolute on Wikipedia data. In contrast, our model is very ro­bust and outperforms the baseline model on the Web test set by about 7 percent absolute.</p><p>These results seem to confirm what we suggested above: that the n-gram model overfitted on the small Wikipedia training corpus. Nevertheless, it is still surprising that our model has a lower performance on Wikipedia than on Web data. The reason for this discrepancy probably lies in the spelling conventions assumed in the transformation rules: it seems that Web writers are closer to these (implicit) spelling conventions than Wikipedia authors. This may be explained by the fact that many Wikipedia articles are translations of existing Standard German articles, and that some words are not completely adapted to their dialectal form. Another reason could be that Wikipedia articles use a proportionally larger amount of proper nouns and low-frequency words which can-<page local="9" global="1159"/></p><p>Table 5: Performances of the word-based model using derivation maps weighted by their discriminative potential.</p><p>Table 6: Performances using derivation maps weighted by word frequency and discriminative potential.</p><p>not be found in the lexicon and which therefore re­duce the localization potential of a sentence.</p><p>However, one should note that the word-based di­alect ID model is not limited on the six dialect regions used for evaluation here. It can be used with any size and number of dialect regions of German-speaking Switzerland. This contrasts with the n-gram model which has to be trained specifically on every dialect region; in this case, the Swiss German Wikipedia only contains two additional dialect regions with an equivalent amount of data.</p></subsection><subsection number="6.4" title="Variations"><p>In the previous section, we have defined the predicted dialect region as the one in which the most probable point <b>(maximum) </b>has a higher probability than the most probable point of any other region. The results suggest that this metric penalizes small regions (BA, FR, ZH). In these cases, it is likely that the most probable point is slightly outside the region, but that the largest part of the probability mass is still inside the correct region. Therefore, we tested another ap­proach: we defined the predicted dialect region as the one in which the <b>average </b>probability is higher than the average probability in any other region:</p><p><i>Region(s) = </i>arg max</p><doubt alpha="100.0" length="6" tooSmall="False" monospace="0.0">Region</doubt><p><b>/<u>E</u></b><u>t</u><b><i><u>e</u></i></b><i><u>Regionp(s | </u></i><i><u>t</u></i><i><u>)</u> \</i> <b>V     </b><b><i>R</i></b><i>egion </i><b><i>J</i></b></p><p>This metric effectively boosts the performance on the smaller regions, but comes at a cost for larger regions (Table 7). We also combined the two metrics by using the <b>maximum </b>metric for the three larger regions and the <b>average </b>metric for the three smaller ones (the cutoff lies at 5% of the Swiss territory). This combined metric further improves the perfor­mance of our system while relying on an objective measure of region surface.</p><p>We believe that region surface as such is not so crucial for the metrics discussed above, but rather serves as a proxy for linguistic heterogeneity. Geo­graphically large regions like BE tend to have internal dialect variation, and averaging over all dialects in the region leads to low figures. In contrast, small regions show a quite homogeneous dialect landscape that may protrude over adjacent regions. In this case, the probability peak is less relevant than the average probability in the entire region. Future work will at­tempt to come up with more fine-grained measures of<page local="10" global="1160"/></p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p></p></td><td class="cell"><p>Web</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>41</p></td><td class="cell"><p>19</p></td><td class="cell"><p>26</p></td><td class="cell"><p>80</p></td><td class="cell"><p>22</p></td><td class="cell"><p>35</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>42</p></td><td class="cell"><p>62</p></td><td class="cell"><p>50</p></td><td class="cell"><p>48</p></td><td class="cell"><p>76</p></td><td class="cell"><p>59</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>17</p></td><td class="cell"><p>33</p></td><td class="cell"><p>22</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>36</p></td><td class="cell"><p>41</p></td><td class="cell"><p>38</p></td><td class="cell"><p>45</p></td><td class="cell"><p>41</p></td><td class="cell"><p>43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>3</p></td><td class="cell"><p>14</p></td><td class="cell"><p>5</p></td><td class="cell"><p>8</p></td><td class="cell"><p>33</p></td><td class="cell"><p>13</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>65</p></td><td class="cell"><p>33</p></td><td class="cell"><p>44</p></td><td class="cell"><p>62</p></td><td class="cell"><p>37</p></td><td class="cell"><p>46</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p>40</p></td><td class="cell"><p>46</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 3: Performances of the word-based model using unweighted derivation maps." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p></p></td><td class="cell"><p>Web</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>34</p></td><td class="cell"><p>31</p></td><td class="cell"><p>32</p></td><td class="cell"><p>38</p></td><td class="cell"><p>17</p></td><td class="cell"><p>23</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>46</p></td><td class="cell"><p>47</p></td><td class="cell"><p>47</p></td><td class="cell"><p>54</p></td><td class="cell"><p>76</p></td><td class="cell"><p>63</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>11</p></td><td class="cell"><p>14</p></td><td class="cell"><p>13</p></td><td class="cell"><p>20</p></td><td class="cell"><p>33</p></td><td class="cell"><p>25</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>34</p></td><td class="cell"><p>50</p></td><td class="cell"><p>40</p></td><td class="cell"><p>53</p></td><td class="cell"><p>59</p></td><td class="cell"><p>56</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>5</p></td><td class="cell"><p>14</p></td><td class="cell"><p>7</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>47</p></td><td class="cell"><p>27</p></td><td class="cell"><p>34</p></td><td class="cell"><p>75</p></td><td class="cell"><p>43</p></td><td class="cell"><p>55</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p>37</p></td><td class="cell"><p>51</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 4: Performances of the word-based model using derivation maps weighted by word frequency." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p></p></td><td class="cell"><p>Web</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>50</p></td><td class="cell"><p>33</p></td><td class="cell"><p>40</p></td><td class="cell"><p>57</p></td><td class="cell"><p>22</p></td><td class="cell"><p>32</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>47</p></td><td class="cell"><p>60</p></td><td class="cell"><p>53</p></td><td class="cell"><p>60</p></td><td class="cell"><p>79</p></td><td class="cell"><p>68</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>29</p></td><td class="cell"><p>31</p></td><td class="cell"><p>30</p></td><td class="cell"><p>46</p></td><td class="cell"><p>50</p></td><td class="cell"><p>48</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>11</p></td><td class="cell"><p>29</p></td><td class="cell"><p>15</p></td><td class="cell"><p>17</p></td><td class="cell"><p>33</p></td><td class="cell"><p>22</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>60</p></td><td class="cell"><p>47</p></td><td class="cell"><p>53</p></td><td class="cell"><p>65</p></td><td class="cell"><p>53</p></td><td class="cell"><p>58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p>44</p></td><td class="cell"><p><b>53</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Wikipedia</p></td><td class="cell"><p></p></td><td class="cell"><p>Web</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"><p>P</p></td><td class="cell"><p>R</p></td><td class="cell"><p>F</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>46</p></td><td class="cell"><p>28</p></td><td class="cell"><p>35</p></td><td class="cell"><p>33</p></td><td class="cell"><p>11</p></td><td class="cell"><p>17</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>47</p></td><td class="cell"><p>62</p></td><td class="cell"><p>54</p></td><td class="cell"><p>58</p></td><td class="cell"><p>84</p></td><td class="cell"><p>69</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>0</p></td><td class="cell"><p>20</p></td><td class="cell"><p>33</p></td><td class="cell"><p>25</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>35</p></td><td class="cell"><p>31</p></td><td class="cell"><p>33</p></td><td class="cell"><p>47</p></td><td class="cell"><p>47</p></td><td class="cell"><p>47</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>8</p></td><td class="cell"><p>29</p></td><td class="cell"><p>13</p></td><td class="cell"><p>14</p></td><td class="cell"><p>33</p></td><td class="cell"><p>20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>63</p></td><td class="cell"><p>53</p></td><td class="cell"><p>58</p></td><td class="cell"><p>66</p></td><td class="cell"><p>51</p></td><td class="cell"><p>58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p><b>46</b></p></td><td class="cell"><p>52</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>Table 7: Comparison of different evaluation metrics. All values refer to F-measures obtained with frequency and discriminative potential-weighted derivation maps. Max refers to the Maximum metric as used in Table 6. Avg refers to the average metric, and Cmb is the combination of both metrics depending on region surfaces. The under­lined values in the Avg and Max columns represent those used for the Cmb metric.</p><p>linguistic heterogeneity in order to test these claims. <b>7  Future work</b></p><p>In our experiments, the word-based dialect identifi­cation model skipped about one third of all words (34% on the Wikipedia test set, 39% on the Web test set) because they could not be found in the lex­icon. While our model does not require complete lexical coverage, this figure shows that the system can be improved. We see two main possibilities of improvement. First, the rule base can be extended to better account for lexical exceptions, orthographic variation and irregular morphology. Second, a mixed approach could combine the benefits of the word-based model with the n-gram model. This would require a larger, more heterogeneous set of training material for the latter in order to avoid overfitting. Additional training data could be extracted from the web and automatically annotated with the current model in a semi-supervised approach.</p><p>In the evaluation presented above, the task con­sisted of identifying the dialect of single sentences. However, one often has access to longer text seg­ments, which makes our evaluation setup harder than necessary. This is especially important in situ­ations where a single sentence may not always con­tain enough discriminative material to assign it to a unique dialect. Testing our dialect identification sys­tem on the paragraph or document level could thus provide more realistic results.</p></subsection></section><section number="8" title="Conclusion"><p>In this paper, we have compared two empirical meth­ods for the task of dialect identification. The n-gram method is based on the approach most commonly used in NLP: it is a supervised machine learning ap­proach where training data of the type we need to process is annotated with the desired outcome of the processing.</p><p>Our second approach - the main contribution of this paper - is quite different. The empirical compo­nent consists in a collection of data (the SDS atlas) which is not of the type we want to process, but rather embodies some features of the data we ultimately want to process. We therefore analyze this data in order to extract empirically grounded knowledge for more general use (the creation of the georeferenced rules), and then use this knowledge to perform the dialect ID task in conjunction with an unrelated data source (the Standard German corpus).</p><p>Our choice of method was of course related to the fact that few corpora, annotated or not, were avail­able for our task. But beyond this constraint, we think it may be well worthwhile for NLP tasks in general to move away from a narrow machine learn­ing paradigm (supervised or not) and to consider a broader set of empirical resources, sometimes re­quiring methods which are quite different from the prevalent ones.</p></section><section title="Acknowledgements"><p>Part of this work was carried out during the first au­thor's stay at Columbia University, New York, funded by the Swiss National Science Foundation (grant PBGEP1-125929).</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Dialect</p></td><td class="cell"><p>Wikipedia Max   Avg Cmb</p></td><td class="cell"><p>Max</p></td><td class="cell"><p>Web</p><p>Avg</p></td><td class="cell"><p>Cmb</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BA</p></td><td class="cell"><p>35</p></td><td class="cell"><p>32</p></td><td class="cell"><p>32</p></td><td class="cell"><p>17</p></td><td class="cell"><p>43</p></td><td class="cell"><p>43</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>BE</p></td><td class="cell"><p>54</p></td><td class="cell"><p>39</p></td><td class="cell"><p>54</p></td><td class="cell"><p>69</p></td><td class="cell"><p>54</p></td><td class="cell"><p>69</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>FR</p></td><td class="cell"><p>0</p></td><td class="cell"><p>7</p></td><td class="cell"><p>7</p></td><td class="cell"><p>25</p></td><td class="cell"><p>11</p></td><td class="cell"><p>11</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>OS</p></td><td class="cell"><p>33</p></td><td class="cell"><p>23</p></td><td class="cell"><p>33</p></td><td class="cell"><p>47</p></td><td class="cell"><p>49</p></td><td class="cell"><p>47</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>WS</p></td><td class="cell"><p>13</p></td><td class="cell"><p>13</p></td><td class="cell"><p>13</p></td><td class="cell"><p>20</p></td><td class="cell"><p>31</p></td><td class="cell"><p>20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ZH</p></td><td class="cell"><p>58</p></td><td class="cell"><p>60</p></td><td class="cell"><p>60</p></td><td class="cell"><p>58</p></td><td class="cell"><p>68</p></td><td class="cell"><p>68</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>W. Avg.</p></td><td class="cell"><p>46</p></td><td class="cell"><p>40</p></td><td class="cell"><p><b>47</b></p></td><td class="cell"><p>52</p></td><td class="cell"><p>55</p></td><td class="cell"><p><b>58</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken Arabic dialect identification using phonotactic modeling. In <i>EACL 2009 Workshop on Computational Approaches to Semitic Languages, </i>Athens. S. Brants, S. Dipper, S. Hansen, W. Lezius, and G. Smith. 2002. The TIGER Treebank. In <i>Proceedings of the Workshop on Treebanks and Linguistic Theories, </i>So-zopol.</p><p>W. B. Cavnar and J. M. Trenkle. 1994. N-gram based text categorization. In <i>Proceedings ofSDAIR'94, </i>Las Vegas.</p><page local="11" global="1161"/><p>Eugen Dieth. 1986. <i>Schwyzertütschi Dialäktschrift. </i>Sauerländer, Aarau, 2nd edition.</p><p>Rudolf Hotzenköcherle, Robert Schläpfer, Rudolf Trüb, and Paul Zinsli, editors. 1962-1997. <i>Sprachatlas der deutschen Schweiz. </i>Francke, Berne.</p><p>Baden Hughes, Timothy Baldwin, Steven Bird, Jeremy Nicholson, and Andrew MacKinlay. 2006. Recon­sidering language identification for written language resources. In <i>Proceedings ofLREC'06, </i>Genoa.</p><p>N. Ingle. 1980. A language identification table. <i>Technical Translation International.</i></p><p>Gideon S. Mann and David Yarowsky. 2001. Multipath translation lexicon induction via bridge languages. In <i>Proceedings ofNAACL'01, </i>Pittsburgh.</p><p>Radim Rehurek and Milan Kolkus. 2009. Language identification on the web: Extending the dictionary method. In <i>Computational Linguistics and Intelligent Text Processing - Proceedings ofCICLing 2009, </i>pages 357-368, Mexico. Springer.</p><p>Jonas Rumpf, Simon Pickl, Stephan Elspaß, Werner König, and Volker Schmidt. 2009. Structural analysis of dialect maps using methods from spatial statistics. <i>Zeitschrift für Dialektologie und Linguistik, </i>76(3).</p><p>Charles Schafer and David Yarowsky. 2002. Inducing translation lexicons via diverse similarity measures and bridge languages. In <i>Proceedings ofCoNLL'02, </i>pages 146-152, Taipei.</p><p>Yves Scherrer and Owen Rambow. 2010. Natural lan­guage processing for the Swiss German dialect area. In <i>Proceedings of KONVENS' 10, </i>Saarbrücken.</p><p>Yves Scherrer. 2007. Adaptive string distance measures for bilingual dialect lexicon induction. In <i>Proceedings of ACL ' 07, Student Research Workshop, </i>pages 55-60, Prague.</p><p>Yves Scherrer. 2010. Des cartes dialectologiques numérisées pour le TALN. In <i>Proceedings ofTALN'10, </i>Montréal.</p><p>Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In <i>Proceedings of ICSLP 02, </i>pages 901-904, Denver.</p></references></body></article>