<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="142"/><title>A Two-Level Morphological Analyser for the Indonesian Language</title><author surname="Pisceldo" givenname="Femphy"><org  name="University of Indonesia" country="Indonesia" city="Jakarta"/></author><author surname="Mahendra" givenname="Rahmad"><org  name="Australian National University" country="Australia" city="Canberra"/></author><author surname="Manurung" givenname="Ruli"><org  name="Pusan National University" country="Korea" city="Pusan"/></author><author surname="Arka" givenname="I Wayan"><org  name="Konan University" country="Japan" city="Kobe"/></author></firstpageheader><frontmatter><p><b>A Two-Level Morphological Analyser for the Indonesian Language</b></p><p><b>Femphy Pisceldo, Rahmad Mahendra &amp; Ruli Manurung I Wayan Arka</b></p><p>Faculty of Computer Science Department of Linguistics</p><p>University of Indonesia Australian National University</p><p>fep40@ui.edu,rama42@ui.edu,maruli@cs.ui.ac.id wayan.arka@anu.edu.au</p></frontmatter><abstract>This paper presents our efforts at developing an Indonesian morphological analyser that pro­vides a detailed analysis of the rich affixation process<footnote anchor="1"/>. We model Indonesian morphology us­ing a two-level morphology approach, decom­posing the process into a set of morphotactic and morphophonemic rules. These rules are modelled as a network of finite state transduc­ers and implemented using <b>xfst </b>and <b>lexc. </b>Our approach is able to handle reduplication, a non-concatenative morphological process. </abstract></header><body><section number="1" title="Introduction"><p>Morphology is the study of the way that words are built up from smaller units called morphemes, the minimal meaning-bearing units in a language (Ju-rafsky, 2000). For example, the English word <i>kind </i>consists of a single morpheme (the root word <i>kind) </i>whilst the word <i>players </i>consists of three mor­phemes: <i>play, -er </i>and -s. The morphemes <i>kind </i>and <i>play </i>can stand alone as words, while affixes <i>-er </i>and <i>-s </i>must appear bound to another morpheme.</p><p>By applying a set of morphological rules, we can produce not just morphemes but also other in­formation relating to the words; for example, the grammatical category of the whole word as well as the subcategorisation frame of the word if it is a verb. This process is called morphological analysis. In this respect, the value of a morphological ana­lyser would be twofold: from a theoretical (linguis­tic) viewpoint, it is a very useful tool for linguistic modelling and for testing certain analyses. On the other hand, from a practical viewpoint, it supports many applications, e.g. information retrieval, search engines, and machine translation, among others.</p><p>There is currently some interest in developing morphological tools for the Indonesian language. In previous work, Siregar (1995) and Adriani et al. (2007) discuss the development of Indonesian stemmers that recover a root from an affixed word, implemented procedurally. However, stemmers are of limited use as they do not provide any more lin­guistic information beyond the stem. Hartono (2002) presents an initial version of a morphologi­cal analyser developed with PC-KIMMO, but un­fortunately, it does not handle reduplication, a key aspect of Indonesian morphology. The morpho­logical analyser that we are developing and that we describe here is designed to be able to handle the rich semantic, lexical and grammatical information associated with words and word formation in NLP applications.</p><p>In Section 2, we first discuss Indonesian mor­phology, followed by a brief explanation of two-level morphology in Section 3. Sections 4 and 5 present our work in applying two-level morphol­ogy for the Indonesian language. Finally, Section 6 presents the results of some evaluations we carried out on our developed analyser.</p></section><section number="2" title="Indonesian Language Morphology"><p>Indonesian is a variety of Malay, which belongs to the Austronesian language family (Gordon, 2005; Sneddon, 2003). It is spoken by about 190 million people in Indonesia and other parts of the world<footnote anchor="2"/>.</p><footnote label="1">This research is part of a collaborative research project funded by ARC Discovery Grant DP0877595.</footnote><footnote label="2">According to Biro Pusat Statistik, as of 2004.</footnote><page local="2" global="143"/><p>Words in Indonesian are built from their roots by means of a variety of morphological operations including compounding, affixation, and reduplica­tion. Indonesian concatenative morphology regu­lates how a stem and its affixes glue together, while a non-concatenative one combines mor­phemes in more complex ways.</p><p>Affixes in Indonesian can be classified as four categories (Alwi et al., 2003). Prefixes precede the base form, i.e. <i>meN-, di-, peN-, per-, ke-, </i>and <i>ter-. </i>Suffixes follow the base form, i.e. <i>-kan, -an, </i>and <i>-i. </i>Infixes are inside the base form, i.e. <i>-el-, -em-, </i>and <i>-er-. </i>Circumfixes wrap around the base form. While circumfixes formally are combinations of allowed prefixes and suffixes, they have to be treated as discontinuous units for semantic and grammatical reasons. In our Indonesian morpho­logical analyser, we have handled prefixes, suf­fixes, and circumfixes.</p><p>Indonesian non-concatenative morphology re­fers to reduplicated morpheme forms. Reduplicated words based on morpheme regularity are grouped into full reduplication (e.g., the word <i>buku-buku </i>is derived from the stem <i>buku) </i>and partial reduplica­tion of different kinds. The latter includes redupli­cated stems with affixes (e.g. the word <i>buah-buahan </i>is derived from stem <i>buah, bertingkat-tingkat </i>is derived from stem <i>bertingkat) </i>and vari­ous (often rather irregular) reduplications (e.g. <i>sa-yur-mayur </i>is derived from <i>sayur).</i></p><p>Indonesian affixation and reduplication are illus­trated in the following example. From the stem <i>pukul, </i>we can derive words like <i>pemukul </i>(by con­catenating the prefix <i>peN-), memukuli </i>(by concate­nating the circumfix <i>meN-i), </i>and <i>pukulan-pukulan </i>(by first concatenating the suffix -an, then apply­ing full reduplication). Other examples include <i>dipukul, pemukulan, </i>and <i>berpukul-pukulan.</i></p><p>All kinds of regular word-formation with or without reduplication are recognized in this re­search.</p><p>Morphology generally makes a distinction be­tween inflectional and derivational processes. In the previous example, the formation of <i>memukuli </i>appears to be 'inflectional' as the formation does not change the category of the stem, and the forma­tion of <i>pemukul </i>and <i>pukulan-pukulan </i>is deriva­tional because the derived words are nouns while the stem <i>pukul </i>is a verb. However, the distinction between inflection and derivation, particularly in Indonesian, is not always clear cut. The formation of verbs such as <i>memukuli </i>from <i>pukul </i>is arguably derivational in nature because the new words have quite different lexical properties, even though both the new verbs and the stems are of the same cate­gory (i.e. 'verb').</p></section><section number="3" title="Two-Level Morphology"><p>A morphological analyser can be used to process a list of morphemes in a lexicon to yield fully derived words. In the other direction, it can be used to identify affixes and roots from derived words. For example, taking the word <i>membaca </i>as input, an Indonesian morphological analyser should produce the string baca<b>+Verb+AV, </b>indicating that the active verb <i>membaca </i>is derived from <i>baca.</i></p><p>However, it is difficult to accomplish morpho­logical analysis in one step only. In Indonesian, affixation does not consist of just fusing the affix with the stem. Modification of phonemes may be necessary, e.g. the concatenation of <i>meN+putar </i>will produce word <i>memutar, </i>while the result of <i>meN+siram </i>is <i>menyiram.</i></p><p>To solve this problem, we adopt a two-level morphology model. Two-level morphology is based on finite-state transducers (Koskenniemi, 1983). In practice, this model has been success­fully applied to several languages such as English, Finnish, Japanese, Russian, and French. It is not only useful to account for various concatenative morphologies, but is also able to handle non-concatenative processes, as has been developed in Malagasy tokenization and morphological analysis (Dalrymple et al., 2006).</p></section><section number="4" title="Design"><p>Our design for an Indonesian morphological ana­lyser is divided into two components: <b>morphotac-tic </b>rules that explain which classes of morphemes can follow other classes of morphemes inside a word, and <b>morphophonemic </b>rules which model the changes that occur in a word. The rules in each component are typically applied in parallel. Addi­tionally, these rules are combined with a lexicon of stems in order to complete the full design.</p><p>A word, in order to be analysed, will follow the path lexicon -&gt; morphotactic rules -&gt; morphopho-nemic rules -&gt; surface. Before the result of the morphological analyser appears at the surface, it will follow the lexicon path to determine the actual morpheme of that word. After moving from the lexicon, that word will be analysed by morphotac-tic and morphophonemic rules.<page local="3" global="144"/> Only after finishing the process in morphotactic and morphophonemic rules, will the result of morphological analyser for that word be delivered.</p><p>The explanation below will explore, in a little more depth, the design of lexicon, morphotactic rules, and morphophonemic rules.</p><subsection number="4.1" title="Lexicon design"><p>Our lexicon equates to a set of Indonesian stem words. Affixes are not stored in the lexicon as they are already accounted for by the morphotactic rules.</p><p>For our initial design, our lexicon is divided into four classes, i.e. verbs, nouns, adjectives, and 'etc', which includes all other stems, e.g. pronouns, ad­verbs, numbers, and particles. Clustering these word classes together is certainly a large oversim­plification, and one which we hope to address in future revisions.</p></subsection><subsection number="4.2" title="Tag design"><p>The design of tags has become very important in the development of morphological analysers, since the tags will deliver linguistic information that oc­curs on a word being analysed.</p><p>In our research, the tags to be designed can be divided into normal tags and special tags. Normal tags can be output by the morphotactics component without any circumstances, whilst special tags only occur if the involved stems are associated with specific markers in the lexicon. The normal tags are <b>+Verb</b>, <b>+Noun</b>, <b>+Adj</b>, <b>+BareVerb</b>, <b>+BareNoun</b>, <b>+BareAdj</b>, <b>+BareEtc</b>, <b>+AV, +PASS, +UV, </b>and <b>+Redup</b>, whilst the special tags are <b>+caus_kan</b>, <b>+appl_kan</b>,</p><p><b>+Caus_i</b>, <b>+Appl_i</b>, <b>+Actor</b>, <b>+I</b><b>nstrument.</b></p><p>Tags like <b>+verb</b>, <b>+noun</b>, and <b>+adj </b>give the part of speech (POS) information for fully inflected words, whereas <b>+bareVerb</b>, <b>+bareNoun</b>, <b>+bareAdj</b>, and <b>+BareEtc </b>give the POS information for stems.</p><p>Other tags provide important linguistic informa­tion, such as voice, a grammatical category playing a central role in Indonesian syntax, e.g. <b>+AV, </b>which indicates active voice, <b>+PASS, </b>which indicates pas­sive voice, and <b>+uV,</b><b> </b>which is reserved for under-goer voice.</p><p>Furthermore, <b>+Redup </b>is a tag indicating redupli­cation. <b>+Caus_kan</b>, <b>+Caus_i</b>, <b>+Appl_kan</b>, and <b>+Appl_i </b>show whether words are causative or applicative with respect to their suffixes. The last two tags (<b>+actor </b>and <b>+instrument</b>) indicate either it is an ac­tor or an instrument that is being brought with the construction of that word.</p></subsection><subsection number="4.3" title="Morphotactic rules"><p>In designing a morphological analyser, morphotac-tic rules are crucial to model how two or more morphemes can be merged.</p><p>Based on (Alwi et al., 2003), the morphotactic rules for Indonesian can be classified into 13 classes. Ten of these classes are determined based on which suffixes are merged with the stem, while the other three are reduplication cases. The first ten classes can be identified as concatenative mor­phology, while the other three are nonconcatena-tive morphology.</p><p>In our design, the ten classes that belong to con-catenative morphology are the morphotactic rules for the prefixes <i>meN-, peN-, di-, per-, ber-, ter-, </i>and <i>ke-, </i>and the morphotactic rules for the suffixes <i>-an, -kan, </i>and <i>-i. </i>Some example cases that belong to these 10 classes are as follows:</p><p><i>• meN </i>+ stem(verb, noun, adjective or etc) + <i>kan</i></p><p>Verb. Example: <i>membersihkan (meN+bersih+kan). </i>In this example, the word <i>bersih </i>is an adjective. After merging with <i>meN-kan, </i>the resulting word is a verb.</p><p><i>• peN </i>+ <i>ber </i>+ stem(verb, noun, adjective or etc) + <i>an </i>-&gt; Noun. Example: <i>pembelajaran (peN+ber+ajar+an). </i>In this example, the word <i>ajar </i>is a verb. After merging with <i>peN-ber-an, </i>the resulting word is a noun.</p><p><i>• ke </i>+ <i>ber </i>+ stem(verb, noun, adjective or etc) + <i>an </i>-&gt; Noun. Example: <i>keberhasilan (ke+ber+hasil+an). </i>In this example, the word <i>hasil </i>is a noun. After merging with <i>ke-ber-an, </i>the resulting word is also a noun.</p><p>• stem(verb, noun or adjective) + <i>i </i>-&gt; Verb. Ex­ample: <i>terangi (terang+i). </i>In this example, the word <i>terang </i>is an adjective. After merging with <i>-i, </i>the resulting word is a verb.</p><p>The last three classes, which belong to the category of nonconcatenative morphology, are the morpho-tactic rules for full reduplication, partial reduplica­tion, and affixed reduplication. Some example cases that belong to these three classes are as fol­lows:</p><page local="4" global="145"/><p>• Full reduplication without affixation. Example: <i>buku-buku. </i>In this example, the word <i>buku-buku, </i>which is a noun, is generated from the word <i>buku, </i>which is also a noun.</p><p>• Full reduplication with <i>ke-an. </i>This case has the following rule: reduplication of <i>(ke</i>+stem (bare verb, adjective or etc)+an) -&gt; Noun. Example: <i>kekayaan-kekayaan </i>(reduplication of <i>ke+kaya+an). </i>In this example, the word <i>kaya </i>is an adjective. After the morphological proc­ess, the resulting word is a noun.</p><p>• Partial reduplication with <i>ber-. </i>This case has the following rule: <i>ber+ </i>reduplicated stem (bare verb) Verb. Example: <i>berlari-lari (ber+lari-lari). </i>In this example, the word <i>lari </i>is a verb. After the morphological process, the resulting word is also a verb.</p><p>• Affixed reduplication with <i>meN-. </i>This case has the following rule: stem (bare verb) +meN+stem (bare verb) Verb. Example: <i>tanam-menanam (tanam-meN+tanam). </i>In this example, the word <i>tanam </i>is a verb. After the process, the resulting word is also a verb.</p><p>During the stage of morphotactic rules, there are several steps that must be followed in order to complete the process. Those steps include the addi­tion of prefixes and preprefixes, the addition of stems and part-of-speech, the addition of suffixes, and the final processing of additional tags. After finishing all of these steps, the process moves on to the morphophonemic process.</p></subsection><subsection number="4.4" title="Morphophonemic rules"><p>All the rules that define how two or more mor­phemes can merge have been designed in morpho-tactic rules. However, the merging process is still not completed; hence we still have to define what changes have to be made after these morphemes merge. For these issues, we define morphophone-mic rules that define the phonetic changes that oc­cur.</p><p>In Indonesian, these rules can generally be di­vided into two groups. The first group consists of four rules that model the phonetic changes in stems, whereas the second group consists of seven rules that model the phonetic changes in affixes.</p><doubt alpha="66.2" length="68" tooSmall="False" monospace="0.0">Based on (Alwi et al., 2003), the four rules in the first group are:</doubt><p><i>• /k/ </i>replacement with <i>/ng/ </i>if the word starts with meN- or peN-. Example: <i>meN+kantuk </i><i>-&gt;</i><i> mengantuk.</i></p><p><i>• /s/ </i>replacement with <i>/ny/ </i>if the word starts with meN- or peN-. Example: <i>peN+sebaran </i><i>-&gt;</i><i> pen-yebaran.</i></p><p><i>• /p/ </i>replacement with <i>/m/ </i>if the word starts with meN- or peN-. Example: <i>peN+pakai </i><i>-&gt;</i><i> pe-makai.</i></p><p><i>• /t/ </i>replacement with <i>/n/ </i>if the word starts with <i>meN- </i>or peN-. Example: <i>meN+tertawakan </i><i>-&gt;</i><i> menertawakan.</i></p><p>Note that the rules above only change the stem-initial segment. To complete the replacement processes, the following seven rules from the sec­ond group are needed to get the correct prefix forms:</p><doubt alpha="60.4" length="48" tooSmall="False" monospace="0.0">• /N/deletion ifmeN-is followed by/l/, /m/, /n/,</doubt><doubt alpha="41.2" length="51" tooSmall="False" monospace="0.0">/r/, /y/, /w/, /t/, /s/, /p/, /k/or if there ispeN-</doubt><doubt alpha="34.5" length="55" tooSmall="False" monospace="0.0">followed by/l/, /m/, /n/, /r/, /d/, /w/, /t/, /s/, /p/,</doubt><p><i>/k/. </i>Example: <i>meN+lukis </i><i>-&gt;</i><i> melukis.</i></p><p><i>• /r/ </i>deletion if there is <i>ber-, ter-, </i>or <i>per- </i>fol­lowed by <i>/r/ </i>or the word that its first syllable ended with <i>/er/. </i>Example: <i>ber+runding </i><i>-&gt;</i><i> berunding.</i></p><p><i>• /N/ </i>replacement with <i>/n/ </i>if there is <i>meN- </i>fol­lowed by <i>/d/, /c/, /j/, /sy/ </i>or there is <i>peN- </i>fol­lowed by <i>/d/, /c/, /j/. </i>Example: <i>peN+jual </i><i>-&gt;</i><i> penjual.</i></p><p><i>• /N/ </i>replacement with <i>/m/ </i>if there is <i>meN- </i>or <i>peN- </i>followed by <i>/b/, /f/. </i>Example: <i>peN+buru </i><i>-&gt;</i><i> pemburu.</i></p><p><i>• /N/ </i>replacement with <i>/nge/ </i>if there is meN- fol­lowed by a word that has only one syllable. Example: <i>meN+rem </i><i>-&gt;</i><i> mengerem.</i></p><p><i>• /N/ </i>replacement with <i>/l/ </i>if there is <i>peN- </i>fol­lowed by the word <i>ajar. </i>Example: <i>peN+ajar </i><i>-&gt;</i><i> pelajar.</i></p><p><i>• /r/ </i>replacement with <i>/l/ </i>if there is ber- followed by the word <i>ajar. </i>Example: <i>ber+ajar </i><i>-&gt;</i><i> bela-</i></p><p><i>jar.</i></p><p>After all sub-processes invoked by the rules in the first group and the second group are parallelized, then the whole morphophonemic process is finished.<page local="5" global="146"/></p><p>The design of morphophonemic rules for redu­plication is very much the same as the one in af­fixation, since basically the morphophonemic process in reduplication occurs in the affixation part of reduplication.</p><p>However, several rules, in particular those that model morphophonemic processes where both af­fix and stem undergo changes, had to be revised to account for their behaviour when applied to redu­plicated forms. For example, the morphophonemic rules <i>'/k/ </i>replacement with <i>/ng/ </i>and <i>'/N/ </i>deletion', which work in tandem, were originally defined as <i>'/k/ </i>deletion' and <i>'/N/ </i>replacement with <i>/ng/\ </i>However, this approach would not have worked for cases involving reduplication. For example, the word <i>mengotak-ngotakkan </i>will not be analysed properly if the morphological analyser uses the rules <i>'/k/ </i>deletion' and <i>'/N/ </i>replacement with <i>/ng/'. </i>The word <i>mengotak-ngotakkan </i>is generated from the stem <i>kotak </i>that is modified with affixed redu­plication <i>meN-kan. </i>If <i>'/k/ </i>deletion' and <i>'/N/ </i>re­placement with <i>/ng/' </i>are being used, the word will become <i>mengotak-otakkan </i>which is not valid. This is why, for the sake of reduplication, the rule is changed to <i>'/k/ </i>replacement with <i>/ng/' </i>that is paral­lelized with <i>'/N/ </i>deletion'. Consequently, <i>mengo-tak-ngotakkan </i>can be properly generated.</p></subsection></section><section number="5" title="Implementation"><p>This Indonesian morphological analyser is imple­mented in <b>xfst </b>and <b>lexc </b>(Beesley &amp; Karttunen, 2003). The morphotactic rules are implemented in <b>xfst </b>while morphophonemic rules are imple­mented in <b>lexc.</b></p><subsection number="5.1" title="Morphotactic rules implementation"><p>The morphotactic rules can be illustrated as the finite-state automata shown in Figure 1. Valid In­donesian words, i.e. those that are constructed through legal morphological processes, are ac­cepted by the automata, whereas invalid words are rejected.</p><p>Starting from the root, each state defines the next possible state whilst emitting (or consuming) a certain symbol. In <b>lexc, </b>these states are called continuation classes. All continuation classes reachable from the root represent prefixes and <i>pre-</i>prefixes. The distinction between the two is neces­sary to encode the possible morphological varia­tions that take two prefixes, e.g. <i>memper-, diper-. </i>From there, the next continuation class is the <b>stem, </b>where the root word is emitted or consumed. This is subsequently followed by several classes repre­senting possible suffixes, but there are also <b>Redup1 </b>and <b>Redup2 </b>classes that appear before and after the suffixes. Their function is to handle reduplication (Section 5.3). Lastly, the <b>TagEmit </b>class processes all tags not yet handled by preceding classes (Sec­tion 4.<page local="6" global="147"/>2).</p><doubt alpha="100.0" length="4" tooSmall="False" monospace="0.0">Root</doubt><figure caption="Figure 1. Illustration of Process Flow"></figure><p>Throughout this morphotactic process, we ex­tensively employ <i>flag diacritics, </i>a crucial feature of <b>lexc </b>which approximates the power of feature structures, i.e. being able to specify certain con­straints to ensure that only valid paths of the net­work may be traversed. One benefit of this ap­proach is the maintenance of a compact network representation. There are three flag diacritics used in our model: positive setting <b>(@P.feat.val@), </b>required test <b>(@R.feat.val@), </b>and disallow test <b>(@D.feat.val@). </b>Using these diacritics, we are able to stipulate values and constraints of certain aspects, which must be consistent throughout a path.</p><p>For example, the prefix <i>meN- </i>may combine with the suffix <i>-kan </i>and <i>-i, </i>but not -an. We can repre­sent this with a feature, e.g. <b>pref</b>, which is set to <b>meN </b>at the appropriate (pre)prefix state <b>(@p.PREF.meN@). </b>Suitable tests are then added to the suffix states, e.g. if the current path went through the <i>meN- </i>prefix state, <b>@ </b><b>r.pref </b><b>.meN@ </b>stipulates that a suffix can be applied, whereas <b>@d.pref.</b><b>meN@ </b>prevents a suffix from being ap­plied.</p><p>To further exhibit the usage of flag diacritics in our morphological analyser, we provide an exam­ple of analysing the word <i>memukuli, </i>which con­sists of the prefix <i>meN-, </i>followed by the stem <i>pu-</i> <i>kul, </i>and the suffix <i>-i.</i><i> </i>Figure 2 shows the path through the network used to analyse this word.</p><p>Starting from the root, the analyser parses the prefix <i>meN- </i>in <i>memukuli </i>and appropriately sets the flag diacritic for the feature <b>pref. </b>This can be seen by the following <b>lexc </b>rule snippet:</p><doubt alpha="55.3" length="38" tooSmall="False" monospace="0.0">LEXICON PrefixMeN &lt;[0   .x. m e "AN "]</doubt><doubt alpha="59.1" length="22" tooSmall="False" monospace="0.0">"@P.PREF.meN@"&gt; Stems;</doubt><p>The <b>lexicon </b><b>PrefixMeN </b>line defines the state in the network that represents the meN- prefix. The next line defines a state transition where, by emit­ting or consuming the meN- prefix, the analyser can proceed to the <b>stems </b>continuation class. Addi­tionally, the <b>pref </b>flag is set to <b>meN. </b>In reality, the <b>PrefixMeN </b>state has five other possible transitions, but they are not presented here.</p><p>Subsequently, at the <b>stems </b>state, the analyser positively sets the flag diacritic for the feature <b>stem </b>with value <b>verb. </b>Various required and disal­low test flags will be called at the <b>Redupi, Suffixi, Redup2, </b>and <b>TagEmit </b>stages. However, for now, we focus on the <b>Suffixi </b>state, detailed in the fol­lowing <b>lexc </b>rule <b>snippet:</b></p><p><b>LEXICON Suffixi</b><page local="7" global="148"/></p><doubt alpha="48.5" length="132" tooSmall="False" monospace="0.0">&lt;"+Verb":i "+AV":0"@D.PREPREF@" "@D.REDUP@"["@R.STEM.Vrb@" |"@R.STEM.Nom@" |"@R. STEM.Adj@" ] "@R.PREF.meN@"   "@P.SUFF.i@"&gt; Redup2;</doubt><figure caption="Figure 2. The path formemukuli"></figure><p>This rule represents a valid transition from the <b>Suffixi </b>state to the <b>Redup2 </b>state by parsing the -i suffix and emitting the +<b>verb </b>and +<b>av </b>tags <b>("+Verb":i </b><b>"+av":o</b>). Additionally, the diacritic tests <b>"@d.prepref@" </b>and <b>"@d.redup@" </b>stipulate that this transition can only be taken if the <b>prepref </b>and <b>redup </b>flags are not set, the disjunction</p><doubt alpha="53.7" length="41" tooSmall="False" monospace="0.0">["@R.STEM.Vrb@"|"@R.STEM.Nom@"|"@R.STEM.A</doubt><p><b>dj@"] </b>states that the <b>stem </b>flag must be either <b>Vrb, Nom, </b>or <b>Adj</b>, and the <b>"@r. pref </b><b>.meN@" </b>test requires that the <b>pref </b>flag is set to <b>meN. </b>Finally, <b>"@p.suFF.i@" </b>states that the <b>suff </b>flag is set to <b>i.</b><b> </b>Should any of these tests fail, the transition cannot be taken. If no other alternative paths are available to reach the <b>End </b>state, the word is rejected.</p></subsection><subsection number="5.2" title="Reduplication Process"><p>As discussed above in Sections 2 and 4, Indonesian morphology includes the non-concatenative proc­ess of reduplication. Handling this with pure regu­lar grammars as implemented by finite state auto­mata is very difficult. Thus, we employ the <b>com-pile-replace </b>feature in <b>xfst </b>(Beesley &amp; Karttunen, 2000). This feature allows the repetition of arbi­trarily complex sublanguages by specifying the brackets <b>"A[" </b>and <b>,,A]" </b>to mark the domain of re­duplication. The right bracket is also augmented with <b>A2 </b>to indicate duplication; thus, the full anno­tation is <b>"A[" </b>and <b>"A2A]". </b>Given this network definition, xfst compiles and post-processes these annotations to produce a new network where the appropriate reduplications have been carried out. For example, <b>"A[bukuA2A]" </b>would eventually be compiled to become <b>bukubuku.</b></p><p>Thus, the idea is to insert the <b>"A["</b><b> </b>and <b>"A2A]"</b><b> </b>annotations in the right places. Due to the various types of reduplication in Indonesian (see Section 2), the rules for reduplication can be found at the <b>Re­dup </b>(pre)prefix state and the <b>Redupi </b>and <b>Redup2 </b>states. The <b>Redup </b>prefix state emits the opening <b>"A["</b><b> </b>brackets and sets an appropriate flag as a reminder that the closing brackets are required. The <b>Redupi </b>state is responsible for closing partial and affixed reduplications, i.e. where the suffix is not included in the reduplication, whilst the <b>Redup2 </b>state is responsible for closing full reduplication, i.e. where the suffix is part of the reduplication process. Both <b>Redupi </b>and <b>Redup2 </b>states check for the value of the <b>redup </b>flag as set by the <b>Redup </b>pre­fix state.</p></subsection><subsection number="5.3" title="Morphophonemic rules implementation"><p>The full transducer composes the morphotactic and morphophonemic rules. As a result, the output of the morphotactic rules implementation serves as the input to the morphophonemic rules implemen­tation.</p><p>From the example in Section 5.1 regarding the implementation of morphotactic rules, it can be observed that the string <i>me^Npukuli </i>has been pro­duced. However, the morphological analysis proc­ess is still incomplete since <i>me'Npukuli </i>is not a valid word in Indonesian. In order to complete the process, the morphophonemic rules must be im­plemented. Since the full transducer composes the morphotactic and morphophonemic rules, the word <i>me^Npukuli </i>becomes the input for the morphopho-nemic process.</p><p>The implementation of morphophonemic rules differs slightly from the implementation of mor-photactic rules. For morphotactic rules, there are several steps that can be illustrated as a flow of process. However, the implementation of morpho-phonemic rules generally implies the rules itself. Each rule is defined as a replacement rule that will collaborate with other rules through composition or parallelization.</p><p>All rules in Section 4.4 have been implemented. However, due to space constraints, we will only explain the implementation of rule '<b>rg4', </b>which encodes <i>/N/ </i>deletion parallelized with four pho­netic stem changes:</p><doubt alpha="41.1" length="73" tooSmall="False" monospace="0.0">defineRG4 %-"-N-&gt;0I| [m e| [pe] ]_"lbraces" * [I|m|n|r|y |w| 11s |p |k] „</doubt><doubt alpha="53.4" length="133" tooSmall="False" monospace="0.0">t-&gt;n|I[m |p]e %AN "lbraces"*_p-&gt;m|I[m|p]e "lbraces"*_s-&gt;n y|I[m|p]e        "lbraces"*_lsk-&gt;n g|I[m|p]e "lbraces"*_; ! pushdefinedRG4;</doubt><p>This rule actually consists of five rules that are parallelized. The first rule is <i>/N/ </i>deletion, where the nasal symbol <i>/AN/ </i>is omitted if it is preceded by <i>/me/ </i>or <i>/pe/ </i>and followed by either of the pho­nemes <i>/l/, /m/, /n/, /r/, /y/, /w/, /t/, /s/, /p/, </i>or <i>/k/.</i></p><p>The next four rules are the phonetic stem changes, which apply to stem words preceded by the morpheme <i>/meAN/ </i>or <i>/peAN/, </i>and whose initial phoneme is one of <i>/t/, /p/, /r/, /s/, </i>or <i>/k/. </i>More spe­cifically, the phoneme <i>/t/ </i>will transform into <i>/n/, /p/ </i>will transform into <i>/m/, </i>/s/ will transform into <i>/ny/, </i>and finally the phoneme <i>/k/ </i>will transform into <i>/ng/.</i></p><p>All morphophonemic rules that implement the various processes are composed into one large rule, from which the morphophonemic transducer net­work can be constructed.<page local="8" global="149"/></p><p>The morphophonemic rule that applies to the word <i>meANpukuli </i>used as an example in Section 5.1 would be the above <i>"/N/ </i>deletion parallelized with four phonetic stem changes", because the word <i>meANpukuli </i>fulfils the requirements of that rule. As can be seen from the implementation, to get the <i>"/N/ </i>deletion process" and <i>"/p/ </i>replacement with <i>/m/ </i>process" <i>("/p/ </i>replacement with <i>/m/ </i>proc­ess" is one of those four phonetic rules), in that word there must appear the phoneme <i>/AN/ </i>preceded by <i>/me/ </i>or <i>/pe/ </i>and followed by phoneme <i>/l/, /m/, /n/, /r/, /y/, /w/, /t/, /s/, /p/, </i>or <i>/k/ </i>and it also must have the phoneme <i>/t/, /p/, /r/, /s/, </i>or <i>/k/ </i>preceded by the morpheme <i>/meAN/ </i>or <i>/peAN/. </i>Since the word <i>meANpukuli </i>fulfils these requirements, <i>/AN/ </i>will be omitted and <i>/p/ </i>will transform into <i>/m/ </i>so that the word <i>meANpukuli </i>will transform into the word <i>memukuli. </i>With the word <i>memukuli </i>being gener­ated, the process is finished and the word <i>memu­kuli </i>is recognised as a valid word in Indonesian.</p></subsection></section><section number="6" title="Evaluation"><p>To assess our implemented system, we ran some test cases in the form of words extracted from an electronic version of the <i>Kamus Besar Bahasa Indonesia'</i>'. We tested the implementations of the morphotactic and morphophonemic rules separately. To evaluate the ability of the analyser to both accept valid forms and reject invalid forms, the test cases included both valid and invalid morpheme combinations. Executing all of the test cases, we obtained the results shown in Table 1, which presents the results of morphotactic test cases, and Table 2, which presents the results of morphophonemic test cases. The 'Analysis' column displays the results of test cases where the surface form of an Indonesian word was provided as input, and our system was tasked with parsing the morphological structure. For example, given the word <i>memukul, </i>our system should return pukul<b>+V</b><b>erb</b><b>+AV. </b>On the other hand, the 'Synthesis' column concerns the opposite situation, i.e. test cases where the input is the string of morphological tags, and our system was tasked with generating the fully inflected form.</p><p>We classify the test case results into three cate­gories. The first category indicates that our system returned exactly one correct analysis or synthesis for a valid test case, or does not produce anything for an invalid test case. The second category is that, when given a valid test case, the system produces several answers, one of which is the desired result. Finally, the last category is seen when the system fails to analyse or synthesize a valid test case, or incorrectly produces an answer for an invalid test case.</p><p>From the tables, we can observe that the results for analysis are more accurate than for synthesis, where the system tends to produce more than one result. For example, our system correctly provides the single analysis for <i>memukul. </i>However, when trying to synthesize <i>pukul</i><b>+V</b><b>erb</b><b>+AV, </b>it produces other alternatives, e.g. <i>memukulkan, memperpukuli, mengepukulkan, </i>etc. This suggests the need for a more refined set of morphological feature tags.</p></section><section number="7" title="Summary"><p>We have presented the design of an Indonesian morphological analyser that provides a detailed analysis of its rich affixation process using the two-level morphology approach, implemented us­ing <b>xfst </b>and <b>lexc. </b>Our approach is able to handle reduplication, a non-concatenative morphological process. Our (not very rigorous) evaluation shows that the implementation is generally able to encode the rules of the various morphological processes.</p><footnote label="3"> http://www.pusatbahasa.diknas.go.id/kbbi</footnote><table caption="Table 1. Results of morphotactic test cases" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Result</b></p></td><td class="cell"><p><b>Analysis</b></p></td><td class="cell"><p><b>Synthesis</b></p></td><td class="cell"><p><b>Total</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1. Correct result</p></td><td class="cell"><p>103</p></td><td class="cell"><p>43</p></td><td class="cell"><p>146</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2. Several results, including correct</p></td><td class="cell"><p>46</p></td><td class="cell"><p>106</p></td><td class="cell"><p>152</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3. Incorrect result</p></td><td class="cell"><p>3</p></td><td class="cell"><p>3</p></td><td class="cell"><p>6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Total</b></p></td><td class="cell"><p><b>152</b></p></td><td class="cell"><p><b>152</b></p></td><td class="cell"><p><b>308</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 2. Results of morphophonemic test cases" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Result</b></p></td><td class="cell"><p><b>Analysis</b></p></td><td class="cell"><p><b>Synthesis</b></p></td><td class="cell"><p><b>Total</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>1. Correct result</p></td><td class="cell"><p>51</p></td><td class="cell"><p>21</p></td><td class="cell"><p>72</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>2. Several results, including correct</p></td><td class="cell"><p>6</p></td><td class="cell"><p>36</p></td><td class="cell"><p>42</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>3. Incorrect result</p></td><td class="cell"><p>1</p></td><td class="cell"><p>1</p></td><td class="cell"><p>2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Total</b></p></td><td class="cell"><p><b>58</b></p></td><td class="cell"><p><b>58</b></p></td><td class="cell"><p><b>116</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="9" global="150"/></section><references><p>Adriani, Mirna, Jelita Asian, Bobby Nazief, Seyed Mohammad Tahaghoghi and Hugh Williams. 2007. Stemming Indonesian: A Confix-Stripping Ap­proach. <i>ACM Transactions on Asian Language In­formation Processing, </i>Vol. 6, No. 4.</p><p>Alwi, Hasan, Soenjono Sardjowidjojo, Hans Lapoliwa, and Anton Moeliono. 2003. <i>Tata Bahasa Baku Ba-hasa Indonesia Edisi Ketiga. </i>Pusat Bahasa dan Balai Pustaka, Jakarta.</p><p>Beesley, Kenneth and Lauri Karttunen. 2000. Finite-State Non-Concatenative Morphotactics. In <i>Proceed­ings of SIGPHON-2000, </i>August 2000, Luxembourg, pp 1-12.</p><p>Beesley, Kenneth and Lauri Karttunen. 2003. <i>Finite State Morphology. </i>CSLI Publication, Stanford.</p><p>Dalrymple, Mary, Maria Liakata, and Lisa Mackie. 2006. Tokenization and Morphological Analysis for Malagasy. In <i>Computational Linguistics and Chinese Language Processing Vol.11, No.4, December 2006, pp 315-332.</i></p><p>Gordon, Raymond (ed). 2005. <i>Ethnologue: Languages of the World 15th edition. </i>SIL International, Dallas, USA.</p><p>Hartono, Hendra. 2002. <i>Pengembangan Pengurai Mor-fologi untuk Bahasa Indonesia dengan Model Mor-fologi Dua Tingkat Berbasiskan PC-KIMMO. </i>Un­dergraduate thesis, Faculty of Computer Science, University of Indonesia</p><p>Jurafsky, Daniel and James Martin. 2000. <i>Speech and Language Processing: An Introduction to Natural Language Processing, Computational Lingusitic, and Speech Recognition. </i>Prentice Hall, New Jersey.</p><p>Koskenniemi, Kimmo. 1983. <i>Two-Level Morphology: A General Computational Model for Word-Form Rec­ognition and Production. </i>Publication 11, University of Helsinki, Department of General Linguistics, Hel­sinki.</p><p>Siregar, Neil. 1995. <i>Pencarian Kata Berimbuhan pada Kamus Besar Bahasa Indonesia dengan mengguna-kan Algoritma Stemming. </i>Undergraduate thesis, Fac­ulty of Computer Science, University of Indonesia.</p><p>Sneddon, James. 2003. <i>The Indonesian Language: Its History and Role in Modern Society. </i>UNSW Press, Sydney, Australia.</p></references></body></article>