<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TEI xmlns="http://www.tei-c.org/ns/1.0" xmlns:ns2="http://www.tei-c.org/ns/Examples">
    <teiHeader>
        <fileDesc>
            <titleStmt>
                <title>eBonsai: An integrated environment for annotating treebanks</title>
            </titleStmt>
        </fileDesc>
    </teiHeader>
    <text>
        <front>
            <div type="abs">
                <head>Abstract</head>
                <p>Syntactically annotated corpora (treebanks) play an important role in recent statistical natural language processing. However, building a large treebank is labor intensive and time consuming work. To remedy this problem, there have been many attempts to develop software tools for annotating treebanks. This paper presents an integrated environment for annotating a treebank, called eBonsai. eBonsai helps annotators to choose a correct syntactic structure of a sentence from outputs of a parser, allowing the annotators to retrieve similar sentences in the treebank for referring to their structures.</p>
            </div>
        </front>
        <body>
            <div>
                <p/>
            </div>
            <div1>
                <head xml:id="sec1">Introduction</head>
                <p>Statistical approach has been a main stream of natural language processing research for the last decade. Particularly, syntactically annotated corpora (treebanks), such as Penn Treebank (Marcus et al., 1993), Negra Corpus (Skut et al., 1997) and EDR Corpus (Jap, 1994), contribute to improve the performance of morpho-syntactic analysis systems. It is notorious, however, that building a large treebank is labor intensive and time consuming work. In addition, it is quite difficult to keep quality and consistency of a large treebank. To remedy this problem, there have been many attempts to develop software tools for annotating treebanks (Plaehn and Brants, 2000; Bird et al., 2002).</p>
                <p>This paper presents an integrated environment for annotating treebanks, called eBonsai. Figure 1 shows a snapshot of eBonsai. eBonsai first performs syntactic analysis of a sentence using a parser based on GLR algorithm (MSLR parser) (Tanaka et al., 1993), and provides candidates of its syntactic structure. An annotator chooses a correct structure from these candidates. When choosing a correct structure, the annotator can consult the system to retrieve already annotated similar sentences to make the current decision. Integration of annotation and retrieval is a significant feature of eBonsai.</p>
                <p>To realize the tight coupling of annotation and retrieval, eBonsai has been implemented as the following two plug-in modules of an universal tool platform: Eclipse (The Eclipse Foundation, 2001). • Annotation plug-in module: This module</p>
                <p>helps to choose a correct syntactic structure</p>
                <p>from candidate structures. • Retrieval plug-in module: This module re-</p>
                <p>trieves similar sentences to a sentence in</p>
                <p>question from already annotated sentences in</p>
                <p>the treebank.</p>
                <p>These two plug-in modules work cooperatively in the Eclipse framework. For example, information can be transferred easily between these two modules in a copy-and-past manner. Furthermore, since they are implemented as Eclipse plugin modules, these functionalities can also interact with other plug-in modules and Eclipse native features such as CVS. Figure 1: A snapshot of eBonsai Figure 2: A workflow of annotation using eBonsai</p>
            </div1>
            <div1>
                <head xml:id="sec2">Annotating treebanks</head>
                <p>Figure 2 shows a workflow of annotating a treebank using eBonsai. 1. An annotator picks a sentence to annotate</p>
                <p>from plain-text corpora. 2. The MSLR parser (Tanaka et al., 1993) per-</p>
                <p>forms syntactic analysis of the sentence. 3. The annotator chooses a correct syntactic</p>
                <p>structure from the output of the parser. If</p>
                <p>necessary, retrieval of structures in the tree-</p>
                <p>bank is available in this step. 4. The annotator adds the chosen syntactic</p>
                <p>structure to the treebank.</p>
                <p>The coverage of Japanese grammar used in the MSLR parser is fairly wide. The number of grammar rules of the current system is almost 3,000. That means we have a lot of outputs as a result of syntactic analysis in step 2. These structures are represented in terms of a special data structure called packed shared forest (PSF) (Tomita, 1986). The main role of eBonsai is supporting annotators to choose a correct one from a lot of candidate structures in step 3.</p>
            </div1>
            <div1>
                <head xml:id="sec3">Annotation plug-in module</head>
                <div2>
                    <head xml:id="sec3.1">Overview</head>
                    <p>The annotation plug-in module helps to choose a correct syntactic structure from a set of structures represented by a packed shared forest which is an output of the MSLR parser.</p>
                    <p>Since there are generally so many syntactic structures given by the parser, it is impractical to findacorrectonebycheckingallofthem. Theannotation plug-in module shows a single structure at a time as shown in figure 1, and asks annotators to specify a constraint. The system reflects the constraint immediately by filtering out the structures violating it. This process is repeated until a single correct structure is identified. The constraints which can be specified by annotators are following two: 1. Annotators can specify a destination con-</p>
                    <p>stituent of a dependent constituent. 2. Annotators can specify a correct label of a</p>
                    <p>node. This plug-in module is a reimplementation of an annotation tools developed by Okazaki (Okazaki et al., 2001) in an Eclipse framework.</p>
                </div2>
                <div2>
                    <head xml:id="sec3.2">Example of annotation</head>
                    <p>Take the following Japanese sentence for an example to explain the usage of the annotation plugin module. 警告の黄色の用紙を (yellow paper for warning-ACC) 郵便受けに (mailboxDAT) 入れておいたが (put, but) ，まだ 支払われていない (not being paid yet). (I put a yellow paper for warning in the mailbox, but it is not paid yet.) 1. An annotator double-clicks a PSF file name</p>
                    <p>in Eclipse “Navigator” (a left window) to</p>
                    <p>pick up a sentence to annotate. 2. A new window opens and one of the struc-</p>
                    <p>tures is shown in terms of a syntactic tree (a</p>
                    <p>right window). This window is called “Annotation editor”. The notation “1/9” in the left-top part of the window indicates that the presented structure is the first one out of nine. 3. A red node (e.g. “ &lt; 動詞句 - 用 - 無 &gt; ” (verb</p>
                    <p>phrase)) indicates that this node has other</p>
                    <p>possible label names. 4. Clicking a right button on the red node</p>
                    <p>makes a list of possible label names pop up</p>
                    <p>as shown below. 5. Annotators can choose a correct label name</p>
                    <p>of the node in the list. In this case, label “ &lt;</p>
                    <p>名詞句 - サ変 - 無 &gt; ” will be selected. 6. Then label “ &lt; 動詞句 - 用 - 無 &gt; ” (verb</p>
                    <p>phrase) changes to “ &lt; 名詞句 - サ変 - 無 &gt; ”</p>
                    <p>(noun phrase) in the tree and its color be-</p>
                    <p>comes black at the same time. Black label</p>
                    <p>names indicate that there is no other possi-</p>
                    <p>ble label for this node.</p>
                    <p>Now, the number of structures shown in the</p>
                    <p>left-top part of Annotation editor decreases</p>
                    <p>to 3. 7. A green node (e.g. “ &lt; 補足節 - を - 無 - 無</p>
                    <p>&gt; ”) indicates the constituent governed by</p>
                    <p>that node can depend on more than one con-</p>
                    <p>stituent. 8. Clicking a right button on node “ &lt; 補足節 -</p>
                    <p>を - 無 - 無 &gt; ” makes a list of destinations of</p>
                    <p>dependency pop up as shown below. 9. Annotators can choose a correct destination</p>
                    <p>inthelist. Inthiscase,“ 郵便受けに入れてお</p>
                    <p>いた (put (a yellow paper) in the mailbox )”</p>
                    <p>will be selected. 10. At this moment, all nodes have turned into</p>
                    <p>black and the number of structure becomes</p>
                    <p>1. That means the annotation of this sentence</p>
                    <p>has been finished. Figure 3: A popup menu of Annotation editor</p>
                </div2>
                <div2>
                    <head xml:id="sec3.3">Other features</head>
                    <p>The following features are also implemented. • Unlimitedly repeatable Undo/Redo. It is</p>
                    <p>possible to undo/redo after saving results by</p>
                    <p>using the popup menu. (figure 3). • Viewing other structures. Items [Previous</p>
                    <p>tree] and [Next tree] in the popup menu</p>
                    <p>shows different structures. • Folding constituents. Clicking a right but-</p>
                    <p>ton on a node and selecting item [Switch</p>
                    <p>folding] makes the structure under the node</p>
                    <p>folded. Selecting the same item again un-</p>
                    <p>folds the structure. • Copying a part of a structure to the retrieval</p>
                    <p>plug-in module. Item [Copy to search] in</p>
                    <p>the popup menu copies a selected structure</p>
                    <p>to the query input window. This feature will</p>
                    <p>be described in detail in the later section.</p>
                </div2>
            </div1>
            <div1>
                <head xml:id="sec4">Retrieval plug-in module</head>
                <div2>
                    <head xml:id="sec4.1">Overview</head>
                    <p>During the course of annotation, annotators usually put constraints to narrow down to a correct structure considering the meaning of a sentence. However, there are cases in which it is difficult to pin down a correct one by referring to only that sentence. Annotators can consult the system to retrieve the similar structure of sentences in the treebank. The retrieval plug-in module provides annotators such functionality. The retrieval plugin module receives a syntactic structure as a query and provides a list of sentences which include the given structure.</p>
                    <p>The retrieval plug-in module has been realized with the method proposed by Yoshida (Yoshida et al., 2004). The method is based on Yoshikawa’s method (Yoshikawa et al., 2001) which was originally proposed for handling XML documents effectively by using relational database (RDB) systems. Yoshida adopted Yoshikawa’s method to deal with syntactic structures in the database. Since an XML document can be represented as a tree, Yoshikawa’s method is also applicable to deal with syntactic structures. Figure 4: An input query for retrieval</p>
                    <p>An input query is given as a tree as shown in Figure 4. The structure is then translated into a SQL query and the retrieval is performed.</p>
                    <p>A query involving a large number of nodes generates a longer SQL query, thus degrades retrieval speed significantly. Yoshida proposed to decompose an input query into a set of subtrees, and to translate each subtree into a SQL query.</p>
                </div2>
                <div2>
                    <head xml:id="sec4.2">Example of structure retrieval</head>
                    <p>1. An annotator puts a query tree in the query</p>
                    <p>input window (upper-left window of Fig-</p>
                    <p>ure 5). The query can be modified by the</p>
                    <p>following way. • A node label can be changed by click-</p>
                    <p>ing a left button on the node and putting</p>
                    <p>a new label in the input area. A label</p>
                    <p>can contain a wild card character “%”. • A child node can be added by clicking</p>
                    <p>a right button on a node and selecting</p>
                    <p>menu item [Add child]. 2. Clicking a right button in the query input</p>
                    <p>window and selecting a menu item starts re-</p>
                    <p>trieval. There are four types of search. • Menu item [Search] retrieves sentences</p>
                    <p>containing a structure which is exactly</p>
                    <p>the same as the query. • Menu item [Partial search] retrieves</p>
                    <p>sentences with less rigid condition than</p>
                    <p>item [Search]. It allows some child</p>
                    <p>nodes missing from the query. Figure 5: An example of structure retrieval • Menu item [Narrow search] searches in</p>
                    <p>the previously retrieved sentences in-</p>
                    <p>stead of in the entire treebank. • Menu item [Partial narrow search] is</p>
                    <p>the combination of [Partial search] and</p>
                    <p>[Narrow search]. 3. Retrieval results are shown in the retrieval</p>
                    <p>result list window (a left-bottom window in</p>
                    <p>Figure 5). 4. Clicking a sentence in the list shows the de-</p>
                    <p>tailed structure of the sentence in the detail</p>
                    <p>window (a right window of Figure 5). A part</p>
                    <p>of the structure matching with the query is</p>
                    <p>colored with blue. 5. If there is more than one substructure match-</p>
                </div2>
            </div1>
            <div1>
                <head xml:id="sec5"></head>
                <p>Since both the annotation plug-in module and the retrieval plug-in module are implemented as ing with the query in a sentence, the system shows the total number of matching parts, and the identifier of the currently colored part by number. Menu items [Previous match] and [Next match] allows annotators to move the other matching parts. Interplay between annotation and retrieval Eclipse plug-ins, they can easily exchange information each other. Thanks for this feature, annotators can copy a part of syntactic structures shown in Annotation editor and submit it to the retrieval module as a query. This can be done by the following procedure. References S. Bird, X. Maeda, K. Ma, H. Lee, B. Randall, and S. Za-</p>
                <p>yat. 2002. TableTrans, MultiTrans, InterTrans and Tree-</p>
                <p>Trans: Diverse tools built on the annotation graph toolkit.</p>
                <p>In Proceedings of the Third International Conference on</p>
                <p>Language Resources and Evaluation (LREC 2002) , pages</p>
                <p>364–370. 1. Dragging a mouse pointer over the area cov-</p>
                <p>ering a target syntactic structure selects the</p>
                <p>structure, of which color changes to blue. 2. Clicking a right button in Annotation editor</p>
                <p>makes a command list pop up, and selecting item [Copy to search] copies the selected structure to the query input window. 3. The annotator can modify the query if</p>
                <p>needed. 4. Clicking a right button in the query input</p>
                <p>window makes a command list pop up and</p>
                <p>selecting one of search commands performs</p>
                <p>a search.</p>
            </div1>
            <div1>
                <head xml:id="sec6"></head>
                <p>This paper introduced eBonsai, an integrated environment for annotating treebanks. eBonsai was implemented as two plug-in modules of Eclipse: the annotation plug-in module for choosing a correct syntactic structure from outputs of a parser, and the retrieval plug-in module for retrieving sentences including similar structure to a query in the treebank. These two modules are tightly coupled, thus during the course of annotation, annotators can refer to already annotated sentences in the treebank by using the retrieval module. This helps to annotate difficult cases.</p>
                <p>The future work includes the following issues. Conclusion and Future Work • Introduction of a project management func-</p>
                <p>tionality by coupling with such as the CVS</p>
                <p>system. • Further improvement of the interface. • Automatic presentation of reference sen-</p>
                <p>tences with their structure without annota-</p>
                <p>tors’ explicit retrieval. • Functionality to share know-how among an-</p>
                <p>notators. Japan Electronic Dictionary Research Institute, 1994.</p>
                <p>”Electronic Dictionary User’s Manual 2.1” . M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz. 1993.</p>
                <p>Building a large annotated corpus of english: The Penn</p>
                <p>Treebank. Computational Linguistics , 19(2):313–330. A. Okazaki, K. Shirai, T. Tokunaga, and H. Tanaka. 2001.</p>
                <p>A syntactic annotation tool with user navigation. In Pro-</p>
                <p>ceedings of the 15th Annual Conference of Japanese So-</p>
                <p>ciety for Artificial Intelligence . O. Plaehn and T. Brants. 2000. Annotate – An effi-</p>
                <p>cient interactive annotation tool. In Proceedings of the</p>
                <p>Sixth Conference on Applied Natural Language Process-</p>
                <p>ing ANLP-2000 . W. Skut, B. Krenn, T. Brants, and H. Uszkoreit. 1997. An</p>
                <p>annotation scheme for free word order languages. In Pro-</p>
                <p>ceedings of the 5th Conference on Applied Natural Lan-</p>
                <p>guage Processing , pages 88–95. H. Tanaka, T. Tokunaga, and M. Aizawa. 1993. Integra-</p>
                <p>tion of morphological and syntactic analysis aased on LR</p>
                <p>parsing algorithm. In Proceedings of International Work-</p>
                <p>shop on Parsing Technologies , pages 101–109. The Eclipse Foundation. 2001. Eclipse official site.</p>
                <p>http://www.eclipse.org/. M. Tomita. 1986. Efficient Parsing for Natural Language .</p>
                <p>Kluwer Academic Publisher. K. Yoshida, T. Hashimoto, T. Tokunaga, and H. Tanaka.</p>
                <p>2004. Retrieving annotated corpora for corpus annota-</p>
                <p>tion. In Proceedings of the Fourth International Con-</p>
                <p>ference on Language Resources and Evaluation (LREC</p>
                <p>2004), pages 1775–1778. M. Yoshikawa, T. Amagasa, T. Shimura, and S. Uemura.</p>
                <p>2001. Xrel: A pathbased approach to storage and re-</p>
                <p>trieval of xml documents using relational database. ACM</p>
                <p>Transactions on Internet Technology , 1(1).</p>
            </div1>
        </body>
        <back/>
    </text>
</TEI>
