<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="6"/><title>Mink: An Incremental Data-Driven Dependency Parser with Integrated Conversion to Semantics</title><author surname="Cantrell" givenname="Rachael"><org  name="Anna University" country="India" city="Chennai"/></author></firstpageheader><frontmatter><p>Mink: An Incremental Data-Driven Dependency Parser with Integrated Conversion to Semantics</p><p>Rachael Cantrell Indiana University Bloomington, Indiana <i>rcantrel@indiana. edu</i></p></frontmatter><abstract>While there are several data-driven dependency parsers, there is still a gap with regards to in­crement ality. However, as shown in Brick and Scheutz [3], incremental processing is necessary in human-robot interaction. As is shown in Nivre et al. [12], dependency parsing is well-suited for mostly incremental processing. However, there is as of yet no dependency parser that combines syntax and semantics by including traditional de­pendency parsing, CCG tagging, and lambda-logical structures in one fast, accurate applica­tion suitable for embodied natural language pro­cessing. This paper addresses that gap by introducing Mink, an incremental data-driven dependency parser with integrated conversion to semantics. We show that Mink is comparable to similar but non-incremental parsers, and that it succeeds at performing some semantic analysis. Keywords incremental parsing, dependency parsing, CCG, semantics con­version </abstract></header><body><section number="1" title="Introduction"><p>Dependency parsing has gained in popularity in the last few years. However, there is still a gap with re­gards to incrementality. As is shown in Nivre et al. [12], dependency parsing is well-suited for mostly in­cremental processing. However, there is as of yet no dependency parser that combines syntax and seman­tics by including traditional dependency parsing, CCG tagging, and lambda-logical structures in one fast, ac­curate application suitable for embodied natural lan­guage processing.</p><p>Our motivation for creating Mink was to meet the need for incremental processing in robots that must interact with humans using natural language. Most natural language processing (NLP) is performed on written texts. However, in embodied NLP systems such as robots, the system needs to understand incom­ing speech from humans. As discussed by Brick and Scheutz [3], humans do not wait for a full sentence in order to begin processing the sentence, parsing it, and resolving references. We immediately begin to process whatever we can as soon as we can, which allows us to look at referents, or reach for things before we know where we'll be told to put them, etc. Because humans have this capability, we become impatient when deal­ing with conversation partners that are slower. Mink takes us one step closer toward achieving a level of in­cremental processing in robots that will allow humans and robots to communicate easily and naturally.</p><p>Because we are working to help our robots "under­stand" verbal input, we are not interested in the parses themselves, but rather in what they can tell us se-mantically. Thus we have integrated semantic output into the system itself. Mink outputs three different types of information: dependency arcs, combinatorial categorial grammar (CCG) tags, and, when possible, lambda-logical semantic conversions.</p><p>In this paper, we evaluate Mink based on accuracy and speed. We evaluate the CCG tagging and seman­tic conversion capabilities together, by using them to attempt to correctly classify utterances into either sen­tence, question, or command. We chose these simple types because they are discourse types that are, in gen­eral, distinguishable from the syntax of the sentence.</p><p>In section 2, we discuss currently available depen­dency parsers, CCG taggers and parsers, and relevant work on semantics, particularly the interface between syntax and semantics. In section 3, we discuss the de­tails of our parser. In section 4, we evaluate our parser. Finally, in section 5, we discuss how we are currently able to use the parser and where we plan to go with it.</p></section><section number="2" title="Related Work"><p>There are three main bodies of knowledge from which this project draws: dependency parsing, CCG tagging and parsing, and semantic representation of parser output.</p><subsection number="2.1" title="Dependency Parsing"><p>One often-used data-driven dependency parser is MaltParser [13], a shift-reduce dependency parser that uses support vector machines to deduce, from train­ing examples, what next action to take in parsing a sentence. Nivre [12] discusses the potential for incre­mentality in MaltParser. It is determined that the algorithm is the best suited for incremental parsing that has been developed to date; however an incre­mental version of MaltParser has not been developed by those researchers.</p><p>MSTParser (cf. McDonald et al. [10]) approaches the problem of finding dependency trees as one of find<i>Student Research Workshop, RANLP 2009 - Borovets, Bulgaria, </i>pages 6-11 ing maximum spanning trees.<page local="2" global="7"/> While this method is shown to be successful, the fact that it searches the entire space of possible dependency trees renders it inefficient for our purposes.</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">6</doubt><p>Johannsen and Nugues [8] combine aspects of Malt­Parser and MSTParser to form a dependency parser that, rather than maximizing the probability of each individual action (like MaltParser), maximizes the probability over a complete sentence. While this gives good results, it is a step away from incrementality.</p></subsection><subsection number="2.2" title="CCG Tagging and Parsing"><p>An utterance that is tagged with CCG tags con­tains more information about the parse tree of the utterance than one tagged with simple POS tags. This is because each tag contains information about both the POS (the return type), and about the to­ken's children (the argument types). In an utterance with just one possible parse, this is enough informa­tion to describe the correct parse tree. For example: <i>the^jp child^p ateg^p^p (</i><i>inp/np </i><i>snack^p.</i></p><p>There is just one way to create a tree from this infor­mation, so, even if this utterance came from a tagger rather than a parser, it is essentially parsed already. However, many utterances have ambiguity, which a tagger is not intended to resolve. For our purposes, in most cases, a tagger gives us the information we need; therefore we consider both taggers and parsers as being of equal utility to us.</p><p>There are two basic types of CCG taggers/parsers, statistical and rule-based. An example of a small rule-based system is found in [1]. It looks up the CCG tag(s) of each word and builds a parse based on combi­natory rules. Another rule-based system is OpenCCG, a freely-available CCG parser written in Java. While it has been widely used in many projects<footnote anchor="1"/>, the nature of a rule-based system limits its robustness, particu­larly for processing spoken data, which often contains unknown words and disfluencies.</p><p>There are several statistic systems, which we would prefer for greater robustness. For example, Clark [5] uses a maximum-entropy based statistical tagger to select CCG tags based on context.</p><p>The problem with statistical systems, for our pur­poses, is that they are based on probabilities of full tags, rather than on the parts.</p><p>Because we wanted to be able to build the tags by combining individual parts rather than selecting from a set of full CCG tags, we decided to use a dependency parser to build the tags based on dependency arcs.</p></subsection><subsection number="2.3" title="Semantic Representation"><p>Che et al. [4] integrates syntax and semantics. They parse the sentence using MSTParser, then use a se­ries of classifiers to identify predicates, classify them according to senses, and assign semantic roles to dif­ferent elements in the sentence. However, they do not integrate the results into any sort of logical framework.</p><p>Bos et al. [2] discusses a method of converting out­put from a wide-coverage CCG parser into semantic representation using lambda calculus, from which our semantic conversion method draws heavily.</p><footnote label="1">http ://comp.ling.utexas.edu/wiki/doku.php/openccg/ projects_using_openccg</footnote><p>Lambda conversion has been shown to be seman-tically useful in robotic NLP systems, cf. Gold and Scassellati [6].</p><p>3 An incremental architecture for dependency parsing and integrated semantics conver­sion</p></subsection><subsection number="3.1" title="Parsing Algorithm"><p>We used the Nivre algorithm, which was previously im­plemented in MaltParser [11], which itself is an adap­tation of the shift-reduce parsing algorithm for con­stituency parsers.</p><p>The parser keeps track of what has already been input in a stack. Each time a token is input, the parser must decide whether to shift, reduce, create a left arc, or create a right arc. These actions are described in Table 1.</p><p>The parser continues processing until there is no more input and the stack is empty again, resulting in a connected, non-projective dependency graph. Cur­rently, the parser needs to know when there is a pause or other possible indicator of a sentence boundary; however, an action is being developed that allows the parser to guess that it should terminate the current graph and begin work on a new one. This, rather than separating actual sentences, is intended to sepa­rate semantically meaningful fragments, since spoken data often consists of such fragments, rather than of complete sentences.</p><p>The main difference between our implementation and others is its increment ality. While the other imple­mentations accept only completed text files and output the same, Mink accepts input from a stream, which in our case is the output of a speech recognizer, and out­puts partial analyses as soon as they are available. As soon as it pops a token off the stack-i.e. as soon as it is clear the token will have no more dependents-it outputs the CCG tag and begins semantic conversion. It outputs each conversion as it makes it so that the module that mediates between semantic representa­tion and action can immediately begin to process the semantics of the input to see what it can do with what it knows so far.</p></subsection><subsection number="3.2" title="Machine Learning Algorithm and Features"><p>To decide which actions to perform, we used a maximum-entropy based classifier, namely, the Lo­gistic algorithm from the Weka Java-based machine learning library [15]. This classifier decides, for a pair of an input token and a token on the top of the stack, which action to perform.</p><p>We trained the classifier on the dependency graph version of the Penn Treebank, created with penncon-verter [7].</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">7</doubt><page local="3" global="8"/><p><b><u>Table 1: </u></b><i><u>Parser Actions</u></i></p><p>For features, we tested several different sets. First, we used the standard set of features used by Malt­Parser, since they have proven to be ideal for parsing English. These features are: the token on top of the stack (TOP), its part-of-speech tag and dependency type, the dependency types of its leftmost and right­most dependents, the input token (NEXT), and its part-of-speech tag and dependency type, and the part-of-speech of the next + 1 input. Next, we eliminated the lookahead feature. Last, we eliminated labels and used only unlabeled arcs. See Table 2 for a complete listing of feature sets.</p><p>We evaluate each of these feature sets in section 4.</p></subsection><subsection number="3.3" title="Building CCG tags"><p>CCG tags give a bit more information than part-of-speech tags. Namely, they give both the return type, and the arguments. The return type is basically the part-of-speech tag, while the arguments are similar to phrasal constituents. Our method of constructing CCG tags from dependency arcs therefore involves two steps: determining the return type, and finding the ar­guments.</p><p>When a new token is input, an "empty" CCG tag is created. This includes the return type and list of arguments. Ultimately, it can accept 0 or more argu­ments. Currently, the return type is determined im­mediately upon creating the new tag in order to reduce the complexity of the computation. This is posed as a classification problem. The features used to make the classification are currently the token and WSJ POS tag that was already found by the tagger.</p><p>Whenever a dependency arc is discovered with a given token as the head, the dependent's return type is added as an argument to the head's CCG tag.</p><p>As an example, the phrase the blue box might be tagged and parsed as follows by a constituent parser using the WSJ tagset:</p><p><i>(NP(the</i><b><i>]jT </i></b><i>bluejj </i>fcoxjvjv))-</p><p>However, with a CCG parser, it would be tagged and parsed <i>(NP (theNP/NP (NP blueNP/NP (NP boxNP))).</i><i></i></p><p>In the future, we would like to evaluate the utility of determining the return type after finding the token's arguments in order to use the arguments as features; however, that has not yet been implemented.</p><p>The CCG representation of a parse tree is then used in the semantic conversion.</p></subsection><subsection number="3.4" title="Semantic Conversions"><p>The conversion to semantics relies on a semantic dic­tionary to translate from the parse trees to semantic representations. Each entry in the dictionary consists of a word, its possible CCG tags, and the lambda-logical expressions for each word/CCG tag combina­tion.</p><p>The goal of the conversion is, in our case, to trans­late input sentences into appropriate responses to that input. For example, the question "Did you get it?" generates a <i>report(get+PST(I,box)) </i>action, in which the robot reports the truth or falsity of the predicate <i>get+PST(I,box).</i></p><p>The conversion will fail if definitions retrieved from the dictionary do not form a connected parse. This is appropriate, since in this particular task, precision is more important than recall: if the robot does not understand an utterance, it can ask for clarification or request to have the utterance rephrased.</p><p>One problem of the conversion is that words can be ambiguous, according to human-level comprehen­sion abilities. For example, take the sentence "there is another one in the corner to my right". This could potentially be given two different readings: 1) the ex­istential ("there exists another one in the corner to my right"), and 2) the locative ("another one is in the corner to my right"). This is a problem because the conversion must be deterministic: given an input, it must arrive at the same output every time. Because we currently choose definitions according to rule, this means that each word can have at most one defini­tion for each word/valency combination. However, in the future, a statistical method of choosing the correct definition will be implemented.</p></subsection></section><section number="4" title="Evaluation"><p>There are three segments of the application to evalu­ate: the dependency parsing, the CCG tagging, and the semantic representation. The last two we evaluate simultaneously based on their success at identifying different types of utterances.</p><subsection number="4.1" title="Data Set"><p>In order to evaluate Mink on a domain relevant to our work, we use a spoken-data corpus of human-human dialogs [14], in which one person has to navigate a labyrinth and perform a task, guided by a second per­son outside the labyrinth. The two persons can only communicate via handheld devices. At present, the corpus comprises transcriptions of 12 dialogues.</p></subsection><subsection number="4.2" title="Dependencies"><p>We evaluate the accuracy of the dependency arcs by training both our parser and MaltParser on the same training material, then testing against identical test sets. We test MaltParser vs. each of the three feature sets from Table 2 and show that our parser is, in each <b><u>Table 2:</u></b><page local="4" global="9"/><b><u> </u></b><i><u>Feature Sets</u></i> case, comparable to MaltParser, as one would expect from a reimplementation of the algorithm.</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">8</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Action</b></p></td><td class="cell"><p><b>Description</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Shift</p></td><td class="cell"><p>Move the input token to the stack</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Reduce</p></td><td class="cell"><p>Pop the top token off the stack</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Left-arc</p></td><td class="cell"><p>Create a dependency arc point­ing from the input to the token on the top of the stack; this is followed by a reduce action</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Right-arc</p></td><td class="cell"><p>Create a dependency arc point­ing from the token on top of the stack to the input token; this is followed by a shift action</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>In order to evaluate, we trained both parsers on the Penn treebank (PTB) [9], and tested them on 60 sen­tences from our corpus. The sentences are very much out of the PTB's domain, so this is purely for parser comparison. The sentences were randomly selected and tagged with Wall Street Journal (WSJ) POS tags using acopost<footnote anchor="2"/>, then hand-corrected in order to ensure correct POS tagging. The results can be seen in Table 3.</p><p><b>Table 4: </b><i>Speed</i> <b>Table 3:</b><b> </b><i>Parser Evaluation</i></p><doubt alpha="48.7" length="39" tooSmall="False" monospace="0.0">Metric || Malt || Set 1 | Set 2 | Set 3</doubt><doubt alpha="57.1" length="35" tooSmall="False" monospace="0.0">Lbld attachmtII0.86II0.85I0.85IN7Ä~</doubt><doubt alpha="43.8" length="32" tooSmall="False" monospace="0.0">Unlbld attachmt0.880.880.87 0.85</doubt><doubt alpha="24.0" length="50" tooSmall="False" monospace="0.0">Parsing Time (s) || 22.7 || 32.6 | 31.1    | 29.1~</doubt><p>As can be seen, our results did not vary significantly from MaltParser's. Our results were just slightly lower, which we attribute to different machine-learning methods. Eliminating the lookahead feature had ba­sically no effect on results, and eliminating all label­ing lowered the unlabeled attachment score somewhat, though not significantly.</p><p>We also evaluate the speed of the system when run­ning with either Mink or Malt. Since the system is used to respond to individual utterances rather than to process large texts, we evaluated this by parsing 5 sentences 10 times each and averaging the results. We show that adding increment ality so that processes can run in parallel, rather than in sequence, speeds up the entire process. The results of this evaluation are shown in Table 4.</p></subsection><subsection number="4.3" title="CCG Tags and Semantics"><p>For this particular task, since we are interested only in practical application, the best way of evaluating our tags is, rather than comparing them to those gener­ated by other applications or to gold standards, seeing how successful they are at a practical task. We chose to use the parser to distinguish between different types of utterances, namely questions, commands, and state­ments. We chose not to use a more complex discourse scheme because we are investigating the interface be­tween syntax, semantics, and discourse. More complex discourse schemes often make distinctions between dis­course types that cannot be distinguished by syntax. That is outside the scope of the current project. Our last reason for choosing this particular task is because it is practically useful: we would like our robot to be able to respond to these types of utterances in dif­ferent ways, by following commands, answering ques­tions, and storing statements (facts that may be useful to it).</p><footnote label="2"> http://acopost.sourceforge.net/</footnote><p>To this end, we use three different semantic dictionaries-a statement dictionary, a command dic­tionary, and a question dictionary-to semantically convert each statement in parallel. See Figures 5 and 6 for an example of a small dictionary, describing one question, one statement, and one command. The con­version itself fails if a given dictionary is not able to find a complete conversion for the statement. The clas­sification fails if the utterance is correctly converted by multiple dictionaries, and thus it is not clear which type of utterance it is.</p><p><b>Table 5: </b><i>Sample utterances </i><b>Type Example</b></p><p>Question     <i><u>dids/s y</u></i><i><u>°unp </u></i><i><u>9ets</u></i><i><u>\np/np Unp</u></i><i>_</i></p><p>Statement    <u>Jjyp <i>got,s</i></u><i><u>\n p/n </u></i><i><u>p that^p/^p one^p </u></i>Command   <i><u>gets</u></i><i><u>/np </u></i><i><u>thatxp</u></i><i><u>/np </u></i><i><u>one^p</u>_</i></p><p>The obvious base for distinguishing between these types of utterances is in the verb subcategorization: Statements will, in general, have a subject to the left of the finite verb, while questions will, in general, have a subject to the right, and commands will not have an <b>Table 6:</b><page local="5" global="10"/><b> </b><i>Sample dictionary</i> overt subject at all. There are, of course, exceptions to these generalities which will ultimately require a more complex method of distinguishing between them. In each case, only one semantic conversion should suc­ceed, thus letting us know which type of utterance it is.</p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">9</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Features</b></p></td><td class="cell"><p><b>Description</b></p></td><td class="cell"><p><b>1</b></p></td><td class="cell"><p><b>2</b></p></td><td class="cell"><p><b>3</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TOP.TOK</p></td><td class="cell"><p>The token on top of the stack</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TOP.POS</p></td><td class="cell"><p>The part of speech of TOP</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TOP.DEP</p></td><td class="cell"><p>the dependency type of TOP</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TOP.LEFT</p></td><td class="cell"><p>The dependency type of TOP's leftmost dependent</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TOP.RIGHT</p></td><td class="cell"><p>The dependency type of TOP's rightmost dependent</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEXT.TOK</p></td><td class="cell"><p>The next input token</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEXT.POS</p></td><td class="cell"><p>The part of speech of NEXT</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>NEXT.LEFT</p></td><td class="cell"><p>The dependency type of NEXT's leftmost dependent</p></td><td class="cell"><p>y</p></td><td class="cell"><p>y</p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>LOOK.POS</p></td><td class="cell"><p>The part-of-speech of the next plus one input</p></td><td class="cell"><p>y</p></td><td class="cell"><p></p></td><td class="cell"><p></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Utterance</b></p></td><td class="cell"><p><b>Mink</b></p></td><td class="cell"><p><b>Malt</b></p></td><td class="cell"><p><b>Ratio</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>what is your goal</p></td><td class="cell"><p>185</p></td><td class="cell"><p>478.4</p></td><td class="cell"><p>0.38</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>keep the lights on</p></td><td class="cell"><p>144</p></td><td class="cell"><p>534.5</p></td><td class="cell"><p>0.26</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>cancel keep lights off</p></td><td class="cell"><p>145.5</p></td><td class="cell"><p>489.1</p></td><td class="cell"><p>0.29</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>what are your orders</p></td><td class="cell"><p>173.9</p></td><td class="cell"><p>516</p></td><td class="cell"><p>0.33</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>and try to report the loca­tions of wounded people</p></td><td class="cell"><p>304.8</p></td><td class="cell"><p>525</p></td><td class="cell"><p>0.58</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>We evaluate 60 sentences-20 randomly chosen from each of three human-selected subsets: questions, state­ments, and commands- from our experimental corpus. There are four possibilities for each test example: it does not parse with any of the dictionaries; it parses with just one dictionary, and it is correctly classified; it parses with just one dictionary and it is incorrectly classified; it parses with multiple dictionaries and thus is left unclassified.</p><p>For determining which category a statement be­longed to, we categorized only the explicit structure rather than the utterance's discoursal meaning. For example, "if you enter the closet you should see a box" can be seen an an implicit command to enter the closet and check for a box. However, it is explicitly a simple descriptive statement. Particularly in our case, where all utterance types are translated into some kind of re­sponse, the system needs to be able to translate only the <i>explicit </i>form of the statement into an action.</p><p>As can be seen from the results in Table 7, this method of classifying sentence types was quite suc­cessful. All utterances classified as either commands or questions were correctly classified; however, some questions and commands were classified as statements; and some statements were not classified at all.</p><p>Looking at the data, it is not surprising that ques­tions were misclassified as statements; some ques­tion phrasing is very statement-like, and some other method will have to be deployed to identify these utterances. A few examples that were misclassi­fied are: we are not supposed to take the blue box though right just the blocks ? and so you went through the second room right ? When humans hear such sentences, prosodie contours tell us how to interpret the sentences; but in the current sys­tem, we are not able to make use of such information.</p><p>The commands that were classified as statements were all similar to the following: you leave it and and then you head back. Again, though they ap­peared command-like to the human classifiers, it is obvious why they were misclassified as statements.</p><p><b><u>Table 7: </u></b><i><u>Semantics Evaluation</u></i></p></subsection></section><section number="5" title="Conclusion and Future Work"><p>In this paper, we have described an incremental data-driven dependency parser that outputs graphs, CCG tags, and semantic representations of the input. We have shown that its accuracy is comparable to that of other data-driven dependency parsers, and that it is successful at creating useful CCG tags for practical semantic tasks.</p><p>As we move forward with this project, we will in­vestigate more feature sets for the parser, in the hopes of finding the smallest possible set that continues to achieve high accuracy. Additionally, there are several aspects of our system that are rule-based, and we will investigate the feasibility of making every aspect sta­tistical.</p><p>Finally, our system has largely been working with features that are extractable from the text of a dia­logue. Clearly, in human-robot interaction, there is much that can be learned from other aspects of the interaction, in particular intonation. In the future, we will experiment with integrating prosodie features into the classification system.</p><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Utterance type</b></p></td><td class="cell"><p><b>precision</b></p></td><td class="cell"><p><b>recall</b></p></td><td class="cell"><p><b>f-score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Commands</p></td><td class="cell"><p>1.00</p></td><td class="cell"><p>0.90</p></td><td class="cell"><p>0.94</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Questions</p></td><td class="cell"><p>1.00</p></td><td class="cell"><p>0.70</p></td><td class="cell"><p>0.82</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Statements</p></td><td class="cell"><p>0.69</p></td><td class="cell"><p>0.90</p></td><td class="cell"><p>0.78</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Word</b></p></td><td class="cell"><p><b>CCG</b></p></td><td class="cell"><p><b>Lambda expression</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>did</p></td><td class="cell"><p>s/s</p></td><td class="cell"><p>A x. report (x)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>get</p></td><td class="cell"><p>S/NP,</p></td><td class="cell"><p>A x. get(you,x),</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>S\NP/NP</p></td><td class="cell"><p>Ax. Ay. get(x, y)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>got</p></td><td class="cell"><p>S\NP/NP</p></td><td class="cell"><p>A x. A y. get+PST(x, y)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>I</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>I</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>it</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>it</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>one</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>one</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>that</p></td><td class="cell"><p>NP/NP</p></td><td class="cell"><p>A x. find-reference(x)</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>you</p></td><td class="cell"><p>NP</p></td><td class="cell"><p>you</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>[1] C. Baral, J. Dzifcak, and T. C. Son. Using answer set programming and lambda calculus to charac­terize natural language sentences with normatives and exceptions. In <i>Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI), </i>pages 818-823, Chicago, Illinois, 2008.</p><p>[2] J. Bos, S. Clark, M. Steedman, J. R. Curran, and J. Hockenmaier. Wide-coverage semantic repre­sentations from a ccg parser. In <i>COLING '0Jt: Proceedings of the 20th international conference on Computational Linguistics, </i>page 1240, Mor-ristown, NJ, USA, 2004. Association for Compu­tational Linguistics.</p><p>[3] T. Brick and M. Scheutz. Incremental natural language processing for hri. In <i>The Second ACM IEEE International Conference on Human-Robot Interaction, </i>pages 263-270, 2007.</p><p>[4] W. Che, Z. Li, Y. Hu, Y. Li, B. Qin, T. Liu, and S. Li. A cascaded syntactic and semantic depen­dency parsing system. In <i>CoNLL, </i>pages 238-242, Manchester, England, August 2008. Coling 2008 Organizing Committee.</p><p>[5] S. Clark. Supertagging for combinatory catego-rial grammar. In <i>In Proceedings of the 6th Inter­national Workshop on Tree Adjoining Grammars and Related Frameworks (TAG+6), </i>pages 19-24, 2002.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">10</doubt><page local="6" global="11"/><p>[6] K. Gold and B. Scassellati. A robot that uses ex­isting vocabulary to infer non-visual word mean­ings from observation. In <i>The Twenty-Second Conference on Artificial Intelligence (AAAI-07), </i>2007.</p><p>[7] R. Johansson and R Nugues. Extended constituent-to-dependency conversion for En­glish. In <i>NODALIDA 2007, </i>2007.</p><p>[8] R. Johansson and P. Nugues. Incremental de­pendency parsing using online learning. In <i>The CoNLL Shared Task Session of EMNLP-CoNLL, </i>pages 1134-1138, 2007.</p><p>[9] M. P. Marcus, M. A. Marcinkiewicz, and B. San-torini. Building a large annotated corpus of en-glish: the penn treebank. <i>Computational Linguis­tics, </i>19(2), 1993.</p><p>[10] R. McDonald, F. Pereira, K. Ribarov, and J. Hajic. Non-projective dependency parsing us­ing spanning tree algorithms. In <i>HLT '0,5: Pro­ceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, </i>pages 523-530, Morristown, NJ, USA, 2005. Association for Computational Linguistics.</p><p>[11] J. Nivre. An efficient algorithm for projective dependency parsing. In <i>The 8th International Workshop on Parsing Technologies, </i>pages 149-160, 2003.</p><p>[12] J. Nivre. Increment ality in deterministic depen­dency parsing. <i>Incremental Parsing: Bringing Engineering and Cognition Together. Workshop at ACL</i><i>-2004, </i>2004.</p><p>[13] J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kübler, S. Marinov, and E. Marsi. MaltParser: A language-independent system for data-driven dependency parsing. <i>Natural Lan­guage Engineering, </i>13(2):95-135, 2007.</p><p>[14] M. Scheutz and K. Eberhard. Towards a frame­work for integrated natural language processing architectures for social robots. In <i>The Fifth Inter­national Workshop on Natural Language Process­ing and Cognitive Science (NLPCS-2008), </i>2008.</p><p>[15] I. H. Witten and E. Frank. <i>Data Mining: Prac­tical machine learning tools and techniques, 2nd Edition. </i>Morgan Kaufmann, San Francisco, 2005.</p><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">11</doubt></references></body></article>