<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="73"/><title>Question Parsing for QA in Spanish</title><pubinfo>Proceedings of the Student Research Workshop associated with RANLP 2011,pages 73-78, Hissar, Bulgaria, l3 September 2011.</pubinfo><author surname="Gayo" givenname="Iria"><org  name="University of Santiago de Compostela" country="Spain" city="Santiago de Compostela"/></author></firstpageheader><frontmatter><p><b>Question Parsing for QA in Spanish</b></p><p><b>Ina Gayo</b></p><p>University of Santiago de Compostela</p><p>iria.delrio@usc.es</p></frontmatter><abstract>Question processing is a key step in Question An­swering systems. For this task, it has been shown that a good syntactic analysis of questions helps to improve the results. However, general parsers seem to present some disadvantages in question analysis. We present a specific tool under development for Spanish question analysis in a QA context: SpQA. SpQA is a parser designed to deal with the spe­cial syntactic features of Spanish questions and to cover some needs of question analysis in QA sys­tems such as target identification. The system has been evaluated together with three Spanish gen­eral parsers. In this comparative evaluation, SpQA shows the best results in Spanish question analysis. </abstract></header><body><section number="1" title="Introduction"><p>In Question Answering (QA) systems, question processing is a crucial step to obtain a right an­swer (Carvalho et al., 2010). For this reason, QA systems usually have a specific module that ad­dresses question analysis (Vicedo, 2004). Ques­tion treatment can have different levels of com­plexity, but, in most cases, it entails a syntactic analysis. Furthermore, in this analysis, the cor­rect processing of the interrogative constituent has a special relevance, taking into account that this el­ement can play an important role in the definition of the question target. Correct syntactic analysis of questions constitutes, therefore, a key stage in QA systems process (Moldovan et al., 2002; Her­mjakob, 2001): if we want a good processing of the question, a good syntactic analysis is a helpful starting point.</p><p>Consequently, in order to get a good syntactic analysis of questions, we need a tool for process­ing them correctly. For Spanish there are free gen­eral parsers that could carry out this task. How­ever, this option presents some disadvantages as we will see in detail in section 2. In this paper we present a tool under development for Spanish question analysis in a QA context, SpQA (Span­ish Parser for QA). SpQA is a parser focused on question analysis, designed to be used in the ques­tion processing module of a QA system. As a result, it is thought to deal with the special syn­tactic features of Spanish questions and to cover some needs of QA systems such as question target identification. The parser has been evaluated com­paring its results with those presented for general Spanish parsers in Gayo (2011). As we will see in section 5, this comparative evaluation shows that, currently, SpQA gets the best results in syntactic analysis of questions.</p><p>The paper is structured as follows: in section 2 we show some data related to the performance of general parsers in question analysis. In section 3 we briefly present SpQA. Section 4 accounts for the evaluation method and section 5 shows the re­sults of this evaluation. Finally, in section 6 we present some conclusions and future work.</p></section><section number="2" title="Parsing Questions"><p>To confront the task of question analysis, we could think to make use of available general parsers. However, this option carries some drawbacks.</p><p>It has been shown, at least for English, that pars­ing accuracies of general parsers drop significantly on out-of-domain data (Gildea, 2001; Mcclosky et al., 2006; Foster, 2010). This fact has also been shown, in particular, for question analysis in En­glish (Petrov et al., 2010).</p><p>Unfortunately, for Spanish there are not such studies that compare general accuracies of avail­able parsers with those obtained parsing questions. Therefore, in order to obtain this kind of data, we can use available studies that measure question parsing performance and comparing them with others that measure general performance.</p><p>Related with question parsing in Spanish, we have only the data of Gayo (2011). Gayo (2011) shows the accuracy in question analysis of three general Spanish parsers:<page local="2" global="74"/> DepPattern (Gamallo and Sanchez, 2009), Txala (Atserias et al., 2005) and Hispal (Bick, 2006). As we will see, this evalua­tion uses PARSEVAL metrics for two variables: constituent recognition and constituent labeling. These variables are applied to all consituents in the question and to the interrogative constituent in particular.</p><p>We can see the results of Gayo (2011) summa­rized in Table 1.</p><p>On the other hand, there are general evaluations only for two of the three parsers measured in Gayo (2011): Hispal (Bick, 2006) and Txala (Lloberes et al., 2010). Comparing these general results with those for questions showed in Table 1, we obtain the following data:<footnote anchor="1"/></p><p>Table 2: Comparison of results in general (G) and question analysis (Q for all constituents; Qint for the interrogative contituent) of two Spanish parsers.</p><p>Because we do not have data for Hispal about general constituent recognition (see note 2), it is only possible to make an exhaustive compari­son of both parsers concerning labeling. As we can see in Table 2, compared with general la­beling (G-Labeling), accuracies of both parsers drop in tasks of question labelling (Q-Recognition and Qint-Recognition). This decrease is es­pecially marked labeling the interrogative con­stituent (Qint-Labeling). However, the distance between accuracies in general and question anal­ysis is considerably bigger in Hispal than in Tx-ala. In fact, Txala only shows a remarkable drop labeling the interrogative constituent. We can conclude that Hispal seems to suffer considerably the change of domain, whereas Txala only shows some problems when it is confronted with one spe­cific aspect of questions syntax: the role of the in­terrogative constituent.</p><footnote label="1">In Lloberes et al. (2010), Txala was evaluated with two different corpora, so there are two different results. Unfor­tunately, Bick (2006) does not show general results for con­stituent identification (G-Recognition).</footnote><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">3 SpQA</doubt><p>SpQA is a parser (under development) designed for Spanish question analysis in a QA context. Therefore, it is thought to be part of the ques­tion analysis module of QA systems. SpQA is a rule-based parser/transducer, wich is generated by means of the AGFL parser generator from an at­tribute grammar written in the AGFL formalism<footnote anchor="2"/>. This grammar (with its lexicons and fact tables) is an extension of a general Spanish grammar for IR applications that is also under development, AS­PIRA. The generated parser is a Top-Down Chart parser, using the Best-Only heuristic (Koster et al., 2007). It can perform constituent and dependency analysis (the latter by transduction).</p><p>The aim of the parser is to obtain as much lin­guistic information as possible from questions to facilitate the extraction of the right answer in a QA system. For this reason, we are interested in syn­tactic as well as semantic information, although, for the time being, the parser gets mostly syntac­tic information. Concerning the type of questions to analyze, we want to cover all types of Span­ish direct interrogative structures (wh- and yes/no questions). At the current stage of development, the parser analyzes only wh- questions like:</p><p><i>iQué dibujo Leonardo Da Vinci en 1492?</i></p><doubt alpha="66.7" length="42" tooSmall="False" monospace="0.0">(What did Leonardo Da Vinci draw in 1492?)</doubt><p>Given a question like this, SpQA</p><p>• recognizes and labels all the syntactic con­stituents in the sentence, showing the depen­dency relations between constituents</p><p>• identifies the syntactic and semantic target of the question (qt)</p><footnote label="2">http://www.agfl.cs.ru.nl/</footnote><table caption="Table 1: Evaluation of three Spanish parsers in question analysis Gayo (2011)." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Hispal</b></p></td><td class="cell"><p><b>Txala</b></p></td><td class="cell"><p><b>DepPatt.</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>All-Recognition</p></td><td class="cell"><p>87.8</p></td><td class="cell"><p>91.6</p></td><td class="cell"><p>86.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Int-Recognition</p></td><td class="cell"><p>97.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>90.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>All-Labeling</p></td><td class="cell"><p>68.2</p></td><td class="cell"><p>71.3</p></td><td class="cell"><p>51.1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Int-Labeling</p></td><td class="cell"><p>52.5</p></td><td class="cell"><p>62.0</p></td><td class="cell"><p>25.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Txala Hispal</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>G-Recognition Q-Recognition Qint-Recognition</p></td><td class="cell"><p>81.1/80.9</p><p>91.6 87.8 100.0 97.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>G-Labeling Q-Labeling Qint-Labeling</p></td><td class="cell"><p>73.9/74.3 95.3 71.3 68.2 62.0 52.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="3" global="75"/><p>• recognizes specific structures as dates, quan­tities and personal NP's</p><doubt alpha="62.2" length="74" tooSmall="False" monospace="0.0">[[PN3: Leonardo Da Vinci ] &lt;SUBJ [ V:dibujar&lt;qtOBJ [ENTITY]&lt;DATEen 1492 ]]</doubt><doubt alpha="59.7" length="72" tooSmall="False" monospace="0.0">([[PN: Leonardo Da Vinci ] &lt;SUBJ [ V:draw&lt;qtOBJ [ENTITY]&lt;DATEin 1492 ]])</doubt><p>Currently, SpQA identifies six different seman­tic targets: PERSON, ENTITY, QUANT, TIME, PLACE, MANNER. To identify them, the parser uses the linguistic information encoded in the wh-words.</p><p>• PERSON: when the target is human.</p><p><i>iQuién era el présidente de Francia durante las prue-bas de armas nucleares en el Pacifico Sur?</i></p><p>(Who was the president of France during the tests of nuclear weapons in the South Pacific?)</p><p>• ENTITY: when the target is no human.</p><doubt alpha="66.7" length="42" tooSmall="False" monospace="0.0">iQuefue levantado el 13 de agosto de 1961?</doubt><doubt alpha="61.0" length="41" tooSmall="False" monospace="0.0">(What was built the 13th of August 1961?)</doubt><p>• QUANT: when the target is a quantity.</p><p><i>iCuântos goles se marcaron en total en el Mundial de Futbol de 1982?</i></p><p>(How many goals were scored in total in the World Cup of 1982?)</p><p>• TIME: when the target is related with time (a date, time, etc).</p><p><i>iCuändo sefirmo el Tratado de Maastricht?</i></p><p>(When was the Maastricht Treaty signed?)</p><p>• PLACE: when the target is a location.</p><p><i>iDônde se celebraron los JJ.OO. de 1992?</i></p><p>(Where were celebrated the Olympic Games of 1992?)</p><p>• MANNER: when the target is a process, a de­scription or an explanation.</p><p><i>iCômo actua la hormona del crecimiento?</i></p><p>(How does growth hormone work?)</p><p>When the question has a more complex interroga­tive constituent like <i>I </i><b>Cuantos kilos de anchoas </b><i>capturo la flota del Cantabrico durante 1994?</i><i></i></p><p>(How many kilos of anchovies did the fleet of the Cantabric fish in 1994?)</p><p>SpQA identifies the semantic target with the nu­cleus of the interrogative constituent:</p><doubt alpha="62.9" length="132" tooSmall="False" monospace="0.0">[[[[N: flota] &lt;PREPde [PN: Cantabrico ]] &lt;DET la] &lt;SUBJ [ V:capturar&lt;qtOBJquant [[N: kilos] &lt;PREPde [N: anchoas]]&lt;DATEdurante 1994]]</doubt><doubt alpha="58.9" length="129" tooSmall="False" monospace="0.0">([[[[N: fleet] &lt;PREPof [PN: Cantabric ]] &lt;DET the] &lt;SUBJ [ V:fish&lt;qtOBJquant [[N: kilos] &lt;PREPof [N: anchovies] ]&lt;DATEin 1994 ]])</doubt></section><section number="4" title="Question Parsing Evaluation"><p>We are interested in a comparative evaluation of SpQA against other Spanish general parsers. However, building the methodology and the nec­essary data for parsing evaluation is a very com­plex and hard task (especially if the parsers have different frameworks, like in our case). For this reason, for our evaluation we have used the same data and evaluation methodology of Gayo (2011), applying them to SpQA and comparing our results with those of Gayo (2011) for DepPattern, Txala and Hispal.</p><p>In this section, we present first the three Span­ish parsers used for the comparative evaluation of SpQA: Txala, Hispal and DepPattern. Then, we explain in detail the comparative evaluation method taken from Gayo (2011).</p><subsection number="4.1" title="Spanish Parsers for the Comparative Evaluation"><p><b>TXALA </b>is the Spanish parser in the suite <i>Freel-ing<footnote anchor="4"/> </i>(Padroetal.,2010). It can be downloaded for free (as a part of <i>Freeling) </i>and it is also available on-line. It offers dependency parsing with func­tional labeling.</p><p><b>HISPAL </b>is the Spanish parser of the VISL<footnote anchor="5"/>project. It is only available for use on-line, but it allows the uploading of files for analysis with a maximum of 2 Mb. It performs constituent parsing with functional labeling in the Constraint Gram­mar framework.</p><p><b>DEPPATTERN </b>is the Spanish parser in the suite <i>DepPattern Toolkit<footnote anchor="6"/> </i>(Gamallo and Sanchez, 2009). It can be downloaded for free and it is also available on-line. It offers dependency pars­ing with functional labeling.</p><footnote label="3">PN = Proper Noun</footnote><footnote label="4">http://nlp.lsi.upc.edu/freeling/  5 http://beta.visl.sdu.dk/</footnote><footnote label="6">http://gramatica.usc.es/pln/tools/deppattern.html</footnote><page local="4" global="76"/></subsection><subsection number="4.2" title="Evaluation Methodology"><p>For the comparative evaluation of SpQA, we have used the parser evaluation methodology presented in Gayo (2011). We applied the metrics of PAR-SEVAL scheme (Black et al., 1991) to measure two variables in question analysis: constituent recognition and constituent labeling. For each variable, we measure</p><p><b>• Precision: </b>number of correct constituents (constituents in the gold standard) in parser output divided by number of constituents in the parser output.</p><p><b>• Recall: </b>number of correct constituents (con­stituents in the gold standard) in parser output divided by the number of constituents in the gold standard.</p><p><b>• F1 score.</b></p><p>We applied these two variables to constituents in general (all the constituents in the sentence) and to the interrogative constituent in particular (for the importance of this element in QA systems). To make possible the comparison of SpQA with the results showed in Gayo (2011), we also used the same testing corpus of questions and the same gold standard.</p><subsubsection number="4.2.1" title="The Testing Corpus"><p>The corpus is made up of 100 questions extracted from monolingual Spanish sets of CLEF<footnote anchor="7"/> 2004, 2006 and 2007. All the examples in the test­ing corpus are wh- questions. Questions were se­lected from CLEF sets according to their syntac­tic structure. The idea was to choose questions that presented a variety of syntactic structures, like different interrogative constituents, subordinated clauses, dates or named entities.</p></subsubsection><subsubsection number="4.2.2" title="The Gold Standard"><p>The gold standard is made up of the 100 ques­tions of the testing corpus analyzed manually by one person. The analysis consists of the identifi­cation of the main syntactic structure (constituents in the sentence): verb and arguments/adjuncts, la­beled with their syntactic function.</p><p><i>iQue robaba el oso Yogui?</i></p><p>What did Yogi Bear steal?</p></subsubsection></subsection></section><section number="3" title="constituents:"><p><b>Verb: </b><i>robaba </i>(did...steal) <b>Interrogative Direct Object:</b><b> </b><i>Que </i>(what) <b>Subject:</b><b> </b><i>el oso Yogui </i>(Yogi Bear)</p><footnote label="7">http://www.clef-campaign.org/</footnote><p>To minimize possible differences between parsers caused by their different frameworks, some linguistic decisions were taken in the anno­tation. These decisions tried to simplify as much as possible the syntactic analysis. For example, we only consider six syntactic labels: subject (S), direct object (O), indirect object (IO), predicative (PR), adjunct (CC; bounded or unbounded) and modifier (MOD); we analyze the verbal phrase al­ways as one constituent (even if it was a complex unit: <i>ha sido premiado, </i>has been awarded); we do not compute as constituents functional clitics as <i>lo </i>(direct object clitic) or <i>se </i>(impersonal clitic); etc.</p><subsection number="4.3" title="Parsers Output Analysis"><p>For Txala, Hispal and DepPattern we use directly the data of Gayo (2011). For SpQA, we ana­lyzed the testing corpus with the parser and we extracted:</p><p>• Number of constituents recognized: total number of constituents in the parser output.</p><p>• Identification of constituents: number of cor­rect and incorrect constituents (compared with the gold standard) in the parser output.</p><p>• Labeling: number of correct and incorrect la­beled constituents (compared with the gold standard) in the parser output.</p></subsection></section><section number="5" title="Results"><p>We show first the results concerning question con­stituents in general. Then, the results related to the interrogative constituents in particular (identifica­tion and labeling for both).</p><subsection number="5.1" title="Question Constituents"><p>We can see the results of general constituent recognition in Table 4.</p><p><b>I </b><b>Hispal Txala DepPatt. SpQA</b></p><doubt alpha="23.1" length="39" tooSmall="False" monospace="0.0">precision      86.9 89.9      88.8 91.2</doubt><doubt alpha="15.0" length="40" tooSmall="False" monospace="0.0">recall          88.7 93.3      83.6 93.6</doubt><doubt alpha="15.4" length="39" tooSmall="False" monospace="0.0">F-score        87.8 91.6      86.1 92.4</doubt><table caption="Table 3: Constituent recognition."></table><p>The four parsers have good results: around or over 90. SpQA has the best results, although they are very close to those of Txala.</p><page local="5" global="77"/><p>In general constituent labeling, we have the next results:</p><p>Again SpQA has the best results. However, for this variable there is a clear distance between SpQA and the other three parsers. Whereas the ac­curacies of Txala, Hispal and DepPattern drop sig­nificantly in this task (comparing their results with Table 4), SpQA maintains its performance (only the recall is a bit lower).</p><p>So, as we can see, SpQA shows very close re­sults in general constituent recognition and label­ing.</p></subsection><subsection number="5.2" title="Interrogative Constituent"><table caption="Table 5 shows the results for interrogative con­stituent recognition:"></table><p>Again, the four parsers have good results, all over 90. Txala has the best accuracy, followed very closely by SpQA.</p><p>The reason that SpQA does not achieve an accu­racy of 100 is simple: the parser fails in the recog­nition of one of the sentences as a question, due to structural syntactic reasons (the question has a syntactic order that is not in the grammar). As a consequence, with the current architecture of the system, this causes it to fail in the recognition of the interrogative constituent. However, the impor­tant thing to note is that the problem is not in the recognition of the interrogative and it can be easily solved.</p><p>Concerning labeling, these are the results: We can see again a substantial difference in parser accuracies between recognition and label­ing. Hispal, Txala and DepPattern especially, have worse results again, whereas SpQA keeps its accu­racy.</p><table caption="Table 6: Interrogative constituent labeling."></table><p>The accuracy in interrogative constituent label­ing is even lower than in general constituent la­beling (Table 5) for Hispal, Txala and DepPattern. From accuracies around 70, Hispal and Txala drop to numbers around 50 and 60, respectively; Dep-Pattern falls from 51 to 25.</p><p>On the other hand, SpQA still maintains its per­formance, and, contrary to the other two parsers, it has even better results labeling the interrogative constituent (94%) than in general labeling (91%).</p></subsection></section><section number="6" title="Conclusions and Future Work"><p>Question processing is a crucial step in QA sys­tems. In this processing, syntactic analysis of questions plays an important role.</p><p>For this task, we have presented SpQA, a parser focused on question analysis in Spanish. Cur­rently, the system recognizes and labels all the constituents in the question. In addition, it iden-tines the syntactic and semantic target of the ques­tions, as well as dates, proper nouns and quanti­ties.</p><p>Compared to three freely available Spanish parsers, Hispal, Txala and DepPattern, SpQA shows the best results in four tasks: recognition and labeling of general constituents and recogni­tion and labeling of the interrogative constituent. Besides this, whereas Hispal, Txala and DepPat-tern show a considerable difference between their accuracies in constituent recognition and label­ing (general and for the interrogative constituent), SpQA keeps its accuracy, which is always over 90.</p><p>Future work concerns syntax and semantics as­pects of SpQA. First, we have to make the gram­mar more complete to cover all possible syntac­tic structures of Spanish questions. Then, it will be necessary to concentrate on semantic aspects of questions, especially on the aspects related to target identification.</p><table caption="Table 4: Constituent labeling." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Hispal</b></p></td><td class="cell"><p><b>Txala</b></p></td><td class="cell"><p><b>DepPatt.</b></p></td><td class="cell"><p><b>SpQA</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>precision</p></td><td class="cell"><p>52.0</p></td><td class="cell"><p>62.0</p></td><td class="cell"><p>25.0</p></td><td class="cell"><p>94.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>recall</p></td><td class="cell"><p>53.0</p></td><td class="cell"><p>62.0</p></td><td class="cell"><p>25.0</p></td><td class="cell"><p>94.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F-score</p></td><td class="cell"><p>52.5</p></td><td class="cell"><p>62.0</p></td><td class="cell"><p>25.0</p></td><td class="cell"><p>94.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Hispal</b></p></td><td class="cell"><p><b>Txala</b></p></td><td class="cell"><p><b>DepPatt.</b></p></td><td class="cell"><p><b>SpQA</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>precision</p></td><td class="cell"><p>72.5</p></td><td class="cell"><p>73.9</p></td><td class="cell"><p>56.1</p></td><td class="cell"><p>94.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>recall</p></td><td class="cell"><p>64.3</p></td><td class="cell"><p>69.0</p></td><td class="cell"><p>46.9</p></td><td class="cell"><p>88.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F-score</p></td><td class="cell"><p>68.2</p></td><td class="cell"><p>71.3</p></td><td class="cell"><p>51.1</p></td><td class="cell"><p>91.4</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 5: Interrogative constituent recognition." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p><b>Hispal</b></p></td><td class="cell"><p><b>Txala</b></p></td><td class="cell"><p><b>DepPatt.</b></p></td><td class="cell"><p><b>SpQA</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>precision</p></td><td class="cell"><p>96.1</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>90.0</p></td><td class="cell"><p>99.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>recall</p></td><td class="cell"><p>98.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>90.0</p></td><td class="cell"><p>99.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F-score</p></td><td class="cell"><p>97.0</p></td><td class="cell"><p>100.0</p></td><td class="cell"><p>90.0</p></td><td class="cell"><p>99.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Jordi Atserias, Elisabet Comelles, and Aingeru Mayor. 2005.  Txala un analizador libre de dependencias para el castellano.<page local="6" global="78"/> <i>Procesamiento del Lenguaje Nat­ural, </i>(35):455-456, September.</p><p>Eckhard Bick. 2006. A Constraint Grammar-based Parser for Spanish. In <i>Proceedings of TIL 2006 -4th Workshop on Information and Human Language Technology.</i></p><p>Ezra Black, Steven P. Abney, D. Flickenger, Clau­dia Gdaniec, Ralph Grishman, P. Harrison, Don­ald Hindle, Robert Ingria, Frederick Jelinek, Ju­dith L. Klavans, Mark Liberman, Mitchell P. Mar­cus, Salim Roukos, Beatrice Santorini, and Tomek Strzalkowski. 1991. A Procedure for Quantita­tively Comparing the Syntactic Coverage of English Grammars. In <i>North American Chapter ofthe Asso­ciation for Computational Linguistics.</i></p><p>Gracinda Carvalho, David Martins de Matos, and Vi-tor Rocio. 2010. Improving IdSay: A Charac­terization of Strengths and Weaknesses in Ques­tion Answering Systems for Portuguese. In Thiago Alexandre Salgueiro Pardo, Antonio Branco, Alde-baro Klautau, Renata Vieira, and Vera Lucia Strube de Lima, editors, <i>PROPOR, </i>volume 6001 of <i>Lecture Notes in Computer Science, </i>pages 1-10. Springer.</p><p>Jennifer Foster. 2010. "cba to check the spelling" Investigating Parser Performance on Discussion Fo­rum Posts. In <i>Human Language Technologies: The 2010 Annual Conference of the North American Chapter ofthe Association for Computational Lin­guistics, </i>HLT '10, pages 381-384, Stroudsburg, PA, USA. Association for Computational Linguistics.</p><p>Pablo Gamallo and Isaac Gonzalez. 2009. Una gramatica de dependencias basada en patrones de etiquetas. <i>Procesamiento del Lenguaje Natural, </i>(43):315-323.</p><p>IriaGayo. 2011. Analisis de preguntas para Biisqueda de Respuestas: evaluation de tres parsers del espanol (Question Analysis for QA: Evaluation of three Spanish Parsers). In <i>Proceedings ofSEPLN'11 (to appear).</i></p><p>Daniel Gildea. 2001. Corpus Variation and Parser Per­formance. In Lillian Lee and Donna Harman, edi­tors, <i>Proceedings ofthe 2001 Conference on Empiri­cal Methods in Natural Language Processing, </i>pages 167-202.</p><p>Ulf Hermjakob. 2001. Parsing and Question Classi­fication for Question Answering. In <i>Proceedings of the workshop on Open-domain question answering -Volume 12, </i>ODQA '01, pages 1-6, Stroudsburg, PA, USA. Association for Computational Linguistics.</p><p>Cornelis H. A. Koster, Marc Seutter, and Olaf Seibert. 2007. Parsing the Medline Corpus. In <i>Proceedings RANLP 2007, </i>pages 325-329.</p><p>Marina Lloberes, Irene Castellon, and Llufs Padro. 2010. Spanish FreeLing Dependency Grammar. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan</p><p>Odijk, Stelios Piperidis, Mike Rosner, and Daniel Tapias, editors, <i>Proceedings of the Seventh con­ference on International Language Resources and Evaluation (LREC'10), </i>Valletta, Malta, may. Euro­pean Language Resources Association (ELRA).</p><p>David Mcclosky, Eugene Charniak, and Mark Johnson. 2006. Reranking and Self-training for Parser Adap­tation. <i>ACL-COLING.</i></p><p>D. Moldovan, S. Harabagiu, R. Girju, P. Morarescu, F. Lacatusu, A. Novischi, A. Badulescu, and O. Bolohan. 2002. Lcc Tools for Question Answer­ing. In Voorhees and Buckland, editors, <i>Proceed­ings of the 11th Text REtrieval Conference (TREC-</i> <i>2002), </i>NIST, Gaithersburg.</p><p>Llufs Padro, Miquel Collado, Samuel Reese, Marina Lloberes, and Irene Castellon. 2010. Freeling 2.1: Five Years of Open-source Language Pro­cessing Tools. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Ros­ner, and Daniel Tapias, editors, <i>Proceedings ofthe Seventh conference on International Language Re­sources and Evaluation (LREC'10), </i>Valletta, Malta, may. European Language Resources Association (ELRA).</p><p>Slav Petrov, Pi-Chuan Chang, Michael Ringgaard, and Hiyan Alshawi. 2010. Uptraining for Accurate De­terministic Question Parsing. In <i>Proceedings ofthe 2010 Conference on Empirical Methods in Natural Language Processing, </i>pages 705-713, Cambridge, MA, October. Association for Computational Lin­guistics.</p><p>Jose Luis Vicedo. 2004. La busqueda de respuestas: Estado actual y perspectivas de futuro. <i>Inteligen-cia Artificial, Revista Iberoamericana de Inteligen-cia Artificial, </i>8(22):37-56.</p></references></body></article>