<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="317"/><title>ID 392:TERSEO + T2T3 Transducer. A systems for Recognizing and Normalizing TIMEX3</title><pubinfo>Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010,pages 317-320, Uppsala, Sweden, 15-16 July 2010. ©2010 Association for Computational Linguistics</pubinfo><author surname="Saquete Boro" givenname="Estela"><org  name="University of Alicante" country="Spain" city="Alicante"/></author></firstpageheader><frontmatter><p><b>ID 392:TERSEO + T2T3 Transducer. A systems for recognizing and</b></p><p><b>normalizing TIMEX3</b></p><p><b>Estela Saquete</b></p><p>Natural Language Processing and Information System Group University of Alicante stela@dlsi.ua.es</p></frontmatter><abstract>The system described in this paper has par­ticipated in the Tempeval 2 competition, specifically in the Task A, which aim is to determine the extent of the time expres­sions in a text as defined by the TimeML TIMEX3 tag, and the value of the fea­tures type and val. For this purpose, a combination of TERSEO system and the T2T3 Transducer was used. TERSEO sys­tem is able to annotate text with TIDES TIMEX2 tags, and T2T3 transducer per­forms the translation from this TIMEX2 tags to TIMEX3 tags. </abstract></header><body><section number="1" title="Introduction"><p>Identification and extraction of explicit and im­plicit temporal information has become a very im­portant field of research within the computational linguistics area since some years ago (Allen, 1983) (Allen, 1984). Moreover, a large number of NLP applications are exploiting this extracted informa­tion, such as question answering and summariza­tion systems, allowing these applications to per­form in a more complex level.</p><p>When dealing with temporal information iden­tification and normalization, different approaches can be taken, depending on the available resources of the target language and the requirements of the system being developed. The most extended ap­proaches to the problem are: a) rule-based ap­proaches, such as Chronos (ITC-irst): recognizes and normalizes temporal expressions in English and Italian (Negri, 2007); TERSEO (University of Alicante, the system used for this work): a knowl­edge based system for Spanish that has been auto­matically extended to other languages, such as En­glish, Italian and Catalan (Saquete et al., 2006), b) machine learning approaches, such as TimexTag (University of Amsterdam): applies data-driven methods for recognition and normalization tasks (Ahn et al., 2005) (Ahn, 2006); CU-TMP (Uni­versity of Colorado): uses machine learning for automatic annotation (Bethard and Martin, 2007), and c) mixed combination of rules and ML ap­proaches, such as, TempEx (MITRE Corpora­tion): combines hand-coded patterns with ma­chine learning rules to tag documents (TempEx, 2008) (Mani and Wilson, 2000); TARSQI (Bran­deis University): currently uses GUTime (2008) for temporal expression annotation, which extends the capabilities of the TempEx tagger while gener­ating TIMEX3 annotations (Verhagen et al., 2005). However, whatever the approach, the output of these systems is a standardized annotation scheme.</p><p>The most popular annotation schemes are TIDES (Mani et al., 2001) and TimeML (Puste-jovsky et al., 2003b). TIDES program followed the efforts started in the context of the Message Understanding Conference, MUC (1998), and de­fined the TIMEX2 tag, with the goal of interpret­ing temporal expressions within a normalized rep­resentation of the times they denote, adopting the ISO 8601 standard (Technical Committee ISO/TC</p><doubt alpha="50.0" length="48" tooSmall="False" monospace="0.0">154, 2004). In 2004, within the ACE program, the</doubt><p>Time Expression Recognition and Normalization (TERN, 2004) evaluation workshop was held, re­quiring by the participation systems to detect and normalize the temporal expressions mentioned in the source data, according to the TIDES annota­tion standard <footnote anchor="1"/>. In spite of the widespread use of this annotation scheme within NLP researchers, it is necessary to identify other types of tempo­ral information such as events or the relations be­tween events and temporal expressions. Motivated by such considerations, the TimeML annotation scheme (Pustejovsky et al., 2003a) (Pustejovsky et al., 2005) (Lee et al., 2007) was developed, speci­fying four major data structures (elements) for annotation:<page local="2" global="318"/> EVENT, TIMEX3, SIGNAL and LINK (Pustejovsky et al, 2005).</p><footnote label="1"> http://fofoca.mitre.org/annotation_guidelines/2005Jimex2 _standard_vL1.pdf</footnote></section><section number="2" title="System Description"><p>The system presented in this paper is a combi­nation of two separated systems. First of all, TERSEO system, which is a knowledge-based system for Spanish automatically extended to En­glish, performs an identification and normaliza­tion of all the temporal expressions in the text, an­notating them with TIMEX2 tags. Once the text has been annotated with TIMEX2, the T2T3 trans­ducer applies a set of translation rules to convert this TIMEX2 output to a TIMEX3 output.</p><subsection number="2.1" title="Description of TERSEO system"><p>TERSEO system first implementation used a hand-made knowledge database in Spanish. How­ever, our main goal was the possibility of working with TERSEO on a multilingual level, but building the different knowledge databases for the new lan­guages through the automatic acquisition of rules (Negri et al., 2006). Therefore, it is possible to create a multilingual system with no need of a pre­vious knowledge of the other languages to which TERSEO system is going to be extended. For this purpose, an architecture similar to the one used by EuroWordNet (Vossen, 2000) was implemented, in order to obtain knowledge databases for the different languages, but all of them connected though a unit denominated TER-ILI or Temporal Expression Rules Interlingua Index. In doing that, TERSEO system have a new knowledge database for each new language and is able to solve any ex­pression in this language. Besides, the system is easily extensible to other new languages. The out­put of TERSEO system is following the guidelines of TIDES annotation scheme.</p><p>This system participated in TERN2004 for En­glish, obtaining the results shown in Table 1.</p><p>It is important to consider the results of the sys­tem annotating TIMEX2 tags, due to the fact that the final results after the translation depends on how correct the annotation was made by TERSEO.</p></subsection><subsection number="2.2" title="Description of T2T3 Transducer"><p>The T2T3 Transducer, developed by University of Alicante and Brandeis University, implements an automatic mapping between TIDES annotation scheme and TimeML, only in English in a first step. This mapping is performed applying a set of rules in two steps:</p><p><b>• Step 1: Rules for the adaptation of the</b></p><p><b>extent: </b>the temporal expression extent is adapted from TIMEX2 to TIMEX3. The ex­tension of the expression is related to recog­nition of the expression. Most expressions which are considered as markable in TIDES are also considered as markable in TimeML. However, TimeML differs from TIDES with respect to the tag span in some cases. There­fore, following the outline of both TIDES 2005 guidelines<footnote anchor="2"/> and TimeML 1.2.1 guide­lines<footnote anchor="3"/>, a mapping is performed in order to properly adapt the TIMEX2 extent to the TIMEX3 extent. Besides, all the possible adaptations from one scheme to the other are clustered in a set of transformation rules.</p><p><b>• Step 2: Rules for the transformation of the attributes: </b>TIMEX2 attributes are trans­formed to TIMEX3 attributes. The attributes are related to normalization of the expres­sion. The transducer has one rule for each TimeML TIMEX3 attribute, extracting and combining the information provided by the TIMEX2 attributes of each temporal expres­sion. In Tempeval 2 competition only type and val attributes are considered. Therefore, only these two transformation rules are pre­sented here:</p><p>- Attribute type: The <b>Type Assignment </b>rule defines the following steps:</p><p>1. If the &lt;TIMEX2&gt; tag has a SET at­tribute which value is "YES", then type = " SET " must be added to the TIMEX3 tag.</p><p>2. If the VAL attribute of the &lt;TIMEX2&gt; tag starts with "P", then type= "DURATION" must be added to the TIMEX3 tag.</p><p>3. If the VAL attribute of the &lt;TIMEX2&gt; tag contains a "T", then type= "TIME " must be added to the TIMEX3 tag.</p><p>4. In any other case, type= " DATE " must be added to the TIMEX3 tag.</p><p>- Attribute value: The attribute value is equivalent to the VAL attribute in<page local="3" global="319"/></p><footnote label="2">Section 5 in TIDES guidelines http://fofoca.mitre.org  3 Section      2.2.1.2      in      TimeML guidelines http://www.timeml.org</footnote><p>TIMEX2 in most cases. Therefore, in general, the translation is direct. How­ever, there is an exception to this rule in the case of time-anchored expressions. Whereas in TimeML, the value of the head expression is always a period, ac­cording to TIDES, there are two differ­ent types of time-anchored expressions: a) Anchored point expressions and b) Anchored duration expressions. There­fore, when the T2T3 transducer detects one of these anchored point expressions, a special treatment with the TIMEX2 at­tributes is performed in order to obtain the proper period value. Moreover, the "DURATION" type is established for the expression.</p></subsection></section><section number="3" title="Evaluation results"><p>In this section all the evaluation results for Task A in English are presented. Table 2 shows the results of the system using the trial corpus provided by the organization, the results of the system using the first delivered training corpus and the whole training data, and finally, the score of the system with the test corpus. Accuracy value is not given in the test results and it can not be calculated from the results data provided.</p><p>As shown in the results of the different evalu­ations, test results are very similar to training re­sults, what means that the system is performing steadily. Besides, in the test evaluation, the type attribute result is the best one obtained, being close to 100%. It would be interesting to have the corpus annotated also with TIMEX2 in order to determine which errors derive from TERSEO and which er­rors derive from the Transducer.</p></section><section number="4" title="Conclusions"><p>Our participation in Tempeval 2 competition was only in Task A, due to the fact that the system pre­sented is a extension of TERSEO system, which only performs identification and normalization of temporal expressions generating TIMEX2 annota­tion output. Events and links are out of the scope of this system currently.</p><p>However, our motivation for participating in Tempeval 2 competition was the possibility to de­termine the performance of the extension applied to TERSEO, by means of a transducer that is able to convert TIMEX2 annotation to TIMEX3, only using the information of the TIMEX2 tags as in­put. The transducer applies a set of rules, in order to transform the extent of the temporal expression according to TimeML annotation guidelines, and a set of rules to translate the TIMEX2 attributes to the attributes established by TimeML also. It is important to consider that TERSEO system is a knowledge-based system, with hand-made rules for Spanish. These rules were automatically ex­tended to other languages (English is one of them) using only automatic resources and without man­ual revision. This automatic extension is very in­teresting since it is possible to create a new knowl­edge for the system very fast and with satisfactory results.</p><p>The results of the evaluation of this combination (TERSEO + T2T3 Transducer) are 76% precision, 66% recall and 71% F1-Measure. For the case of the attributes, it obtained 98% for type and 65% for value.</p></section><section title="Acknowledgments"><p>This research has been partially supported by the Spanish government, projects TIN-2009-13391-C04-01 and PROMETEO/2009/119. Further­more, I want thank James Pustejovsky for being the co-author ofT2T3 Transducer.</p><table caption="Table 1: Results obtained by TERSEO in TERN2004 competition for TIMEX2" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Tag</b></p></td><td class="cell"><p><b>Precision</b></p></td><td class="cell"><p><b>Recall</b></p></td><td class="cell"><p><b>F-Measure</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2</p></td><td class="cell"><p>0.954</p></td><td class="cell"><p>0.786</p></td><td class="cell"><p>0.862</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:ANCHOR_DIR</p></td><td class="cell"><p>0.818</p></td><td class="cell"><p>0.566</p></td><td class="cell"><p>0.669</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:ANCHOR_VAL</p></td><td class="cell"><p>0.703</p></td><td class="cell"><p>0.487</p></td><td class="cell"><p>0.575</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:MOD</p></td><td class="cell"><p>0.444</p></td><td class="cell"><p>0.111</p></td><td class="cell"><p>0.178</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:SET</p></td><td class="cell"><p>0.882</p></td><td class="cell"><p>0.455</p></td><td class="cell"><p>0.600</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:TEXT</p></td><td class="cell"><p>0.687</p></td><td class="cell"><p>0.567</p></td><td class="cell"><p>0.621</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>TIMEX2:VAL</p></td><td class="cell"><p>0.686</p></td><td class="cell"><p>0.709</p></td><td class="cell"><p>0.698</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 2: Results obtained by TERSEO+T2T3 Transducer with trial corpus for English" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Measure</b></p></td><td class="cell"><p><b>Trial</b></p></td><td class="cell"><p><b>Training 1</b></p></td><td class="cell"><p><b>Training 2</b></p></td><td class="cell"><p><b>Test</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PRECISION</p></td><td class="cell"><p>0.83</p></td><td class="cell"><p>0.78</p></td><td class="cell"><p>0.83</p></td><td class="cell"><p>0.76</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>RECALL</p></td><td class="cell"><p>0.72</p></td><td class="cell"><p>0.66</p></td><td class="cell"><p>0.55</p></td><td class="cell"><p>0.66</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>F1-MEASURE</p></td><td class="cell"><p>0.77</p></td><td class="cell"><p>0.72</p></td><td class="cell"><p>0.66</p></td><td class="cell"><p>0.71</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ACCURACY</p></td><td class="cell"><p>0.99</p></td><td class="cell"><p>0.98</p></td><td class="cell"><p>0.98</p></td><td class="cell"><p>-</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ATT. TYPE</p></td><td class="cell"><p>0.86</p></td><td class="cell"><p>0.87</p></td><td class="cell"><p>0.87</p></td><td class="cell"><p>0.98</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ATT. VAL</p></td><td class="cell"><p>0.64</p></td><td class="cell"><p>0.58</p></td><td class="cell"><p>0.63</p></td><td class="cell"><p>0.65</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>David Ahn, Sisay Fissaha Adafre, and Maarten de Ri-jke. 2005. Towards task-based temporal extrac­tion and recognition. In Graham Katz, James Puste­jovsky, and Frank Schilder, editors, <i>Annotating, Ex­tracting and Reasoning about Time and Events, </i>vol­ume 05151 of <i>Dagstuhl Seminar Proceedings. </i>Internationales Begegnungs- und Forschungszentrum fur Informatik (IBFI), Schloss Dagstuhl, Germany In­ternationales Begegnungs- und Forschungszentrum fur Informatik (IBFI), Schloss Dagstuhl, Germany.<page local="4" global="320"/></p><p>D.Ahn. 2006. The stages of event extraction. In Asso­ciation for Computational Linguistics, editor, <i>ARTE: Workshop of 44th Annual Meeting of the Association for Computational Linguistics, </i>pages 1-8, Sydney, Australia.</p><p>J. Allen. 1983. Maintaining knowledge about tem­poral intervals. <i>Communications of the ACM 26, </i>(11):832-843.</p><p>J. Allen. 1984. Towards a general theory of action and time. <i>Artificial Intelligence, </i>(23):123-154.</p><p>S. Bernard and J.H. Martin. 2007. CU-TMP: Tempo­ral Classification Using Syntactic and Semantic Fea­tures. In <i>Proceedings ofthe 4th International Work­shop ofSemEval-2007, </i>pages 129-132.</p><p>GUTime.        2008.        Georgetown University. http://www.timeml.org/site/tarsqi/ modules/gutime/index.html.</p><p>K. Lee, B. Boguaraev, H. Bunt, and J. Pustejovsky. 2007. ISO-TimeML and its Applications. In <i>Pro­ceedings ofthe 2007 Conference for ISO Technical Committee 37.</i></p><doubt alpha="64.7" length="51" tooSmall="False" monospace="0.0">I. Mani and G. Wilson. 2000. Processing of news. In</doubt><p><i>Proceedings of the 38th Annual Meeting of the As­sociation for Computational Linguistics (ACL2000), </i>pages 69-76.</p><p>I. Mani, G. Wilson, B. Sundheim, and L. Ferro. 2001. Guidelines for annotating temporal information. In <i>Proceedings of HLT 2001, First International Con­ference on Human Language Technology Research, J.</i><i> Allan ed., Morgan Kaufmann, San Francisco, </i>pages 142-144.</p><p>1998. <i>MUC-7: Proc. of the Seventh Message Under­standing Conf. </i>Defense Advanced Research Projects Agency.</p><p>M. Negri, E. Saquete, P. Martnez-Barco, and R. Muoz. 2006. Evaluating Knowledge-based Approaches to the Multilingual Extension of a Temporal Ex­pression Normalizer. In Association for Computa­tional Linguistics, editor, <i>ARTE: Workshop of44th Annual Meeting ofthe Association for Computa­tional Linguistics, </i>pages 30-37, Sydney, Australia.</p><p>M. Negri. 2007. Dealing with italian temporal expres­sions: The ita-chronos system. In <i>Proceedings of EVALITA 2007, Workshop held in conjunction with AI*IA.</i></p><p>J. Pustejovsky, J. Castao, R. Ingria, R. Saur, R. Gaizauskas, A. Setzer, and G. Katz. 2003a.</p><p>TimeML: Robust Specification of Event and Tem­poral Expressions in Text. In <i>Proc. ofthe Fifth Int. Workshop on Computational Semantics(IWCS-5)</i>.</p><p>James Pustejovsky, Jose M. Castano, Robert Ingria, Roser Sauri, Robert J. Gaizauskas, Andrea Setzer, Graham Katz, and Dragomir R. Radev. 2003b. TimeML: Robust Specification of Event and Tempo­ral Expressions in Text. In <i>New Directions in Ques­tion Answering, </i>pages 28-34.</p><p>J. Pustejovsky, R. Knippen, J. Littman, and R. Saur. 2005. Temporal and event information in natural language text. <i>Language Resources and Evaluation, </i>39:123-164.</p><p>E. Saquete, R. Muoz, and P. Martnez-Barco. 2006. Event ordering using terseo system. <i>Data and Knowledge Engineering Journal, </i>(58):70-89.</p><p>Technical Committee ISO/TC 154. 2004. Processes, data elements and documents in commerce, industry and administration "ISO 8601:2004(E)".</p><p>TempEx.       2008.       MITRE Corporation.</p><p>http://timex2.mitre.org/taggers/ timex2_taggers.html.</p><doubt alpha="58.3" length="24" tooSmall="False" monospace="0.0">TERN. 2004. Time Expres-</doubt><p>sion      Recognition      and Normalization.</p><p>http://timex2.mitre.org/tern.html.</p><p>Marc Verhagen, Inderjeet Mani, Roser Sauri, Jes­sica Littman, Robert Knippen, Seok Bae Jang, Anna Rumshisky, John Phillips, and James Puste-jovsky. 2005. Automating Temporal Annotation with TARSQI. In ACL. The Association for Com­puter Linguistics.</p><p>P. Vossen. 2000. EuroWordNet: Building a Multilin­gual Database with WordNets in 8 European Lan­guages. <i>The ELRA Newsletter, </i>5(1):9-10.</p></references></body></article>