<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="654"/><title>Establishing Implementation Priorities in Aiding Writers of Controlled Crisis Management Texts</title><pubinfo>Proceedings of Recent Advances in Natural Language Processing,pages 654-659, Hissar, Bulgaria, 12-14 September 2011.</pubinfo><author surname="Temnikova" givenname="Irina"><org  name="University of Wolverhampton" country="United Kingdom" city="Wolverhampton"/></author></firstpageheader><frontmatter><p><b>Establishing Implementation Priorities in Aiding Writers of Controlled</b></p><p><b>Crisis Management Texts</b></p><p><b>Irina Temnikova</b></p><p>Research Institute in Information and Language Processing University of Wolverhampton, UK</p><p>irina.temnikova@gmail.com</p></frontmatter><abstract>As clarity in the Crisis Management do­main is crucial, and there exists an enor­mous amount of Crisis Management doc­uments, a specific language resource (the Controlled Language for Crisis Manage­ment, CLCM) for editing Crisis Manage­ment instructions in English has been pre­viously developed. Based on a specially designed controlled language evaluation experiment, we have determined that man­ual simplification, far from being easy, is an extremely time-consuming process and thus automatization is essential in order to facilitate the writing of clear instruc­tions. This article describes this experi­ment which also aims to determine which operations should be privileged and are more urgent to be implemented in order to address the most critical issues first. </abstract></header><body><section number="1" title="Introduction"><p>Attention paid to the Crisis Management domain has strongly increased in recent years (Schneid and Collins, 2001), due to the urgent need to guar­antee safe and efficient management of emergency situations. There exists an enormous amount of al­ready written crisis management documents and new ones are being created with exponentially growing speed. Efficient communication between crisis management teams and local populations is crucial in situations in which there is a very short reaction time (Ogrizek and Guillery, 1999; Winerman, 2009). It is also known that human comprehension under stress is different from the one in normal conditions (Kiwan et al., 1999). For this reason the clarity and conciseness of the information exchanged during emergency situa­tions is crucial. A controlled language (CL) is a very good manual approach for ensuring clar­ity of crisis management texts. Unfortunately, it has been previously shown (Goyvaerts, 1996; Hui-jsen, 1998) that manual editing of texts accord­ing to controlled language guidelines is a diffi­cult and highly time-consuming process. Natural Language Processing (NLP) techniques are thus a good way to at least partially automatize and thus improve and speed up manual text simplifi­cation. This article introduces a controlled lan­guage for text simplification in the crisis manage­ment domain and its evaluation in terms of time and cognitive efforts for the human simplifiers and draws conclusions about which operations to im­plement first, in order to speed up and facilitate manual controlled language-based simplification. The article is structured as follows: Section 2 presents the related work in the crisis management domain and in Controlled Languages aids, Sec­tion 3 presents the Controlled Language for Cri­sis Management (CLCM), Section 4 describes the text simplification evaluation experiment, Section 5 discusses the results of the experiment and Sec­tion 6 provides some conclusions and future work.</p></section><section number="2" title="Related Work"><p>Although the large recent activity in crisis man­agement, there are not many NLP works in this domain. Roman (2008) has worked on redundancy identification from personal web blogs on emer­gency topics. Ireson (2009) has worked in Infor­mation Extraction in detection and monitoring of emergency events from open discussion forums. During the project EPIC, managed by the Univer­sity of Colorado at Boulder and the University of California, some work was done in the extraction of important information relevant to mass emer­gencies signaled in Twitter (Corvey et al., 2010). In medical crisis management, Chapman et al. (2005) have applied NLP approaches to syndromic surveillance in order to obtain free text classifi­cation of chief complaints. As the existing NLP approaches deal mainly with emergencies detection, this is the first known attempt to process text simplicity of instructions delivered to the general population during mass emergencies.<page local="2" global="655"/> In terms of controlled languages, CLCM is the first controlled language for English for this domain. The existing NLP tools for controlled language simplification are Controlled Language Editors (CLEs) and Con­trolled Language Checkers (CLCs). The CLEs are used in order to facilitate writing text according to controlled Language specifications, while the CLCs - to check whether an already created text is written in accordance with the controlled language rules. Compagnon LiSe is an example of a very simple CLE that facilitates writing sentences ac­cording to the controlled language rules but does not apply any NLP techniques. It has been de­veloped for the French version of CLCM (Renahy et al., 2010). Two other CLEs have been de­veloped for the machine-oriented controlled lan­guages PENG and ACE (Schwitter, 2008; Kuhn, 2009). Mitamura and Nyberg (2001) have de­veloped the KANT controlled language re-writing system which checks compliance with a machine translation-oriented controlled language. This ar­ticle aims to assist with the initial design of an NLP-based CLCM Editing Aid, in order to facil­itate text simplification in the crisis management domain.</p></section><section number="3" title="The Controlled Language for Crisis Management"><p>The Controlled Language for Crisis Management has been adapted to English on the basis of a controlled language for French (Renahy, 2009) in the context of MESSAGE Project<footnote anchor="1"/>. As a result of MESSAGE, four controlled languages for dif­ferent European languages have been developed (French, Spanish, Polish and English), together with two prototypes for Modern Greek and Bul­garian (Temnikova and Margova, 2009). CLCM has been developed on the basis of a collected cor­pus of crisis management documents, amounting to over 2.5 million words and collected from the web.</p><p>The existing version of CLCM applies only to instructions for the general public (GP), as these are considered to be the documents which most need simplification, as their audience members are not trained specialists. Although covering differ-</p><footnote label="1">http://message-project.univ-fcomte.fr/ Accessed 12 May 2011.</footnote><p><b>In_L_05: </b>Keep preposition and verb together in phrasal verbs.</p><p><u>Switch the lights-effi</u> <u>| <b><i>Switch offûie </i></b>lights.</u></p><p><b><i>Explanation: Preposition and verb separated by many words create difficulties for both non-native speakers and machine translation engines.</i></b></p><figure caption="Figure 1: Example of a CLCM rule."></figure><p>ent topics, these instructions have high document structure and language similarity, which allowed the development of a specific CL tailored to them. The role of CLCM is two-fold: on one hand to provide rules for the efficient simplification of ex­isting crisis management documents, and on the other hand - to provide rules for writing new crisis management documents. CLCM is easily trans­ferable to other domains' documents, containing instructions.</p><p>The CLCM features thirty pages of over eighty simplification rules, which address different text aspects, starting from general text structure and ending with punctuation, as well as different doc­ument elements (titles, conditions, instructions, lists). Below are provided examples of some of the existing rule types and in Figure 1 - a screen­shot of a rule taken from the CLCM guidelines:</p><p>• General: If there are distinguished situations:</p><p>— Identify the specific situations.</p><p>— Divide the blocks of instructions regarding the specific situations into subsections.</p><p>— Write first the most specific situation.</p><p>— Write the next more general situation.</p><p>— End with the most general situation.</p><p>• Formatting: Separate with a new line each block of instructions.</p><p>• Lexical: Use only words defined in the dictionary.</p><p>• Syntactic: Avoid passive voice</p><p>• Punctuation: Avoid any punctuation signs at the end of the titles.</p><p>As can be seen from Figure 1, each rule has a reference number which is formed by: the type of document ("In" = "instructions"), a number of rules are document type-specific; the type of rule ("L" = "lexical"); and a standard number. Also below each rule is shown an example of how text should not look according to this rule (stroke through) and how it should look instead.<page local="3" global="656"/> Some­times below these illustrative examples less im­portant information is provided, as for example an explanation of why the rule is necessary. Previ­ously two experiments have been conducted in or­der to evaluate CLCM. One experiment evaluated the impact of the controlled language on human translation (Temnikova and Orasan, 2009), while the second experiment evaluated the impact of the controlled language simplification on machine translation (Temnikova and Orasan, 2009; Tem-nikova, 2010). These experiments have shown that although CLCM was written for human readers, it had a significant impact on and improved the re­sults of both human and machine translation.</p></section><section number="4" title="Description of the Evaluation Experiment"><p>The aim of the experiment carried out was to eval­uate the quality of the CLCM guidelines. The experiment consisted in asking six linguists - En­glish advanced and native speakers with a compu­tational linguistics background, to read carefully and familiarise themselves with the CLCM simpli­fication guidelines and to simplify manually four texts of a total of two thousand words according to the simplification rules in these guidelines. In order to direct the participants and simplify their task in remembering over eighty rules, an assist­ing leaflet was provided. The leaflet contained the thirty most important rules to be consulted during simplification. The rules in the leaflet were clas­sified into three natural language generation-like groups:</p></section><section number="1." title="Rules for discourse structure organisation at text level;"><p>2. Rules for discourse structure organisation at paragraph level; 3. Concrete linguistic realization rules.</p><p>The experiment was performed in two stages dis­tributed over two days to avoid the impact of the factor of tiredness. The four texts were taken from the previously collected Crisis Management Cor­pus and represent instructions for the general pop­ulation in different emergency situations: precau­tions to be taken after a flood, instructions how to clean chemicals from clothing, actions to be taken after volcanic eruptions. Time is measured dur­ing the first time reading guidelines and during the manual simplification of each text. Table 1 shows the text lengths per text for each day of the exper­iment calculated in words.</p><p>As can be seen from the Table 1, the first two columns show which texts were presented to the participants each day while the last two columns provide the text lengths in words and in charac­ters. A text complexity analysis of the four origi­nal texts was run, by examining the main text com­plexity features according to literature. The results of this analysis are provided in Table 2.</p><p>In Table 2, the first column contains the text ref­erence number, while the following five columns -the text complexity features they have been anal­ysed for, namely:</p><p>• SL - average sentence length, measured in number of</p><doubt alpha="83.3" length="6" tooSmall="False" monospace="0.0">words;</doubt><p>• WL - average word length, measured in number of let­ters;</p><p>• SM - percentage of subordinating markers;</p><p>• LD - lexical diversity, measured as types/tokens ratio;</p><p>• WS - average number of word senses per word</p><p>In order to do this text complexity analysis, the texts were pre-processed using Connexor parser <footnote anchor="2"/> and WordNet (Fellbaum, 1998) was used for cal­culating the average number of senses per word.<page local="4" global="657"/></p><footnote label="2"> www.connexor.eu Accessed 12 May 2011.</footnote><table caption="Table 1: Lengths of texts used for the CLCM guidelines evaluation." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Day</b></p></td><td class="cell"><p><b>Text</b></p></td><td class="cell"><p><b>Words</b></p></td><td class="cell"><p><b>Chars</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Day 1</p></td><td class="cell"><p>Text 1</p></td><td class="cell"><p>166</p></td><td class="cell"><p>900</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Text 2</p></td><td class="cell"><p>833</p></td><td class="cell"><p>5018</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Total</b></p></td><td class="cell"><p><b>Dayl</b></p></td><td class="cell"><p><b>999</b></p></td><td class="cell"><p><b>5918</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Day 2</p></td><td class="cell"><p>Text 3</p></td><td class="cell"><p>271</p></td><td class="cell"><p>1562</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Text 4</p></td><td class="cell"><p>728</p></td><td class="cell"><p>4486</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Total</b></p></td><td class="cell"><p><b>Day 2</b></p></td><td class="cell"><p><b>999</b></p></td><td class="cell"><p><b>6048</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Total</b></p></td><td class="cell"><p><b>Day 1 and 2</b></p></td><td class="cell"><p><b>1998</b></p></td><td class="cell"><p><b>11966</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table caption="Table 2: Text Complexity analysis of the four texts." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Text</b></p></td><td class="cell"><p><b>SL</b></p></td><td class="cell"><p><b>WL</b></p></td><td class="cell"><p><b>SM</b></p></td><td class="cell"><p><b>LD</b></p></td><td class="cell"><p><b>WS</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text1</p></td><td class="cell"><p>12.69</p></td><td class="cell"><p>4.24</p></td><td class="cell"><p>3.64</p></td><td class="cell"><p>0.50</p></td><td class="cell"><p>11.48</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text 2</p></td><td class="cell"><p>16.76</p></td><td class="cell"><p>4.94</p></td><td class="cell"><p>4.67</p></td><td class="cell"><p>0.42</p></td><td class="cell"><p>8.08</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text 3</p></td><td class="cell"><p>14.83</p></td><td class="cell"><p>4.68</p></td><td class="cell"><p>5.62</p></td><td class="cell"><p>0.39</p></td><td class="cell"><p>7.95</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text 4</p></td><td class="cell"><p>14.74</p></td><td class="cell"><p>5.03</p></td><td class="cell"><p>4.87</p></td><td class="cell"><p>0.44</p></td><td class="cell"><p>8.86</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>At the end of the second day, the participants were asked to fill in a questionnaire asking details regarding the work they had done in the previous two days. The questionnaire collected data in three parts - Part 1 was asking for personal comments in the form of free text, Part 2 was providing a list of rules to be evaluated in terms of how difficult they are to be applied, while Part 3 was suggesting a list of implementations to be rated. The personal com­ments in the first part of the questionnaire were requested by the following question: "Could you think of what was most difficult for you while simplifying?". The rules, provided in the second part were those thirty most important rules, con­tained in the assisting leaflet. The participants were asked to write next to each rule whether it was easy or difficult to apply and mark those ones which, if automated, would speed up their work. The suggested implementations in Part 3 were of­fering preliminary easy-to-implement operations which would result in highlighting different text elements. The text elements to be highlighted ranged from single words to whole paragraphs and were CLCM-specific. The participants were asked to give scores to these operations, according to the following ranking: 1 - Implementing this opera­tion will not help at all; 2 - Implementing this op­eration will help to a certain extent; 3 - Implement­ing this operation will help very much.</p><p>Additionally, most of the participants provided useful feedback on the design of the experiment and the future implementation, thanks to their NLP background.</p></section><section number="5" title="Results of the Experiment"><p>The analysis of the results of the experiment pro­vided useful information about the internal pro­cess of manual simplification of texts according to the CLCM rules and shows that text simplification is not a trivial task, even when precise guidelines are provided. The analysis of the times and speed for reading the guidelines show that the average time for reading the guidelines for the first time was between 30 to 45 minutes. The speeds for simplifying manually the texts are given in Table 3.</p><p>The first column of Table 3 indicates the par­ticipant, while the next four columns - the four texts. The values in the table are given in characters/minute, in order to take into consideration the different length of texts. Obviously, although the value of Subject 2 is the outlier of the sample. can be clearly seen that the speed ranges between 18.7 to 358.4 characters per minute, depending on the text complexity and the subject. I.e. the speed dif­ference is 20 times. Even if a clear learning effect is visible from the data, it is still clear that simpli­fying text takes a large amount of time. The ta­ble also provides the mean and standard deviation values with and without the outlier. Row "mean 6" provides the mean values of all six participants, to­gether with the outlier, while row "mean 5" - only of the five participants, excluding the outlier. In a similar way, row "st.dev. 6" provides the stan­dard deviation values of all six participants, while row "st.dev. 5" the standard deviations without the outlier. The standard deviation values excluding Subject 2 decreases significantly, for example, for Text 2 from 120.4 to 14.9.</p><p>Another demonstration of the fact that manual simplification is not a trivial task is the results provided by the questionnaire the language ex­perts were asked to fill in at the end of the second day. The aim of this questionnaire was to deter­mine which rules were most difficult to apply and which simplification operations would need to be automatized. As mentioned before, the question­naire was composed of three parts: Part 1 contain­ing personal comments in free text, Part 2 contain­ing the list of the thirty main rules to be evalu­ated as "easy"/"difficult" to be manually applied and whether to be implemented or not and Part 3 containing a list of suggested implementations to be ranked as "will not help at all"/"will help to a certain extent"/"will help very much". Surpris­ingly, most of the subjects have very similar per­sonal comments in answering the question, posed in Part 1 "Could you think of what was most difficult for you while simplifying?<page local="5" global="658"/>". The answers were put in a common table and were given a mark "1" if a Subject has mentioned it in the free com­ments and "0" if not. The marks were added and averages were obtained. In this way, the top four results were, ordered from the one with the highest score to the one with the lowest score:</p><table caption="Table 3: Manual simplifying speed per subject and per text, measured in characters per minute." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Subject</b></p></td><td class="cell"><p><b>Text 1</b></p></td><td class="cell"><p><b>Text 2</b></p></td><td class="cell"><p><b>Text 3</b></p></td><td class="cell"><p><b>Text 4</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub1</b></p></td><td class="cell"><p>21.9</p></td><td class="cell"><p>69.7</p></td><td class="cell"><p>71.0</p></td><td class="cell"><p>203.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub2</b></p></td><td class="cell"><p>75.0</p></td><td class="cell"><p><b>358.4</b></p></td><td class="cell"><p>173.6</p></td><td class="cell"><p>263.9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub3</b></p></td><td class="cell"><p>30.0</p></td><td class="cell"><p>47.8</p></td><td class="cell"><p>78.1</p></td><td class="cell"><p>149.5</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub4</b></p></td><td class="cell"><p>30.0</p></td><td class="cell"><p>83.6</p></td><td class="cell"><p>97.6</p></td><td class="cell"><p>121.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub5</b></p></td><td class="cell"><p><b>18.7</b></p></td><td class="cell"><p>52.3</p></td><td class="cell"><p>33.9</p></td><td class="cell"><p>48.2</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Sub6</b></p></td><td class="cell"><p>33.3</p></td><td class="cell"><p>72.7</p></td><td class="cell"><p>67.9</p></td><td class="cell"><p>115.0</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>mean 6</b></p></td><td class="cell"><p><b>34.8</b></p></td><td class="cell"><p><b>114.1</b></p></td><td class="cell"><p><b>86.5</b></p></td><td class="cell"><p><b>150.3</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>st.dev. 6</b></p></td><td class="cell"><p><b>18.7</b></p></td><td class="cell"><p><b>110.0</b></p></td><td class="cell"><p><b>43.0</b></p></td><td class="cell"><p><b>68.7</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>mean 5</b></p></td><td class="cell"><p><b>26.8</b></p></td><td class="cell"><p><b>65.2</b></p></td><td class="cell"><p><b>69.7</b></p></td><td class="cell"><p><b>127.6</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>st.dev. 5</b></p></td><td class="cell"><p><b>5.5</b></p></td><td class="cell"><p><b>13.3</b></p></td><td class="cell"><p><b>20.7</b></p></td><td class="cell"><p><b>50.6</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>• Avoiding   negatives/Re-phrasing negative phrases.</p><p>• Remembering to remove pronouns/Avoiding pronouns.</p><p>• Being mindful of word difficulty/Replacing technical terms.</p><p>• Re-organizing and re-grouping the content of the original.</p><p>In Part 2, the marks were given different weights in the following way:</p><doubt alpha="53.8" length="26" tooSmall="False" monospace="0.0">•"no answer" or "easy" = 0</doubt><doubt alpha="53.3" length="15" tooSmall="False" monospace="0.0">•"simplify" = 1</doubt><doubt alpha="47.1" length="17" tooSmall="False" monospace="0.0">•"moderate" = 1.5</doubt><doubt alpha="56.2" length="16" tooSmall="False" monospace="0.0">•"difficult" = 2</doubt><doubt alpha="64.5" length="31" tooSmall="False" monospace="0.0">•"difficult" and "simplify" = 3</doubt><doubt alpha="61.9" length="21" tooSmall="False" monospace="0.0">•"very difficult" = 4</doubt><doubt alpha="66.7" length="36" tooSmall="False" monospace="0.0">•"very difficult" and "simplify" = 5</doubt><p>As the participants have used different combina­tions of marks, the conclusive marks have been given weights in correspondence with the apparent ranks of the different marks. The results of Ques­tionnaire Part 2 were added and averages were ob­tained. The top twelve results are shown in the</p><p>Table 4.</p><p>Table 4 has two columns, the first one indicating the CLCM rule, while the second one - the average score, obtained after adding the scores, provided by the participants. "N" stands for "noun", while "V" stands for "verb". The top ten results of Part 3 are given in Table 5. The results were again added and averages were ordered from the highest to the lowest one.</p><p>Table 5 is composed, like Table 4, by two columns. The first column contains the suggested implementations, while the second column - the average scores, obtained by adding the participant scores and dividing them per number of partici­pants. The top results confirm that the rules found more difficult to apply manually by participants are those which are tackling cognitively hard to process linguistic phenomena (negation, passive, ambiguity). This makes an NLP application a good solution to the aforementioned problems.</p></section><section number="6" title="Conclusions and Future Work"><p>The analysis of the results of the experiment shows that human simplifiers employ too much time in simplifying even short texts and thus simplifying is not a trivial task. More particularly, the re­sults collected from the questionnaire show that the simplifiers mostly agreed on the set of dif­ficult rules and on the set of suggested imple­mentations. Future work will include cognitively analysing rules' formulations before proceeding with any NLP implementation. For example the rule "Avoid negative forms" may be difficult to apply, as which negative forms to avoid are not concretely defined in the guidelines. Otherwise NLP techniques may be applied in a way to per­form negative forms recognition and suggestion of alternative positive forms. While some of the suggested implementations could be solved by an appropriate training, others, such as "highlight in the text the phrasal verbs in case the main verb and the preposition are split up" cannot and would help to be implemented.<page local="6" global="659"/> On the basis of the con­clusions drawn from this very useful experiment, future work will be to apply some of the sug­gested implementations and to find automatic so­lutions to the highest ranked manual rules as first steps towards a high-level NLP-based Controlled Language Editing Aid. As "avoiding negatives" was listed as first choice in Part 1 and Part 2 and also had one of the highest scores in Part 3, we choose it as the most urgent issue to be solved and possibly implemented. Negation implementation would include constructing patterns for recogniz­ing negation to avoid in emergency instructions, based on the collected corpus and building a gram­mar to help supplying the user with positive al­ternatives to negated phrases. Another candidate for implementation is, of course, "Highlighting the ambiguous lexical terms", which has emerged as the suggested implementation with the highest ranking score (2.67). Future work would also in­clude testing whether more appropriate training of human simplifiers would change the rules consid­ered difficult to apply.</p><table caption="Table 5: Questionnaire Part 3 results." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Rule</b></p></td><td class="cell"><p><b>Score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Try to avoid negative forms</p></td><td class="cell"><p><b>3</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Replace passive with active voice</p></td><td class="cell"><p><b>2.17</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Avoid any pronouns (person., poss., demonst.)</p></td><td class="cell"><p><b>2</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Avoid ambiguous words</p></td><td class="cell"><p><b>1.83</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Replace idiomatic expressions with literal ones</p></td><td class="cell"><p><b>1.83</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Replace techn. terms with common synonyms</p></td><td class="cell"><p><b>1.83</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Order instructions in logic. and chronol. order</p></td><td class="cell"><p><b>1.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>If 2+ complem. determ. the same N, repeat the N</p></td><td class="cell"><p><b>1.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Write only one action per line</p></td><td class="cell"><p><b>1.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>If a prep./adj. refers to 2+ N, repeat the prep/adj.</p></td><td class="cell"><p><b>1.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Use standard word order</p></td><td class="cell"><p><b>1.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Place conditions before instructions</p></td><td class="cell"><p><b>1.5</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Table 4: Questionnaire Part 2 results.</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Rule</b></p></td><td class="cell"><p><b>Score</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the ambiguous lexical terms</p></td><td class="cell"><p><b>2.67</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the phrasal verbs</p></td><td class="cell"><p><b>2.5</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the separate thematic situations</p></td><td class="cell"><p><b>2.5</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the negative phrases</p></td><td class="cell"><p><b>2.33</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the ambiguous syntact. expressions</p></td><td class="cell"><p><b>2.33</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the technical terms</p></td><td class="cell"><p><b>2.33</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the beginning of instructions</p></td><td class="cell"><p><b>2.17</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the beginning of conditions</p></td><td class="cell"><p><b>2.17</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the beginning of explanations</p></td><td class="cell"><p><b>2.17</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Highlighting the acronyms and abbreviations</p></td><td class="cell"><p><b>2.17</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table></section><references><p>Chapman, W.W., et al. (2005) <i>Classifying free-text triage chief complaints into syndromic categories with natural language processing. </i>Artif Intell Med. 2005 Jan; 33(1):31-40.</p><p>Corvey, W. J., Vieweg, S., Rood, T. and Palmer, M. (2010) <i>Twitter in Mass Emergency: What NLP Techniques can Contribute. </i>In Proceedings of the</p><p>NAACL HLT 2010 Workshop on Computational</p><p>Linguistics in a World of Social Media (Los Ange­les, California, June 2010), 2324.</p><p>Fellbaum, C. (1998) <i>WordNet: an Electronic Lexical Database </i>MIT Press.</p><p>Goyvaerts P. (1996) <i>Controlled English, curse or blessing? A users perspective. </i>Proceedings of CLAW 1996.</p><p>Huijsen, W.O. (1998) <i>Controlled Language An In­troduction. </i>Proceedings of the Second International Workshop on Controlled Language Applications. Pi-itsburgh, Pennsylvania.</p><p>Ireson, N. (2009) <i>Local Community Situation Aware­ness During an Emergency. </i>Proceedings of the IEEE International Conference on Digital Ecosys­tems and Technologies (IEEE-DEST 2009).</p><p>Kiwan, D., Ahmed, A. and Pollitt, A. (1999) <i>The ef­fects oftext comprehension and performance in ex­aminations. </i>Proceedings of BPS London Confer­ence, December, 1999.</p><p>Kuhn, T. (2009) <i>Controlled English for Knowledge Representation. Ph.D. Thesis. </i>University of Zurich, Switzerland.</p><p>Mitamura, T. and Nyberg, E. (2001) <i>Automatic rewrit­ing for controlled language translation. </i>Proceed­ings of the NLPRS-2001 Workshop on Automatic Paraphrasing: Theories and Applications. Tokyo, Japan. Pages 1-12.</p><p>Ogrizek, M. and Guillery, J-M. (1999) <i>Communicat­ing in crisis. </i>Transaction Publishers.</p><p>Quirk, R., Greenbaum, S., Leech, G., Svartvik, J. (1985) <i>A comprehensive grammar </i><i>ofthe</i><i> English language. </i>Harlow: Longman. Pp. 1779.</p><p>Renahy J. (2009) <i>Controlled Languages: a Scien­tific Popularization through the Example of the Con­trolled Language "LiSe". </i>ISMTCL Proceedings, International Review Bulag, pp 215-222.</p><p>Renahy, J. et al. (2010) <i>Development and Evalua­tion </i><i>ofa</i><i> Controlled Language and </i><i>ofa</i><i> computer­ized writing assistant LiSe to improve the quality and safety ofmedical protocols. </i>International Fo­rum on Quality and Safety of Health Care. 20-23 April 2010, The Nice Acropolis, Nice, France.</p><p>Roman, J.H., et al. (2008) <i>Reducing information over­load in emergencies by detecting themes in Web con­tent. </i>Proceedings of the 5th International ISCRAM Conference 2008;101-6.</p><p>Schneid, T.D. and Collins, L. (2001) <i>Disaster man­agement and preparedness. </i>Lewis Publishers.</p><p>Schwitter, R. (2008) <i>A Controlled Natural Language for the Semantic Web. </i>Journal of Intelligent Sys­tems, 17, pp.125-141.</p><p>Temnikova, I. (2010) <i>A Cognitive Evaluation Approach for a Controlled Language Post-Editing Experiment. </i>Proceedings of the International Conference "Language Resources and Evaluation" (LREC2010), Valletta, Malta.</p><p>Temnikova, I. and Margova, R. (2009) <i>Towards a Con­trolled Language in Crisis Management: The Case of Bulgarian. </i>Proceedings of the International Sym­posium on Data and Sense Mining, Machine Trans­lation and Controlled Languages (ISMTCL), Besan­con, France, July 1-3, 2009.</p><p>Temnikova, I. and Orasan, C. (2009) <i>Post-editing Ex­periments with MT for a Controlled Language. </i>Pro­ceedings of the International Symposium on Data and Sense Mining, Machine Translation and Con­trolled Languages (ISMTCL), Besancon, France, July 1-3, 2009.</p><p>Winerman, L. (2009) <i>Crisis Communication. </i>Nature, vol. 457, p 376.</p></references></body></article>