<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="1062"/><title>Enhancing extraction based summarization with outside word space</title><pubinfo>Proceedings of the 5th International Joint Conference on Natural Language Processing,pages 1062-1070, Chiang Mai, Thailand, November 8 - 13, 2011. ©2011 AFNLP</pubinfo><author surname="Smith" givenname="Christian"><org  name="Santa Anna IT Research Institute Linkoping" country="Canada"/></author><author surname="Jönsson" givenname="Arne"><org  name="Santa Anna IT Research Institute Linkoping" country="Canada"/></author></firstpageheader><frontmatter><p><b>Enhancing extraction based summarization with outside word space</b></p><p><b>Christian Smith, Arne Jonsson</b></p><p>Santa Anna IT Research Institute AB Linkoping, Sweden christian.smith@liu.se, arnjo@ida.liu.se</p></frontmatter><abstract>We present results from improving vec­tor space based extraction summarizers. The summarizer uses Random Indexing and Page Rank to extract those sentences whose importance are ranked highest for a document, based on vector similarity. Originally the summarizer used only word vectors based on the words in the docu­ment to be summarized. By using a larger word space model the performance of the summarizer was improved. Along with the performance, robustness was improved as random seeds did not affect the summa-rizer as much as before, making for more predictable results from the summarizer. </abstract></header><body><section number="1" title="Introduction"><p>Many persons have, for various reasons, problems assimilating long complex texts. Not only persons with visual impairments or dyslexia, but also, for instance, those having a different mother tongue or persons in need of a quick summary of a text. A tool for automatic summarization of texts from different genres as an aid in reading can thus be useful for many persons and purposes.</p><p>Automatic summarization can be done in vari­ous ways. A common distinction is extract versus abstract summaries. An extract summary is cre­ated by extracting the most important sentences from the original text so that the result is a shorter version of the original text with some information still present, for instance the most important sen­tences or words. An abstract summary on the other hand is a summary where the text has been broken down and rebuilt as a complete rewrite to convey a general idea of the original text. Furthermore, the summaries can be indicative (for instance only providing keywords as central topics) or informa­tive (content focused) (Firmin and Chrzanowski, 1999). The former might be more usable when a reader needs to decide whether or not the text is in­teresting to read and the latter when a reader more easily needs to get a grasp of the meaning of a text that is supposed to be read.</p><p>In this paper we will examine and try to increase the performance of an automatic extraction-based summarizer. Previously, the summarizer has been functioning without aid of outside corpora or train­ing material. While the performance have been good, some improvements utilizing an outside cor­pus can be achieved.</p><p>The technique behind the summarizer will first be described in more detail, after which some re­sults are presented which indicates that the perfor­mance can be enhanced by using outside training material.</p></section><section number="2" title="The word space model"><p>The word space model, or vector space model (Elden, 2007), is a spatial representa­tion of a word's meaning that can reduce the linguistic variability and capture semantically related concepts by taking into account the positioning of words in a multidimensional space, instead of looking at only shallow linguistic properties. This facilitates the creation of sum­maries, since the positioning in the word space can be used to evaluate the different passages (words or sentences for instance) in relation to a document with regards to informational and semantic content.</p><p>Every word in a given context occupies a spe­cific point in the space and has a vector associated to it that can be used to define its meaning.</p><p>Word spaces are constructed according to the distributional hypothesis and the proximity hy­pothesis. In the distributional hypothesis, words that occur in similar contexts have similar mean­ings so that a <i>word </i>is the sum of its contexts and the <i>context </i>is the sum of its words, where the context can be defined as the surrounding words or the entire document.<page local="2" global="1063"/> The proximity hypothesis states that words close to each other in the word space have similar meaning while those far from each other have dissimilar meaning.</p><p>The word space can be constructed from a ma­trix where text units are columns and the words in all text units are rows in the matrix. A certain entry in the matrix is nonzero iff the word correspond­ing to the row exists in the text unit represented by the column. The resulting matrix is very large and sparse which makes for the usage of techniques for reducing dimensionality and get a more com­pact representation. Latent Semantic Analysis is one such technique that, however, can be com­putationally expensive unless used with alterna­tive algorithms (Gorrell, 2006). Random Index­ing (Sahlgren, 2005; Kanerva, 1988) is another di­mension reduction technique based on sparse dis­tributed representations that provide an efficient and scalable approximate solution to distributional similarity problems.</p></section><section number="3" title="The summarizer"><p>CogSum is an extraction based summarizer, us­ing the word space model Random Indexing (RI), c.f. Hassel (2007) and a modified version of PageRank (Brin and Page, 1998).</p><p>In Random Indexing context vectors are accu­mulated based on the occurrence of words in con­texts. Random Indexing can be used with any type of linguistic context, is inherently incremental, and does not require a separate dimension reduc­tion phase as for instance Latent Semantic Analy­sis.</p><p>Random Indexing can be described as a two-step operation:</p><p><b>Step 1 </b>A unique d-dimensional <i>index vector </i>is assigned and randomly generated to each context (e.g. each document or each word). These index vectors are sparse and high-dimensional. They consist of a small number, p, of randomly distributed +1s and -1s, with the rest of the elements of the vectors set to 0.</p><p><b>Step 2 </b><i>Context vectors </i>are produced on-the-fly. As scanning the text, each time a word occurs in a context, that context's d-dimensional in­dex vector is added to the context vector for the word. The context window defines a re­gion of context around each word, and the number of adjacent words in a context win­dow is called the context window size, w. For example, with w — 2, i.e. a 2x2 context win­dow, the word on is represented by the con­text window cm as:</p><doubt alpha="30.0" length="30" tooSmall="False" monospace="0.0">=[(°n-2)(°n-1)on(on+l)(on+2)]&gt;</doubt><p>and the context vector of o<b>n </b>in c<b>m </b>would be updated with:</p><doubt alpha="50.0" length="4" tooSmall="False" monospace="0.0">Cm —</doubt><doubt alpha="43.8" length="32" tooSmall="False" monospace="0.0">R(on-2)+R(on-l)+R(on+l)+R(on+2),</doubt><p>where R(x) is the random index vector of x. This process is repeated every time we ob­serve o<b>n </b>in our data, adding the correspond­ing information to its existing context vector <i>C</i>. If the context cm is encountered again, no new index vector will be generated. Instead the existing index vector for c<b>m </b>is added to C to produce a new context vector for o<b>n.</b></p><p>Words are thus represented by d-dimensional context vectors that are effectively the sum of the index vectors of all the contexts in which the word appears.</p><p>Additionally, the vectors within the sliding contex window can be weighted according to the distance to the focus word. one example context window providing a larger weight for words closest to the focus word (Karlgren and Sahlgren, 2001).</p><doubt alpha="37.0" length="46" tooSmall="False" monospace="0.0">is2(1-distance),or [0.5,1, 0,1, 0.5] for a 2x2</doubt><p>After the creation of word context vectors, the similarity between words could be measured by calculating the cosine angle between their word vectors, by taking the scalar product of the vectors and dividing by their norms such as:</p><doubt alpha="44.4" length="18" tooSmall="False" monospace="0.0">Cos(x,y)=T^-pr (1)</doubt><doubt alpha="33.3" length="6" tooSmall="False" monospace="0.0">|x|\y\</doubt><p>Random Indexing is useful for acquiring the context vectors of terms, it is however not clear how a bigger context, such as a sentence, could be built from the word vectors. A crude way of cre­ating sentence vectors from word vectors would be to simply summarize the vectors of the words in the sentence after they have been normalized to unit length. However, as the number of words in a sentence increases, so will the sentence similar­ity to the mean vector.<page local="3" global="1064"/> Comparing sentences or documents in this way using cosine will make for larger similarity just by a larger number of words, regardless of relatedness. To alleviate this prob­lem, the mean document vector is subtracted from each of the sentence's word vectors before sum­marizing the vectors (Higgins and Burstein, 2007), see Equation 2.</p><doubt alpha="50.0" length="18" tooSmall="False" monospace="0.0">1s^ sentj=—doc)(2)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">i=1</doubt><p>where S denotes the number of words, w, in sen­tence j and <i>doc </i>is calculated as in Equation 3.</p><doubt alpha="56.2" length="16" tooSmall="False" monospace="0.0">-1Ndoc=jjJ^Wi(3)</doubt><p>where N denotes the number of unique words.</p><p>Words that are similar to the document vector will come closer to the zero vector, while those dissimilar to the document vector will increase in magnitude. When later summarizing the vectors, those of greater magnitude will have increased im­pact on the total sentence vector so that common, non-distinct, words not contribute as much to the sentence vector. As this reduces the impact of common non-distinct words, there is essentially no need for a stop word list.</p><p>CogSum also uses the Weighted PageRank al­gorithm in conjunction to its RI-space to rank the sentences (Chatterjee and Mohan, 2007).</p><p>The method of using graph-based ranking algo­rithms for extracting sentences in summarization purposes was proposed by Mihalcea (2004), who introduce the TextRank model. In graph-based al­gorithms such as TextRank the text need to be rep­resented as a graph, where each vertex depicts a unit of text and the edges between the units repre­sent a connection between the corresponding text units. Graph-based ranking algorithms may be used to decide the importance of a vertex within a graph, by taking into account global information from the entire graph, rather than from only the local context of the vertices. The ranks are thus recursively computed so that the rank of a vertex depends on all the vertices' ranks. In TextRank, PageRank is used to rank the sentences, although it is noted that other ranking algorithms are possi­ble. PageRank is a graph-based ranking algorithm which originally was used to rank home pages au­tomatically and objectively in the Google search engine (Brin and Page, 1998). In TextRank, for the task of sentence extraction, each sentence in a text is represented as a vertex and the relation between sentences are based on their overlap or "similarity", denoted by Equation 4.</p><doubt alpha="65.8" length="73" tooSmall="False" monospace="0.0">Similaritv(S-S-)=|{w|wfcGSi&amp;WfcGSj}|Simüarity(Si,Sjjiog(\Si\) + log(\S31)</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(4)</doubt><p>Thus, if a sentence adresses certain concepts, the other sentences that share content will get rec­ommended by that sentence in the recursive fash­ion provided by PageRank.</p><p>To use PageRank and Random Indexing for summaries an undirected fully connected graph is created where a vertex depicts a sentence in the current text and an edge between two different ver­tices is assigned a weight that depicts how similar these are based on a cosine angle comparison of their meaning vectors, see Figure 1. As it is fully connected, all vertices are connected with each other and all out-links are also considered as in­links.</p><p>sentence^ sentenceg</p><p>Figure 1: A simplified graph where sentences are linked and weighted according to the cosine values between them.</p><p>The algorithm rank ingoing and outgoing links to pages depending on the number of links as fol­lows, Equation 5:</p><doubt alpha="51.7" length="29" tooSmall="False" monospace="0.0">PRW(si)= (1-d)+d*]TWj,PRW(Sj)</doubt><p><b><i>s</i></b><i>j</i><b><i>In(s</i></b><i>i) e</i><b><i>Out(Sj) </i></b>W<b>kj</b></p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(5)</doubt><p>where <i>s</i><b><i>i </i></b>is the sentence under consideration, <i>In(s</i><b><i>i)</i></b><b><i> </i></b>is the set of sentences that link to si, <i>Out(s</i><b><i>j) </i></b>is the set of sentences that link from siand <i>d </i>is the damping factor.</p><page local="4" global="1065"/><p>The damping factor was originally set to ac­count for the possibility of a surfer clicking a ran­dom web link when (s)he gets bored (Brin and Page, 1998). With regards to the ranking of sen­tences, we see the damping factor as the possi­bility of a sentence containing some implicit in­formation that a certain reader might consider more important at the time, following an analogy by Mihalcea and Tarau (2004). The PageRank-algorithm utilizes the "random surfer model" and using weighted PageRank in text comparison uti­lizes "text surfing" in the context of text cohesion. The links in the sentence graph might be attributed to links between connected concepts or topics se-mantically, creating a "web" of understanding on which a reader might surf.</p><p>The computation is carried out on all sentences iteratively until node weights converge, see Fig­ure 2.</p><doubt alpha="0.0" length="16" tooSmall="False" monospace="0.0">0 10 20 30 40 50</doubt><p>Iterations</p><p>high-ranked sentences provide bigger support than a greater number of low-ranked sentences. This leads to a ranking of the sentences by their impor­tance to the document at hand and thus to a sum­mary of desired length only including the most im­portant sentences.</p><p>When the text has been processed using RI and PageRank, the most important sentences are ex­tracted using the final ranks on the sentences, for instance 30% of the original text, resulting in a condensed version of the original text with the most important information intact, in the form of extracted sentences. Since all sentences are ranked, the length of the summary is easy to spec­ify, in CogSum this is implemented as a simple slider. CogSum is designed for informative sum­maries, but it is also possible to have indicative summaries by clicking a "keywords" check box.</p><p>CogSum is written in Java and utilizes a Ran­dom Indexing toolkit available at Hassel (2011a). No outside material is used which makes the sum-marizer highly portable and usable for several lan­guages and domains.</p><p>Previous evaluations of CogSum with human users show that summaries produced by Cog-Sum are useful, considered informative enough and readable (Jonsson et al., 2008). CogSum has also been evaluated on gold standards for news texts and authority texts showing that it is bet­ter than another Swedish summarizer, SweSum, (Dalianis, 2000) on authority texts and almost as good on news texts, texts that the other summa-rizer was especially adapted to handle (Gustavsson and Jonsson, 2010).</p></section><section number="4" title="Multi-document word vectors"><p>Figure 2: Each line represents a sentence from a single text with its weight plotted on the y-axis on each iteration on the x-axis. 50 iterations are plot­ted.</p><p>The ranking algorithm does not rely only on lo­cal context information (vertex) but draws infor­mation recursively from the entire graph. Sen­tences with similar content will then contribute with positive support to each other through a rec­ommendation process, where the sentences' ranks are increased or decreased each iteration. This does not exclusively depend on the number of sen­tences supporting a sentence, but also on the rank of the linking sentences. This means that a few</p><p>CogSum has previously worked without aid from any outside source making it highly portable and more or less language independent. However, some problems have been detected. We identified some abruptness in the resulting summaries, af­fected by the random factor of the index vectors. This was regardless of setting of dimensionality and other parameters.</p><p>To investigate the effect of randomness several summaries with different index vectors were cre­ated. The final ranks of the sentences in a text after the summarization process were calculated and plotted on each random seed that held its own distribution of the ones in the index vectors, Fig­ure 3. The figure shows 10 different summaries with their own seeds.<page local="5" global="1066"/> The final values after the ranking are plotted on the y-axis mapped to each seed on the x-axis. A straight line would mean that the results are predictable and not affected as much by randomness. As can be seen in Figure 3 there is quite some randomness in which sentences that are chosen depends on the seed to the Random Indexing algorithm.</p><doubt alpha="0.0" length="10" tooSmall="False" monospace="0.0">2 4 6 8 10</doubt><doubt alpha="100.0" length="5" tooSmall="False" monospace="0.0">Seeds</doubt><p>Figure 3: Ten different seeds without pretrained space. Each series represents a sentence in a text. The values on the y-axis are the final values of each sentence after the PageRank-algorithm.</p><p>For COGSUM, we wanted to extend the method by using an outside larger RI-space. Using a large RI-space, a better semantic representation of the words can be acquired (Sahlgren, 2006). By extending the method to use an outside training space we thus believe that the quality and robust­ness of the summaries can be improved.</p><p>COGSUM takes as input the text to be summa­rized, but now also a previously trained RI-space is supplied, containing the semantic vectors of the words.</p><p>The RI-space was created from several Swedish texts from different genres, all in all approxi­mately 240 000 words. The articles consisted of a number of novels in a subset of the Stockholm-Umea Corpus (Ejerhed et al., 2006), a set of newspaper articles available at the concordances of Sprakbanken, specifically from the Parole cor­pus (Ridings, 2011), and some popular science ar­ticles from the same place. The text is processed by assigning each of the words the corresponding semantic vector from the space. Sentence vectors are constructed as pro­posed by Chatterjee and Mohan (2007) and Hig-gins and Burstein (2007), i.e. the words in a sen­tence are summarized after the subtraction of the mean space vector, and divided by the number of words in the sentence, as in Equation 2.</p><p>To investigate the effect on randomness we cre­ated 10 summaries with different seeds, the same way as in Figure 3. Figure 4 shows 10 trials on different seeds as before on the same text, but us­ing the larger outside RI-space described above. Comparing igures 3 and 4 reveals a more straight line when using a large RI-space. Thus, by us­ing an outside RI-space, the effect of randomness is reduced and a more predictable result between seeds is achieved.</p><p>Figure 4: Ten different seeds with pretrained space, each line representing a sentence in a text. The values are the inal ranks of each sentence af­ter the PageRank algorithm.</p><p>Another problem that has emerged is that the weighted PageRank sometimes fail to con­verge. This might happen in the original PageR­ank when the number of outlinks from a vertex is zero (Elden, 2007). The corresponding phe­nomenon in the weighted PageRank-algorithm is when the sum of the out weights from a given sentence is zero or close to zero. One reason is that since sentence similarity can be both posi­tive and negative it is possible that they even out. Also, nearly orthogonal sentence vectors makes for weights around zero. This only happens when<page local="6" global="1067"/></p><p>Iterations</p><doubt alpha="0.0" length="10" tooSmall="False" monospace="0.0">2 4 6 8 10</doubt><p>not using an outside space. Since a small docu­ment does not contain the distribution of words across a large number of context, it is likely that several sentences contain words that occur only in that sentence. When the context vectors for the sentences thereby are created, their vectors may be too sparse and the angle between them becomes nearly orthogonal, and the weights sum up to zero.</p><p>Figure 5 is produced similar to Figure 2 , where each line represents a sentence. The ranks in the graph are plotted on the y-axis and each itera­tion on the x-axis. It is clear that the values fail to converge and stabilize which might be a prob­lem when extracting sentences based on the ranks. This does not happen when using a large RI-space, since the sentence vectors are built from context vectors using a large number of contexts.</p><p>The problem can be alleviated simply by redo­ing the random indexing-phase using a different random seed, which is what is done in the cur­rent implementation when not using any outside source.</p><p>Figure 5: No convergence during the weighted PageRank-algorithm. Each line represents a sen­tence in a text with the rank plotted on the y-axis and the corresponding iteration during the PageR­ank on the x-axis. The irst ten iterations are plot­ted.</p></section><section number="5" title="Evaluation"><p>By equipping the summarizer with a better se­mantic understanding an evaluation was also per­formed investigating the information quality ofthe summarizer.</p><p>By using a pre-trained RI-space it was hypoth­esized not only that the random factor could be eliminated, but also that the quality of the sum­maries would improve. A comparison was made between using an outside text source and using only the document to be summarized to build the RI-space. Since several random seeds provided different summaries on the same text, the average performance measure of 10 seeds for each text was calculated when not using an outside random in­dex space.</p><p>The pre-trained RI-space used a dimensionality of 1800, a window size, w = 2 with a weighting of [0.5,1, 0,1, 0.5], and 8 non-zeroes in the index vectors, similar to Karlgren and Sahlgren (2001).</p><p>Using no pre-trained space, the dimensionality was set to 100, window size, w = 2 with the same weighting as above, and 4 non-zeroes, to account for a much smaller space, as in Chatterjee and Mo­han (2007).</p><p>For the evaluation 13 Swedish newspaper arti­cles with a length ranging from 100 to 800 words, see Table 1, were summarized to 30% and com­pared to human created gold standard summaries of the same length, available at KTHeXtractCor-pus (Hassel, 2011b).</p><p>Several automatic evaluation packages are available, most notably ROUGE (Lin, 2004). We used, however, also the more recent package Auto-SummENG (Giannakopoulos et al., 2008) since it is reported as having a higher correlation with hu­man evaluations than ROUGE (Giannakopoulos, 2009). For AutoSummENG, the comparison was performed by means of graph-value similarity tak­ing content similarity between different texts on character level into consideration. The texts are represented as graphs where each vertex depicts a character n-gram. The graphs from the model and system summaries are then compared resulting in a similarity measure denoting the performance of the system.</p><p>It should be noted that no preprocessing in terms of stop word removal and stemming were performed during the ROUGE evaluation since the package is tuned for English and no Swedish lexi­con for that purpose were available at the time.</p><p>Table 2 shows the values acquired using Auto-SummENG for each text and we see that for most texts the summaries produced using the larger RI-space are better than the ones without RI-space.</p><page local="7" global="1068"/><p>Using no space, the mean value from all texts of the comparison was 0.420. By using an outside space, the mean value was 0.547 which is a signif­icant improvement (p &lt; .05).</p><p>As a comparison, evaluations using the more known ROUGE package was performed. When using ROUGE a similar result is obtained, see ta­ble 3. Comparing the results of AutoSummENG and ROUGE yields a correlation of « .96.</p></section><section number="6" title="Conclusion"><p>By using a large word space model the perfor­mance of the extraction based summarizer COG­S UM could be improved. Along with the per­formance, robustness was improved, as the ran­dom factor between seeds was reduced, making for more predictable results from the summarizer. The performance was evaluated using AutoSum-mENG, a tool to compare generated texts with gold standard texts created by humans. The eval­uation was performed without input from humans, although humans created the gold standard and thus affected the results indirectly, no measures re­garding readability were taken. Thus, the measure does not capture readability, only that the extracted sentences can be seen as the most important for the document and that they correspond to human cre­ated gold standards.</p><p>Evaluations were also performed using ROUGE to have a point of reference since AutoSummENG is a lesser known method of evaluation and the re­sults from these two different packages correlated strongly.</p><p>Table 2: Evaluation of each summary. Each sum­mary has been compared to a gold standard cre­ated by humans. The left column shows the values acquired for the summaries using no outside ran­dom indexing-space and the right column shows the values after using an outside space. The values are acquired by means of graph value similarity using AutoSummENG.</p><p>Table 3: Evaluation of each summary using ROUGE-1 n-gram. Each summary has been com­pared to a gold standard created by humans. The left column shows the ROUGE scores acquired for the summaries using no outside random indexing-space and the right column shows the scores after using an outside space.</p><table caption="Table 1: Text characteristics, the number of words and sentences on each text." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p></p></td><td class="cell"><p>Words</p></td><td class="cell"><p>Sentences</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text1</p></td><td class="cell"><p>110</p></td><td class="cell"><p>7</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text2</p></td><td class="cell"><p>688</p></td><td class="cell"><p>40</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text3</p></td><td class="cell"><p>701</p></td><td class="cell"><p>37</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text4</p></td><td class="cell"><p>400</p></td><td class="cell"><p>27</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text5</p></td><td class="cell"><p>227</p></td><td class="cell"><p>13</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text6</p></td><td class="cell"><p>153</p></td><td class="cell"><p>9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text7</p></td><td class="cell"><p>441</p></td><td class="cell"><p>24</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text8</p></td><td class="cell"><p>179</p></td><td class="cell"><p>10</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text9</p></td><td class="cell"><p>483</p></td><td class="cell"><p>33</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text10</p></td><td class="cell"><p>838</p></td><td class="cell"><p>67</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text11</p></td><td class="cell"><p>388</p></td><td class="cell"><p>24</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text12</p></td><td class="cell"><p>169</p></td><td class="cell"><p>9</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text13</p></td><td class="cell"><p>471</p></td><td class="cell"><p>32</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>AutoSummEnG</p></td><td class="cell"><p>Without space</p></td><td class="cell"><p>With space</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text1</p></td><td class="cell"><p>0.301</p></td><td class="cell"><p>0.751</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text2</p></td><td class="cell"><p>0.484</p></td><td class="cell"><p>0.484</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text3</p></td><td class="cell"><p>0.497</p></td><td class="cell"><p>0.509</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text4</p></td><td class="cell"><p>0.569</p></td><td class="cell"><p>0.447</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text5</p></td><td class="cell"><p>0.276</p></td><td class="cell"><p>0.556</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text6</p></td><td class="cell"><p>0.321</p></td><td class="cell"><p>0.520</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text7</p></td><td class="cell"><p>0.510</p></td><td class="cell"><p>0.502</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text8</p></td><td class="cell"><p>0.239</p></td><td class="cell"><p>1.000</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text9</p></td><td class="cell"><p>0.340</p></td><td class="cell"><p>0.465</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text10</p></td><td class="cell"><p>0.347</p></td><td class="cell"><p>0.419</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text11</p></td><td class="cell"><p>0.487</p></td><td class="cell"><p>0.556</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text12</p></td><td class="cell"><p>0.574</p></td><td class="cell"><p>0.384</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text13</p></td><td class="cell"><p>0.520</p></td><td class="cell"><p>0.517</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mean</p></td><td class="cell"><p><b>0.420</b></p></td><td class="cell"><p><b>0.547</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>ROUGE-1</p></td><td class="cell"><p>Without space</p></td><td class="cell"><p>With space</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text1</p></td><td class="cell"><p>0.386</p></td><td class="cell"><p>0.695</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text2</p></td><td class="cell"><p>0.538</p></td><td class="cell"><p>0.551</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text3</p></td><td class="cell"><p>0.570</p></td><td class="cell"><p>0.540</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text4</p></td><td class="cell"><p>0.647</p></td><td class="cell"><p>0.522</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text5</p></td><td class="cell"><p>0.290</p></td><td class="cell"><p>0.590</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text6</p></td><td class="cell"><p>0.368</p></td><td class="cell"><p>0.599</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text7</p></td><td class="cell"><p>0.600</p></td><td class="cell"><p>0.573</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text8</p></td><td class="cell"><p>0.359</p></td><td class="cell"><p>0.975</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text9</p></td><td class="cell"><p>0.452</p></td><td class="cell"><p>0.541</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text10</p></td><td class="cell"><p>0.454</p></td><td class="cell"><p>0.560</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text11</p></td><td class="cell"><p>0.574</p></td><td class="cell"><p>0.665</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text12</p></td><td class="cell"><p>0.682</p></td><td class="cell"><p>0.369</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Text13</p></td><td class="cell"><p>0.599</p></td><td class="cell"><p>0.625</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Mean</p></td><td class="cell"><p><b>0.502</b></p></td><td class="cell"><p><b>0.600</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="8" global="1069"/><p>Further improvements can be seen with regards to stabilizing weights in the weighted PageRank-algorithm. By using a large word space the sen­tence vectors become more dense since they are built from context vectors from a large number of contexts. The sentence vectors are thus not as likely to be nearly orthogonal which becomes a problem when summarizing the weights as out­links, since the sum then might be close to zero.</p><p>An increased quality in semantic representa­tion however comes with some tradeoffs. A large word space reduces the portability somewhat, and increases the computational effort since a large space uses a much larger dimensionality. Also, the word space makes it language dependent, a previ­ously strong argument for this method. Creating a larger RI-space for a new language is, however, not such a difficult task if a large enough corpus is available.</p><p>The word space that was used was produced from rather general texts and it would be interest­ing for the future to investigate the effect of dif­ferent RI-spaces on different genres and domains, both in terms of training material but also on the quality of the summaries. Since Random Indexing is incremental, it is easy to add documents to the semantic space.</p><p>Although previous work (Smith and Jonsson, 2011) have looked at readability and concluded that the readability may be increased using extrac­tion based summarization, it is still unclear ex­actly how cohesive they are. Mihalcea and Tarau (2004) draws an analogy between the PageRank "random surfer model" and "text surfing" which relates to the concept of text cohesion. The links in the graph might be attributed to links between connected concepts or topics in a semantic way so it would not be surprising to find that the sum­maries have acceptable cohesion. Future research will have to conclude the cohesiveness of the sum­maries and how they may need to be improved.</p><p>We have, however, shown that the quality and robustness can be improved by using an outside previously trained random indexing space in the process of vector space model extraction based au­tomatic summarization.</p></section><section title="Acknowledgments"><p>This research was partly supported by a research grant from The Swedish Post and Telecom Agency (PTS).</p></section><references><p>Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine. <i>Computer Networks and ISDN Systems, </i>30(1-7):107-117.</p><p>Nilhadri Chatterjee and Shiwali Mohan. 2007. Extraction-based single-document summarization using random indexing. In <i>Proceedings of the 19th IEEE international Conference on Tools with Artifi­cial intelligence - (ICTAI2007), </i>pages 448—455.</p><p>Hercules Dalianis. 2000. Swesum - a text summarizer for swedish. Technical Report TRITA-NA-P0015, IPLab-174, NADA, KTH, Sweden.</p><p>Eva Ejerhed, Gunnel Kallgren, and Benny Brodda. 2006. Stockholm umea corpus version 2.0.</p><p>Lars Elden. 2007. <i>Matrix Methods in Data Mining and Pattern Recognition. </i>Society for Industrial &amp; Applied Mathematics (SIAM).</p><p>Therese Firmin and Michael J Chrzanowski, 1999. <i>An</i> <i>Evaluation of Automatic Text Summarization Sys­tems, </i>volume 6073, pages 325-336. SPIE.</p><p>George Giannakopoulos, Vangelis Karkaletsis, George Vouros, and Panagiotis Stamatopoulos. 2008. Sum­marization system evaluation revisited: N-gram graphs. <i>ACM Transactions on Speech Language Processing, </i>5(3):1-39.</p><p>George Giannakopoulos. 2009. <i>Automatic Summa­rization from Multiple Documents. </i>Ph.D. thesis, University of the Aegean.</p><p>Genevieve Gorrell. 2006. <i>Generalized Hebbian Algo­rithm for Dimensionality Reduction in Natural Lan­guage Processing. </i>Ph.D. thesis, Linkoping Univer­sity.</p><p>Par Gustavsson and Arne Jonsson. 2010. Text sum­marization using random indexing and pagerank. In <i>Proceedings ofthe third Swedish Language Technol­ogy Conference (SLTC-2010), Linkoping, Sweden.</i><i></i></p><p>Martin Hassel. 2007. <i>Resource Lean and Portable Automatic Text Summarization. </i>Ph.D. thesis, ISRN-KTH/CSC/A-07/09-SE, KTH, Sweden.</p><p>Martin Hassel. 2011a. Java random indexing toolkit, January 2011. http://www.csc.kth.se/ ~xmartin/java/.</p><p>Martin Hassel. 2011b. Kth extract corpus (kthxc), January 2011. http://www.nada.kth.se/ -xmartin/.</p><p>Derrick Higgins and Jill Burstein. 2007. Sentence sim­ilarity measures for essay coherence. In <i>Proceed­ings of the 7th International Workshop on Computa­tional Semantics (IWCS), Tilburg, The Netherlands.</i></p><page local="9" global="1070"/><p>Arne Jonsson, Mimi Axelsson, Erica Bergenholm, Bertil Carlsson, Gro Dahlbom, Par Gustavsson, Jonas Rybing, and Christian Smith. 2008. Skim reading of audio information. In <i>Proceedings ofthe The second Swedish Language Technology Confer­ence (SLTC-08), Stockholm, Sweden.</i></p><p>Pentii Kanerva.   1988.   <i>Sparse distributed memory.</i></p><p>Cambridge MA: The MIT Press.</p><p>Jussi Karlgren and Magnus Sahlgren. 2001. From words to understanding. In Y. Uesaka, P.Kanerva, and H. Asoh, editors, <i>Foundations of Real-World Intelligence, </i>chapter 26, pages 294-308. Stanford: CSLI Publications.</p><p>Chin-yew Lin. 2004. Rouge: a package for automatic evaluation of summaries. pages 25-26.</p><p>Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing order into texts. In <i>Conference on Em­pirical Methods in Natural Language Processing, </i>Barcelona, Spain.</p><p>Rada Mihalcea. 2004. Graph-based ranking al­gorithms for sentence extraction, applied to text summarization. In <i>Proceedings of the ACL 2004 on Interactive poster and demonstration sessions, </i>ACLdemo '04, Morristown, NJ, USA. Association for Computational Linguistics.</p><p>Daniel Ridings. 2011. Parole corpus at sprakbanken.</p><p>http://spraakbanken.gu.se/parole/.</p><p>Magnus Sahlgren. 2005. An Introduction to Random Indexing. <i>Methods and Applications ofSemantic Indexing Workshop at the 7th International Confer­ence on Terminology and Knowledge Engineering, TKE 2005.</i></p><p>Magnus Sahlgren. 2006. <i>The Word-Space Model: Us­ing distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. </i>Ph.D. thesis, Stockholm University, Department of Linguistics.</p><p>Christian Smith and Arne Jonsson. 2011. Auto­matic summarization as means of simplifying texts, an evaluation for swedish. In <i>Proceedings ofthe 18th Nordic Conference of Computational Linguis­tics (NoDaLiDa-2010), Riga, Latvia.</i></p></references></body></article>