Centering involved has been in proposed mutual as a model of the of local attentional (Brennan, states of Friedman, speakers and hearers Pollard 1987; Grosz the Sidner construction 1986, 1998; Walker conversation 1998). Centering mechanisms and to model and coherence of are designed in the of inferential discourse by characterizing intended intuitions load and hence their naturalness. transitions (Chafe 1979) These between utterances terms their are about the “flow” or char- acterizations process of meaning” to capture (Halliday 1994) in the “ongoing nisms In which work, we examine is a corpus Because of Japanese discourse. this e-mail to investigate the of by achieved. this corpus contains a high number mecha- coherence discourse elements that interplay are inferable from the discourse context, we have an op- portunity to examine of inferable the between standard native presence discourse entities. We centering on transition and claim the basis of intuitions definitions the speakers the of coherence in the is much of that inferable often implies, actual level difficult primarily specify. virtue of corpus higher account by the fact that transitions involving than the centering entities are major model to We conclude in in that the account the coherence this of identity corpus. of Detailed standard centering cannot one accurately problem analysis re- veals in that in order lies the requirement discourse elements adjacent problem utterances propose for those elements to contribute to coherence. more two additions to the usual repertoire of We de- scribe this and enable of coherence in this corpus, while transitions that within a authentic account remaining mechanisms The a centering is organized framework. article role in as modeling follows. In Section We 2, we on briefly and coherence. go outline describe centering of in Section their 3 illustrate to corpus how standard centering mechanisms the features the and coherence in corpus, suggesting that these mechanisms charac- terize not transitions and adequate the task. In Section this 4, we describe more general problems with are for the ∗ Infant Columbia, Studies V6T Centre, 1Z4 Canada. University E-mail: of British Columbia, Room 1401, 2136 West Mall, Vancouver, British jwlab@psych.ubc.ca. © c 2004 Association for Computational Linguistics Computational Linguistics Volume 30, Number 2 inclusion of inferable of discourse entities in more centering theory and propose a revision standard accurately describes the corpus. In to the tion well, set we transitions that Section the we implications outline of this proposal for other improvement areas of this sec- as In explore 5, discourse analysis. proposal, we in some possibilities final for and extension of the and conclude the section.

The central intuitions or of centering in concern the relationships in among the (Walker, discourse Joshi, en- tities Prince appearing 1998). Each represented adjacent in utterances is a discourse and utterance U i a discourse or Cfs. considered These to contain a set of discourse Cf entities called forward-looking centers, entities are ranked in the follow, in list for each utterance ordering according to language-specific Japanese in ranking Walker, principles. (1994) for given Iida, We (with general, the principles noted in (1)): and Cote some adjustments for possessive phrases as example Cf ranking for Japanese: (Grammatical OR ∅ ) topic > empathy > subject > object2 > object > others A special of member is of the Cf list, the backward-looking Cf on Cf center, the 1 U and highest-ranked the list of or Cb, U which represents in “topic” . In i the of , or Cp, is i − 1 i preferred center U the highest-ranked Cf is in realized U may Given addition, the i and U i − of 1 , i then, is there are four different ways in which their Cbs and Cps U i . U There be related; each these in defined approach: as a type of transition state (Table 1). are two rules a centering is Rule Cb 1: If some . entity in the Cf list for U i − 1 is realized as a pronoun in U i , then so Rule the 2: Transition for U i in order states are ordered such , that CONTINUE (Walker, is most by SMOOTH and SHIFT Joshi, preferred, lowed SHIFT ROUGH and Prince fol- 1998). RETAIN, Rule 2 captures of the centering intuitions in concerning prominent coherence: impose Utterances that infer- CON- TINUE the topic a previous prominent thus or more utterance a position which coherent, than utterances which a lower ential load, and are relegate the topic to less The majority positions of change interpret sentences in the vast our topic. of the corpus in are complex sentences. Thus, the ignored. question We will how to centering principles of complex sentences cannot be consider the basic utterance unit centering to be the tensed clause Table Transition 1 definitions. [BAR] Cb(U i ) = Cb(U i − 1 )ORCb(U)=? i Cb(U i ) = Cb(U i − 1 ) Cb(U) i = Cb(U ) Cp(U) Cp(U i i = i ) RETAIN CONTINUE SMOOTH ROUGH SHIFT SHIFT Fais Transitions and Coherence and will assume that these output clauses of form first a flat, linear in sequence of is discourse on, the clause the sentence the input units, such that the centering in of Kameyama (1998) Suri McCoy (1994). to the next, and so the spirit and and 2 Because we the notion of backward-looking notion in center will be critical According to the discussion (1986, greater Walker, Joshi, detail here. Prince [1998]), to Grosz, is Joshi, that follows, Weinstein look at this in and quoted if it of and a center realized utterance element the described by U or in an interpretation U of “is an of (page situation 4). As Walker, Joshi, the some U” and Prince semantic out, subpart “pronouns, zero pronouns, (1998) point this covers inferable from explicitly (page realized 4). The discourse entities, and ... entities Grosz, Joshi, Weinstein the discourse inferable situation” is, definition pro- posed not by and of allows or entities, immediately that entities that at surface level the utterance recoverable from are expressed the the subcategorization of properties of However, the verb (as, for example, zero not pronouns within utterance. the theory does make are) to constitute centers which an We the class of difficulty permissible inferable explicit the parameters on to characterize will later. elements or the constraints doing so. return to this

3.1 The The Nature of the Corpus five data examined of consist Japanese of a collection from of June 32 e-mail 5 to June messages 16, 1995. exchanged The messages among employees a company constitute interesting a collective five attempt in to group. schedule Thus, a sports-watching to the tone is outing convenient and usually casual. The for all in messages: of written the nonstandard text with authors combine information standard aspects various strategies of for encoding speech- like writing the words), English; uses nonstandard punctuation, katakana (the syllabary for markers, foreign and spelling; fillers tense, and formality typical of emoticons; discourse is (Fais in Table 2001; Fais sentence-final 2. Yamura-Takei particles, 2003). Quantitative information speech; and and for the corpus given Table Number 2 author. of messages, paragraphs, sentences, clauses, and characters per Authors Messages Paragraphs Sentences Clauses Characters H I M R U Total [BAR] hierarchical reported interpretation is warranted: Computational Linguistics Volume 30, Number 2 3.2 Inferables Example (1) and Transition illustrates States in the Corpus messages how centering mechanisms apply to the beginning model of one of the and how the CONTINUE and RETAIN transitions can coherence: motomoto noriki was datta actually enthusiasm 1c. Clause (1a) has no Cb, since , it occurs of at the beginning it is of the message. The in virtue fact The Cp of (1a) the that topic-marked. subject of is Swallowsvs.Baystarsgame (1b) is omitted; by copular 1 implies that the referent for this zero the is desu , which rule is A process argument Swallows vs. Baystars game in (1c). Note a correct assignment. similar resolves the referent for kore intuition ‘this’ that the of CONTINUE message is and RETAIN transitions the this segment the coherent is do, in fact, Swallows capture Baystars that and “about” the well Example (2) vs. is (1). much more game. Notice typical of Cbs the in messages in the corpus inferable and “from is not as behaved as of that . The all the this 330 in situation” U i − 1 preponderance of example inferable are Cbs is typical; out the discourse Cbs 250 (more 75%) pronouns, zero of the corpus, than are entities other than [BAR] node since Swallows Cb = Cf (beginning) = Swallows vs. Baystars game, 4 18th, enthusiasm CONTINUE Cb = Cf SvsB SvsB game ∅ = game ∅ RETAIN Cb = it = Baystars Swallows vs. Cf U-san, M-san, game = interest, it, taekwondo Fais Transitions and Coherence tokorode way enseki nan by the restaurant desuga is 2b. sendagaya kinpen neighborhood ni in sake wo nomeru sendagaya alcohol OBJ can drink umai sobaya ga aruto is miminishimashita good soba way, shop SUBJ have the which the restaurant, I’ve heard “By about around heard there is Sendagaya.” a good soba shop, also serves alcohol, 2c. heiten jikoku ga hayaisounanode shop closing time SUBJ early seems because 2d. ∅ chotto ∅ a little The (2b) Cb of (2a) is null, since in that clause Cf is All the first centers, the list. three of in the discourse segment. has three of listed (2a). We mentioned these entities is no are inferable from the discourse context to the list of inferables of previously it is that there more difficult, principled way determine an utterance; even then, to mazui kamoshiremasen might ga bad be but Cb=? Cf = restaurant Cb=? Cf = Sendagaya soba shop, neighborhood, alcohol CONTINUE? SMOOTH SHIFT? Cb = (inferable) shop closing time Cf = shop closing time SMOOTH SHIFT Cb = choosing (inferable) this restaurant Cf ∅ = choosing this restaurant ∅ ROUGH SHIFT Cb = (about information (inferable) restaurant) Cf = (about I-san, information restaurant) Clause Computational Linguistics Volume 30, Number 2 Table 3 Transitions occurring in the corpus, by type. Transition type Number Percentage CONTINUE R S ETAIN MOOTH R SHIFT OUGH SHIFT Transition N to inferable Cb ULL Total determine the order which of in which Cfs in inferables (2b) is should be listed on a Cf list. Therefore, cannot we the which the Cb, “highest we ranked” and thus the Cb we Since say Cb of is for (2b). cannot (2b) is determine Cf, fact is inferable that there is as its Cp, the same Cf and so we likewise cannot determine whether the The the only one for (2c) cannot label simplifies the problem the transition at all. which Cb from (2b), must be the Cb of (2c), somewhat. but since we That of (2b), . we do not also know if the transition from (2b) to (2c) is could not ascertain or the a SHIFT Again, the presence of only one Cf in (2d) makes matters a CONTINUE we SMOOTH from (2c) (2d) . easier; are We able to label the transition may not inferable to that from a SMOOTH SHIFT surmise I-san be the discourse context, 6 restaurant) is, thus the latter becomes the Cb of (2e). Because though information this from Cb is neither (about the (2e) is the same as the Cb and (2d) . of (2d) nor the same as its own Cp, the transition Using to a ROUGH SHIFT the standard centering definitions, supplemented number label transitions to utterances that contain no with Cb, we the notion of NULL transitions to of of occurring in (Table hand-tabulated the transitions each type , the corpus 3). The figures for CONTINUE, RETAIN, SMOOTH SHIFT pronominal, and ROUGH or SHIFT zero-argument are those Cbs. for transitions Given to utterances difficulties in containing either explicit, inferable Cbs, the grouped accurately labeling transitions pronominal, or latter zero-argument together separately from to utterances containing transitions to utterances with we these Cbs. explicit, There number are of two points of special interest in Table 3. inferable First, of course, Cbs is the particularly high transitions At over 40%, to utterances containing involved in and the number of NULL transitions. inferable Cbs make the utterances of transitions up substantial portion the corpus. Second to utterances containing relative proportions of a RETAIN, SMOOTH SHIFT , are the Note CONTINUE, and ROUGH SHIFT transitions. that, considering proportions just these roughly transitions what to we utterances of containing explicitly Most realized centers, these , are expect number of coherent text: of the shifts are CONTINUE followed by a respectable RETAIN s, and a very few SHIFT s. Fais Transitions and Coherence

4.1 We Inferable in Centers Section 3.2 in Centering Theory saw that the indeterminate introduction of inferable identification. centers We into a centering anal- ysis leads to with inferable transition make also proportion saw that (over transitions is up a large 40%) of to utterances in It centers of important, then, if a centering analysis is the tran- sitions nature this corpus. in accurately, we make to notion corpus that principled provisions represent the coherence this inferable in theory. for the Only captured rarely inferable by centers the when are entities is actually listed in analyses in the or supported by previous explicit mention literature, and then clear, usually their presence involved, and in by semantic relationship between the (3), a tight from (17) in Kameyama syntactic (1998): entities as taken 3b. but the heart stopped beating Cb = Possessor Republican of heart) Party (inferable the 3c. and the lifeblood congealed Cb = Possessor Republican of Party (inferable the lifeblood ) Note is only that one (3) avoids at least with, one of the difficulties is in (2c) we encountered of deal as there (2d) in (3b) (2); when to and and and (3c), there Cb center is However, in (2a) (2e), or in (3b) if it the choice trivial. cases such as and something Cb but the heart and lifeblood stopped pumping , we had been like Further, in it is impossible identify cannot determine appropriate what the should be. these cases, (2b)), to the when the Cb of to U i − 1 is the transition following utterance; (in (2c)) is undetermined (as it is for the transition to U i this case, undeterminable. 4.2 There Logical Difficulties are In problems with Inferable further our resulting far, from Centers we taking the use of inferable with centers to its logical in extreme. analysis in so have inferable been from concerned explicit entities U i that realize inferable centers U i − need 1 that are in Cf the discourse context. accurate, we to be listed the list for U i − 1 . However, To be process those incrementally, centers from the conclusion that since we do not if which discourse inferable this leads entity U i 1 will to inferable know in Cf of − . This is be evoked in U i , we need to list every in of entity the of list U i − 1 both computationally inferable lack any parameters for determining what untenable and, view Cb, the impossible. Even if it were constitutes an grammatical-role desirable, how could we allowable inferable of possible information and define for entities of the type on Cf essential to determining the placement Again these entities of a inferable list? problem. Every taking the definition not , centers inferable to its extreme, just 1 evokes There we note is nothing another utterance, U i − in (4) (a centers. of (2) in the illustrative theory to preclude a situation (implicit such inferable as that shown in version italics): simplified for purposes) centers are given Computational Linguistics Volume 30, Number 2 4a. sendagaya kinpen neighborhood ni sendagaya in umai sobaya ga aruto is miminishimashita good soba shop “I’ve heard is SUBJ have heard there a good soba shop around Sendagaya.” 4b. demo heiten jikoku ga hayaisounandesu but shop closing time early.” SUBJ is early “But the shop closes In Cf (4), an of inferable (4a) is center from the Cf Cb. list of Of (4b) matches an inferable as the course, there could be numerous center list and obviously centers on chosen Cf lists of of i and U i − 1 , all of “vying” inferable for Cb of identical is the U inclusion in U i . Although this theory, is nothing, an absurd unfortunately, extension in the itself it centers out. centering there the theory to rule 4.3 Possible Solutions to the Problems of Inferable ing Inferable Centers as Bridging References. Inferable Centers 4.3.1 (Clark is 1977); centers are similar have a conceptual relationship to entities in to bridg- references There they utterance. in sense that bridging references should participate in a previous a (Hahn, Markert, Strube 1996). But work the on creation of coherence a discourse and references this relationship or the bridging in characterizes which as referential is anaphoric; this can be seen indirect the various terms under this phenomenon discussed: however, anaphora, identity case of and partial anaphora. Bridging bridging references, anaphora, functional anaphora, may be mediated not only references, unlike the usual made-of, but also by any number of other semantic relationships (is-a, by a strict condition, has-a, Unfortunately, at-time, etc.). its the establishment is notoriously of the difficult. semantic Poesio relationship between (2000), an anchor and bridging reference on types of et al. even severely relationships Every to be labeled, had extremely poor inter- after restricting the labeler reliability a first pass. account in which bridging well-defined references are Poesio to a small, relatively set (Vieira ad- dressed restricts and Hahn 1999). 2001; allowable Poesio relationships Cote (1998) proposes et al. 2000; Murata, of Isahara, and Nagao 1999; Strube instead and in Cf the use lexical-conceptual primitives approach might templates and suggests that the conceptual information of grammatical relations that necessary this provides of be rich enough the references. She to supply number of other resolution of bridging points out part-whole as well, however, information to a semantic relationships manifested in that would identifiable types from information. Thus, bridging references on not be lexical-conceptual mantic involved, has attempted what to provide a characterization of although work bridging references is the possible se- of relationships success has been achieved limited to a small subset cases. Cb=? Cf = soba shop, Sendagaya neighborhood, udon shop, sake shop, shop clerk, menu, food, customers ... Cb = Cf shop clerk = shop closing time, shop opening time, shop, shop clerk, door ... inferable from the Fais Transitions and Coherence 4.3.2 Restrictions problems on the Notion introduced of Inferable inferable Center. is One into possible way only around the logical by centers This are would in centers to take that realized an utterance when account inferable explicitly mean, determining the Cb of that utterance. from (2b) in for example, considering Cb (2c). But soba shop, Sendagaya we make neighborhood , and alcohol choosing Any of the for how do inferable the choice among these possibilities? be Cb. But these is three different, explicit, exactly the difficulty: We have no centers could chosen make to be the We this no way of which of principled inferable way to is such a choice. in Cf have ranked the list for (2a) knowing (2b). so that we these can select that Cf centers highest to be the Cb of A This second possibility is approach to allow only explicitly Cb imply the by Passonneau realized (i.e., be taken (1998); noninferable) her definition of centers. seems to null seems Cbs to a Cb must in (noninferable a null that which be is no and) explicitly Pear the cases there explicit Cb. In realized, and her of constitute Stories Chafe (recordings 1980), of her examination the people describing (transitions to another person a movie majority transitions to an utterance with a null they had seen; represent of NULL Although her is Cb) Passonneau the note transitions. of the patterning of concern discourse transition types does not segmentation, does that accurately reflect the coherence Allowing inferable only the stories. Cbs explicit centers null Cbs would mean, proportion for the corpus of studied in this article, that 75.7%. Under become and the would NULL transitions becomes this assumption, this native-speaker corpus intuitions be characterized (an as extremely of in- coherent, a claim belied was both by native acceptable level co- herency for these texts was confirmed by three of messages speakers) was and also by the fact that the task group that the central number concern of opinions these of successfully information com- pleted; the exchanged a outing, with and came regarding their sports no pieces message and to a consensus sion information contained in previous messages. Thus, showing the solution of confu- only about centers not accurate characterization of allowing explicit does yield an the coherence of this English Hurewitz corpus. (1998) chose to define allowable or inferable Cbs fairly narrowly Cb dependency a poset relationship to hold in in order her data, we a to requiring functional for be recognized. Even with in our this definition, which is more Brown to be previous discussion, she finds constrained that 21% than have taken “inferable” (taped (a of written of texts) what and 28% of of the corpus variety no-Cb the switchboard (2000) she calls a condition. Poesio corpus telephone conversations) consist proportion of nonexplicit Cbs in English et al. They report a similarly number high minimize a of configurations of parameters of their text corpora. test would number in of null Cbs (reasoning centering theory attempt of that the best configuration of to to the parameters result the fewest violations in the constraints of the case, one constraint that all utterances the discourse except for the first theory, in this Cb). the One way in which improve have at least of they are of able nonidentity to their results significantly is by allowing is, a restricted set three of types well-defined inferables. relationships However, between centers, that by type of recognizing inferables three types still does not address the issue of the indeterminacy simply limiting the allowed transitions to number utterances of nonexplicit containing Cbs inferable Cbs. in naturally And of the occurring central question raised by the high How we found which in a in which Cbs texts can coherence text are so remains unaddressed: inferable The thus in characterize transition types are often indeterminate? often and inferable crux of the In problem nonproblematic lies in the application Cb in of standard is centering processes case, recognized by virtue of to centers. a a U i its Computational Linguistics Volume 30, Number 2 identity But to now a Cf in U i This process − 1 . is (relatively) inferable straightforward Cb this to centers. In in the case of explicit centers. in apply This i we recognized must by of make its a standard identity centering to an inferable approach, can virtue center inferable in a U implies also be U i − 1 . in in that 1 order somehow i recognize (possibly) one available of all the possible them as the Cb for U . We centers U noted − to is Even if we inferable i have already that this position untenable. only recognize in the 1 by considering those that U , it is centers of U i − a posteriori which of is in appear . What we i still select highest U need, impossible way to these Cb in centers not ranked of its identity i with instead, is a by virtue a preestablished list of a (explicit to recognize and inferable) U i of centers, . We but propose by virtue of a relationship, of other than identity fill , with the explicit The centers U i − 1 of the relationship lexical cohesion inferable to this function. without recognition a lexically uncontrollable cohesive relationship, inferable in Cf proliferation then, centers the virtually of admits allowing the only lists utterances. We propose hard-to-define centers not for problems a principled way to define this relationship that avoids of the discussed above but also more accurately characterizes the coherence this corpus. 4.4 Coherence Halliday (1994) and Cohesion characterizes not cohesion [grammatical] in text as the establishment of “relations of subject to limitations; relations from that may words involve within the text that are elements any of extent, both smaller may and across larger than of clauses, single intervenes” hold 309). gaps any extent ... without to lengthy nature passages of whatever text; and that to (page 7 Cohesion is regard the of meaning is into of that instead aspect “whereby of the flow out in channelled a traceable (page current 311). It discourse formlessly direction” is spilling every possible is meant model. this “traceable current of discourse” Lexical that centering to cohesion contributes to textual words coherence. in In other Coherence a text help to make words, strong semantic and structural relationships is among property of is that property piece of of text “make sense.” Centering models a discourse; cohesion a dis- course elements. of We coherence it is not only by characterizing relationships of identical between elements discourse. claim that the strong continuation explicit discourse elements that creates coherence, but also cohesion among discourse elements. 8 Halliday identifies four features of text that create cohesion ements: reference, ellipsis, and lexical cohesion. Insights among discourse el- conjunction, number interactions works with (Nariyama the processes 2000; of concerning con- junction types and in their of Nakaiwa referent resolution Shirai 1996; have been Kuno elaborated 1973) a but not been well integrated into the approach. and have of of mainstays of centering course, the centering research. Lexical Reference and ellipsis however, are, is an some of text that has had only cohe- sion, in approach, aspect it coherence a although has been incorporated into other a trivial application centering processing (see aspects of natu- ral Halliday, language through subsequent discussion). of items “Lexical cohesion,” in according “comes selection that related way to about the those before” (page 330). In are some to that have gone centering, that relationship has been Fais Transitions and Coherence assumed to be one in of identity (or in of its centers; an element in in U order i is required an U 1 discourse context) for it to be the same as Cb. element However, i − we difficulties of to of saw the admitting inferables be considered a approach to characterize coherence for inferable and the inability a earlier centering notion of involving inferables centers. Applying the lexical intuitively relatedness to cases allows us to capture what seems to constitute the relationship between clauses containing inferable centers. A number in of other These accounts provide relevant information concerning accounts upon the characterization of lexical cohe- sion text. are based (Harabagiu 1998, 1999), elements by reference (Harabagiu to semantic information semantic 1998; contained in WordNet relations among discourse Honda 1994), or thesauruses (Kasahara Morris 1996; Kozima and Hirst and al. 1993; 1991; Okumura 1993). In of dictionaries approaches, et or Kozima and Fu- rugori (or words all is these in most the of semantic distance similarity between among) of computed, and these accounts, the results are applied to the The segmentation number intuition discourse. of other behind in natural the importance of processing lexical relatedness has been well. applied Lotfipour- to a Saedi (1997) tasks language rigorous and notion analysis of as Boguraev uses lexical Neff cohesion (2000) to develop improve a “translation equiv- alence”; Sack (1999) and of to document newsgroup summarization techniques; mura Honda to create (1994) “diagrams social cohesion” word for postings; and Oku- Halliday and to disambiguate means asserts interaction senses. that “this between through lexical cohesion and the tracking participant the discourse” (page reference Centering provided with way of character- 332), ... is, principal modeling for a that izing for focus. of has the the addition of us in more notion a principled tracking reference; the of lexical way, cohesion to empirically comprehensive by making allows centering function possible next principled we outline inclusion an even the of what have been called theory. how lexical cohesion can be incorporated inferable into centers. In the section, a centering is With 4.5 C OHESIVE works earlier, of Transition the sorts of and COMPLETE SHIFT the use it is techniques to establish semantic notion distance of described in the In cited we possible to be precise about is the lexical section, how semantic distance established one cohesion. this discuss measure from Gainen using Base (‘Concept of these techniques, Database’) (Kasahara a semantic similarity 1996). We indicate derived the notion et of al. then how semantic new of as a element in distance can be to lexical cohesion crucial the creation of used define the types and COMPLETE SHIFT , which two transition: in COHESIVE proportion of allow is corpus containing a from high machine-readable nonexplicit us to adequately characterize The Gainen coherence Base a Cbs. Japanese. Each word in a knowledge base built from is defined word. The by number a list of weighted dictionaries of the knowledge base extracted in dictionary definition of the weight of keywords the word’s times a appears determines the for the keyword. Keywords keyword the definitions are standardized to take into weights account the normalized presence of semantically into similar words in the definitions, of and their in are The to take account the differing lengths is definitions the of dictionaries. nearness of semantic words distance a the the two in between two words calculated as function a vector space. 9 For Computational Linguistics Volume 30, Number 2 Table 4 Transition definitions. our present when Cf(U purposes, ) is then, we say that Cf( there is ) lexical cohesion between U i close to U 1 as determined using a well-defined i and U i − 1 semantic in measure semantically over Gainen Base. i − similarity of whether the 10 Section 4.6 similarity is We will examine in more detail Cfs of will or semantic individual best viewed as holding utterances discourse entities in between the sets now, we between of those utterances. For Cfs talk equally lexical in cohesion between utterances and lexical among defining cohesion for those utterances. We cohesion the that the ≈ indicating participate relation strong lexical with define one indicating identity. as cohesion, a lexical cohesion factor of Table We 4, supplement with the standard in table of transition The states, We defined the right side. sense of shown in the this table is left side of transitions centers be explicit, that is, pronouns, zero arguments, or as follows. assume all The to of Table 4 model explicitly realized well-behaved entities. left portion allows of us to transition contexts, the focus the discourse in states in way. It includes explicit in which tracking or Cb of U i − 1 might be “?,” that is, a specific, local the cases the might no Cb, while that U i − 1 might be the discourse-initial Cb. However, where utterance is not simply identity have (explicit) U i does have (an explicit) in (explicit) there element in , strict we between any 1 have the situation in any propose interpretation − which Cb( element U i and U i U i ) = new in of Table 4. ?. We Cb. Table a If 4 is new for these one Cf types of cases, described the right two shift to utterances that do not portion defines Cf(s) in contain an least U i that has a high lexical cohesion value with explicit in there i , at illustrated in then the transition from U i − 1 to U i isa COHESIVE transition. This some is U − 1 (2b), (2c), proposal, (2d), (2e) of example (2). The situation clauses and transitions , to these clauses, under of the present (2) (entities are reanalyzed as COHESIVE as shown in a reanalyzed one version example in claimed in to bear close semantic relationships to another are shown boldface here and subsequent examples): the following conditions: Fais Transitions and Coherence 2a. tokorode way enseki nan desuga by the restaurant is 2b. sendagaya kinpen neighborhood ni in sake wo nomeru sendagaya alcohol OBJ can drink umai sobaya ga aruto is miminishimashita good soba way, shop SUBJ the which the restaurant, I’ve have heard “By about around heard there is Sendagaya.” a good soba shop, also serves alcohol, 2c. heiten jikoku ga hayaisounanode shop closing time SUBJ early seems because 2d. ∅ ∅ mazui bad chotto a little kamoshiremasen might ga be but gozonjidewa information naideshouka have “Choosing early. I-san, this restaurant may not be good because it closes do you know about this restaurant?” of Each strong COHESIVE transition in the modified version one Cf in of (2) is justified by the a For relation at least U i and at least one presence semantic between utterance. is probably example, soba shop in (2b) is Cf in the previous in (2a), as alcohol, semantically a lesser extent, and Sendagaya (if our related to restaurant information interpretation is to is name of a restaurant). Note database that the that the transition from includes the (2e) in this to one, rather than ROUGH SHIFT . Certainly (2d) (2e), Cp this from a restaurant is no to I-san, COHESIVE but is a means since restaurant still present in the Cf the changes incoherence. The by as abrupt as the designation ROUGH SHIFT implies. list for the transition presence The of a null of Cb in and is of itself, then, instead is not necessarily proportions indicative of of level coherence in captured various states present corpus, including by the the Not transition of If containing no null a Cbs COHESIVE are no felt ances, and Cf to in be cohesive with transitions. all utterances is Cb U such that it previous has strong utter- course. with Cf there in , explicit is i a 1 then the shift considered COMPLETE . This is illustrated lexical cohesion in (5), which U i − example shows the continuation of example (1): Cb=? Cf = restaurant COHESIVE Cb=? Cf = soba shop , Sendagaya neighborhood, alcohol COHESIVE Cb=? Cf = shop closing time COHESIVE Cb=? Cf = choosing this restaurant ∅ COHESIVE Cb=? Cf = information I-san, ) (about restaurant Computational Linguistics Volume 30, Number 2 5a. demo tekondoh ni in kyoumishinshin interest datta but taekwondo keen was 5b. douyara tsuyuiri shitarashii toiu somehow rainy season has come sengenmodeta youde with kyou mo shikkari ame “With today also heavily rain the declaration today.” that the rainy season come, raining heavily There is very low cohesion , between of the (5a), Cfs namely, of (5b), that is, today, rain, declaration Thus, rainy we season and those from (5a) U-san, M-san, game, taekwondo , and beginning interest. of intuition transition to (5b) as a COMPLETE SHIFT , which and matches corroborated our designate in the that, fact, (5c) the topic of the Cp message of (5b) has changed. its Cb; in This by that RETAIN the other is further on the fact new in s (5b). as words, it goes Since to develop the two of Table topic 4 begun the sides are complementary, of it is perfectly possible herence Example characterized by various combinations the transitions in for co- to (6) be illustrates interaction of Table 4: the table. the between the two sides 6a. konouchi of jinguu Jingu no hou wa 17 17th nichi to 18 18th nichi these TOP and C ONTINUE , from above Cb = these 11 Cf = these, Jingu Stadium, Yokohama declaration ga futteimasu SUBJ is raining has it is Cb = it = Cf game = U-san, M-san, it, taekwondo, interest COMPLETE SHIFT Cb=? Cf = rain, today, declaration, beginning of rainy season RETAIN Cb = Cf 18th, rain = probability, rain , Fais Transitions and Coherence no yokohamasen Yokohama ga iikato is omotteimasu ga game versus SUBJ good suppose but kuwashiikoto further information ga fumei nanode SUBJ unknown since “Of these, on for 17th Jingu or Stadium 18th I would suppose the game As versus J-league, Yoko- hama I the the information be good. for since don’t me have further if information.” such as the match-ups, please let know you have any bokushingu boxing 6e. watashi I “Since would there are like to skip it this time.” of The transition from (6a) to (6b) is COHESIVE (6c) , since (6b) has no Cb and yet the the RETAIN match-up information from Cfs (6b), which two utterances is are semantically Cb with related. then identical Cb of (6d) and in s COHESIVE match-ups boxing its own (6d). an both to the and to Cp, Since (6e) does in fact have (6e) is explicit . This is of non-discourse-initial the transition to CONTINUE latter transition an example a utterance wa kongetsu month wa ii kahdo matches ga nai no node TOP this TOP good SUBJ since wa ∅ pasu TOP no skip good boxing this shitai would match-ups like in month, I game , 17th, 18th COHESIVE Cb=? Cf = J-league, match-up information RETAIN Cb = match-up information Cf = person, match-up information ∅ COHESIVE Cb=? Cf = month, boxing , this match-ups CONTINUE Cb = Cf boxing = boxing Computational Linguistics Volume 30, Number 2 without OR a Cb that in licenses a CONTINUE of transition Table 4, by virtue namely, of the second Cb( part ) of the disjunction the second column that the U i 1 = Cb(U ) = 4, ?. (Note that (5) also demonstrates the interaction − ? and Table i The with between transition following a COMPLETE SHIFT in .) the two sides of a RETAIN proposal to include COHESIVE and COMPLETE SHIFT identical centering in theory is moti- vated in by order concerns about having to specify two need inferable coherence. The entities adjacent for exhibit requirement However, it is of utterances those utterances to identity to play part. often leads the to allow entities to a inferable Cbs. impossible to characterize notion of transition states to utterances we containing of By including the need COHESIVE invoke inferable transitions, capture we the relatedness two entities without the to in centers, and can far better characterize the apparent coherence this corpus. 4.5.1 Transition States we in the Corpus, Revisited. Given of the inclusion and SHIFT the labeling in of COHESIVE COMPLETE in Table 3) s, reinterpreted transitions of the corpus (as represented (Table 5). The and hand-tabulated a revised inferable distribution or designate transitions to Cbs transition types decision to was made on of intuitive and NULL transitions as COHESIVE COMPLETE SHIFT the in basis an assessment However, of the possible method semantic relations between entities is adjacent in utterances. next a possible for automatic determination described the section. 4.6 A Preliminary Implementation 4.6.1 Preparation of the Corpus. , we In order to explore the computational feasibility Before the corpus to an analysis of of COHESIVE and COMPLETE Gainen SHIFT Base. subjected implementing we processed semantic distance using the it only of the analysis, corpus so contained the Base canonical forms first the words in the that run accepted by Gainen The step in each the the algorithms. this procedure utterance, forms through morphological analyzer, ALT-JAWS (Nippon Telegraph was to (usually Telephone the corpus and [Chinese Corporation a 1996), which This rendered is necessary all word forms in their ing most character]) form. a and standard preprocess- canonical kanji step for word computational indicate analyses morphological of Japanese text, which In contains writers no spaces between may forms to words hiragana or their (the structure. addition, e-mail katakana two syllabaries used for writing of use (Fais and words, respectively) 2003). These words even for need words primarily function foreign Yamura-Takei that have standard kanji well. forms The and order of this analysis were to be in which checked and corrected by native rendered in kanji as Japanese in results incorrect speakers of to eliminate cases the analyzer chose the kanji form for a homophonous hiragana representation. Table Reanalysis 5 of transitions occurring in the corpus, by type. Transition type Number Percentage R C ONTINUE S ETAIN ROUGH MOOTH SHIFT C SHIFT C OHESIVE Total OMPLETE SHIFT Fais Transitions and Coherence The next step notion was to delete all but noun in forms from the corpus. tiates the central factors the centering approach This step substan- the of that (we will in Section 5.3). are Further, the discourse entities of utterances pronominal return zero-argument to this were made the in antecedents all and references explicit the text. 12 was by inserting kanji forms for these antecedents, they appeared in This done text, into the utterance in which the pronominal or zero-argument as the the were native of Japanese. The form appeared. in by into a speaker same native An- tecedents determined Both of identifying speaker assisted dividing the into text clauses. these methods sentences (Huls, Bos, can in steps, and clauses, completed, theory, antecedents parsing Claassen be 1995; Nakaiwa by Shirai automatic, 1996; Paul compu- tational Sumita 2001; Yamura-Takei and 2002), and et but the of these approaches is and to on al. success rate them for completely analyses of this of not high enough Because rely Base in our intent is accurate to examine not the effectiveness implement of type the use of corpus the Gainen at this time. we not determining lexical cohesion, and methods to in a preprocessing fully automatic of process, did attempt to use entirely automatic the the text. 4.6.2 Determining Of 670 Semantic of Similarity nouns with in the Gainen Base: The Problem machine-readable present the e-mail corpus, only 235 of Cov- erage. the types with which Gainen Base was are found in the makes more dictionaries problematic is the number of missing constructed. words What this ratio , even that a , ‘lacrosse,’ these names of (e.g., supo-tsu kansen ‘sports-watching words in event,’ rakurosu and the sports teams) one The are (Hutchins problem high-frequency of in this corpus. 1995; coverage Sag 2002; automatic Fujita language-processing Bond 2002). At systems is of a common provided Gainen Base, et al. however, we and the level well coverage by the in cannot usefully whole. assess However, how we semantic similarity characterizes coherence in which this corpus In for those clauses every noun as a can be found in the Gainen can make this order assessment into to get an accurate of assessment we of Base. in this itself subset the corpus, included lexical cohesion between those cases in which adjacent clauses that fall well in it were all clause the before found the Gainen the nouns Out the of original as 443 as those an there 66 clause clauses that meet Base. We measured clauses, are these semantic between adjacent clauses in two criteria. first, we measured similarity group of nouns two in ways. In the group of nouns in the semantic . In distance we between measured the U i − 1 and the individual noun U in i the second, group of nouns in the semantic . The distances method − 1 13 i i information first method between each computationally, U and the U is far “lighter” of noun but in the second gives us useful the lexical cohesion between We about contribution information each from U i to the approaches hereafter. utterances. will compare the derived these two 4.6.3 well Evaluation of the Use of Semantic Similarity. We assessed in three ways how semantic similarity of can define in COHESIVE First, in transitions 66 and thus with contribute to the characterization Gainen Base, coherence we the text. 18 instances the of what clauses we on full coverage by intuitive the examined the In had, particular, the we basis of human judgment, designated COHESIVE transitions. focused Computational Linguistics Volume 30, Number 2 on the 5) noun(s) we providing had singled out in our in revised hand-tabulation of the ble lexical cohesion these 14 Using corpus (Ta- as individual the measures of (the transitions. method results from the in We the noun semantic in similarity U i with second the highest nouns level out of described earlier), we determined i − . semantic similarity to the nouns U 1 then compared with the picked by human judgment lexical those in determined computationally was to see if they matched. as providing cohesion which only one noun in We eliminated three cases there entity, which would, U i and thus only one possible choice for the lexically cohering Out by of similarity to the preceding utterance. of default, the 15 have had the highest semantic amples transitions, in 13 cases, or 87% of remaining ex- COHESIVE matched. the time, the human judgments and The the computationally other determined of processing of choices two assessments of the results are based on centering in claims 2: for tive transitions, claims captured rule C the rela- ease Since impose the different transitions lowest inferential load on processors, ONTINUE the ROUGH SHIFT identity s the highest. COHESIVE on Cbs with transitions act like CONTINUE we transitions but replace the a condition, conjecture that they place only con- dition on processing similarity Likewise, a slightly higher load than CONTINUE transitions. (there since COMPLETE not only SHIFT greater ROUGH SHIFT being no s Cb represent in an even discontinuity no than s the we utterance, but also entity even similar impose to entities in processing the previous ut- terance), conjecture that COMPLETE SHIFT s a higher load than ROUGH We SHIFT s. Granted reason that greater semantic similarity corresponds to a lower processing load. 15 this assumption, then, similarity, utterances that are connected by CONTINUE transitions processing have the highest semantic since CONTINUE represents the ity, COMPLETE SHIFT s, the lowest semantic similar- lowest load; those connected by since of COMPLETE SHIFT has the highest ranking from SMOOTH, ROUGH , processing load. Although and COHESIVE transitions is problematic, the particular RETAIN, saved with make we an exact determination by , the fact that, among the 66 are having to clauses well-represented full coverage, (we only note CONTINUE, COHESIVE and COMPLETE SHIFT transitions are is simply the results meaningful). for the two a enough sample to be Our RETAIN transitions, since this hardly large reasoning measures, then predicts that CONTINUE transitions have the highest semantic similarity followed by COHESIVE SHIFT s, followed by COMPLETE SHIFT s. 16 In our first assessment over 66 of this prediction, with we averaged semantic obtained groupwise coverage individual Gainen measures Base. for transition full by the Table each type the clauses of both the processing and analyses. results the ranking for load predicted on These 6 gives the averages for of support approximation measures. of However, transitions averaging over all messages provides only the a gross basis similarity we the values for each transition type. To get a more involved detailed in look at this of claim, within examined messages. the semantic distances between entities each type transition The messages 66 clauses with with only full one coverage include six messages with only one in clause and four transition type represented; this resulted the eliminaFais Transitions and Coherence Table 6 Average similarity individual measures for each transition type, for both groupwise and analyses. Number Groupwise Average analysis Individual Average analysis similarity similarity Transition type CONTINUE R ETAIN C OHESIVE COMPLETE Total SHIFT tion of 19 of clauses (since it is impossible with to determine of 47 relative grouped values into of 12 transitions messages. in either We these cases), measure. For of leaving us a total sorted types we these clauses within clauses the transitions to whether it each message by similarity each transition, inferential ascertained was made on fulfilled the of our prediction for ranking transitions by load that it concerning appropriately semantic distance. We the basis earlier as- sumptions whether was with assigned two scores for each transition: with positioned in respect Those to the preceding respect transition the sort. transitions with transition and to the following and lowest measures were only with the highest the semantic respectively, scored respect one to the following Two and the preceding identical transitions, were and thus received just score. consecu- tive one Table 7 transitions an message of scored “correct.” gives example how this scoring was in performed Gainen for Base. the clauses representative have full the It from type of that coverage lists the transition to whether each clause, the semantic is appropriately distance to the previous with clause, and the scores designating that transition (Recall positioned previous following transitions. that the predicted order is respect to the and .) with So, the CONTINUE transition to clause 235 CONTINUE, COHESIVE, COMPLETE for example, fulfills the ranking measure prediction is respect to of only both one the previous other and which the following lower is clauses; its semantic is of than that The clause, also a CONTINUE and other that a COHESIVE transition. COMPLETE SHIFT to clause 227, , on higher than the hand, fulfills the prediction vis-` a-vis the previous transition on the list Table Ranking 7 transitions in a representative message by similarity measure. [BAR] Clause Transition Semantic measure (semantic to clause distance to previous clause) Score CONTINUE CONTINUE w/? COHESIVE COMPLETE COHESIVE COMPLETE Correct Correct v Correct ↑ Correct v Correct ↑ Correct v Incorrect ↑ Incorrect v Correct ↑ Correct v ↑ Computational Linguistics Volume 30, Number 2 Table 8 Evaluation of relative similarity measures for transition types.

Groupwise analysis Individual analysis Number Percentage Number Percentage Transition type CONTINUE R ETAIN C C OHESIVE OMPLETE Total SHIFT a Since not, we we make not no claim it in as to the correct do count these totals. (i.e., its semantic with measure is lower than that of a (i.e., COHESIVE its transition) measure but is violates the prediction of respect to the following transition For 47 semantic over 12 messages, higher than number that another measures of COHESIVE in was 70: transition). scores message, 24 clauses then, the total such scores for the transitions having 23 the highest and lowest each and two each for the remaining their above, other transitions, one for positions relative to the transitions and the for their positions relative of nouns Recall to the transitions in we below them. that determined group similarity of nouns in in two i the U , different ways: first, for and the group the and second, for of group in U − 1 i − 1 each individual noun in U i . 17 Table i 8 reports the total number nouns U centage of and individual incorrect (the scores for each transition type in both the groupwise and per- correct and and mum the measure analyses out of measure measures taken for U i nouns in the in latter case ). The is the maxi- similarity the for all the U i results in this table suggest that similarity in scores can accurately That represent is, relative coherence by this small sample. the similarity we as characterized the transitions here the relative load on processing imposed scores have examined with reflect 86% 100% accuracy, with groupwise by each type of transition more between and measures than those based on the highest individual score. In scores being slightly accurate never with processing addition, similarity predict a CONTINUE transition . That is, a higher of load (i.e., lower similarity score) than a COMPLETE SHIFT the relative positions CONTINUE and COMPLETE We SHIFT are always correct. examined the cases in in which of similarity processing scores make The one an incorrect incorrect prediction about relative placement (in the scale nouns; individual-analysis load. which CONTINUE the method) of nouns 1 contains the similarity each individual involves prediction concerning a of noun an i eight in example in group U − (the in − 1 is “diluted” by the high number of nouns in U i in U i to the The number U incorrect of i nouns per clause involve in the corpus is 2.3). this case average remaining predictions the assignment scores than to COMPLETE SHIFT Some of of lower similarity to COHESIVE transitions of world s. these infer are the result the fact that knowledge is lower necessary scores for COHESIVE transitions connection clauses. This is the for example, in (7): to a between the two case, Fais 7a. toode demo Toukyo at a distance though Tokyo 7b.

enkai taimu niwa drinking if it’s party we time for “Even far can party.” come back time for a drinking It requires world is knowledge to understand it will that there a connection Although between how far away make something inference and intuitively, the time that take to travel is not there. in humans Gainen can Base. this that understanding represented the 18 Other incorrect judgments are the result of how similarity don’t match the database our intuitive handles determining the (8): scores, “quirks” that seem to judgments, as in jikantekiniwa choudo taimingu wa iidarou shi timewise TOP exact timing TOP seems good and 8b. chiritekini mo geographically too ∅ tookuwa nainde far TOP not “Timewise, either.” the timing seems good, and the night game is not far away The Gainen yoru, Base yielded a our relatively strong low score for the similarity and despite that these words between ‘time,’ ‘night,’ sense two should be closely related. 19 Overall, however, of similarity scores of seem to provide This a fairly accurate the coherence this corpus. result, with measure of relative subset the coupled the high Transitions and Coherence Cb=? (from previous clause) Cf = distance Tokyo , COHESIVE Cb=? Cf = party, time is Cb=? (from previous Cf clause) = time, timing COHESIVE Cb=? Cf = night game ∅ , geography jiken, Computational Linguistics Volume 30, Number 2 level of correlation discussed earlier between lexically cohesive similarity, entities designated our by human judgment and those determined as measured by semantic supports proposal that lexical cohesion, well-defined notion by semantic distance, of can feasibly be included as a to capture crucial aspects text coherence. 4.7 Exploring the Implications of the COHESIVE The Transition and 4.7.1 Need Identify Relation usefulness of COMPLETE SHIFT The identity Cbs to Type. Poesio (2000) relaxing others the (Hahn, notion 1996; recognized by et al. and Markert, of and Strube for has been identity Murata, with Isahara, and Nagao 1999). Poesio et al. supplemented Cfs in the relation membership, three different possible semantic relations utterances: induced number of subset, and “generalized possession.” Murata, between Ishara, the Nagao set and a possible relations between using frame dictionary and of Japanese bridging reference and anchor a verb nominal case a corpus AnoB expressions and arguments and the AnoB encodes wide (where A B are nominal construction [Shimazu, Naito, a Nomura variety of semantic 1987]). relations between the two arguments and of As noted in of Section 4.3.1, however, in all of these approaches, only a small identify subset examples bridging relations can be handled, because This they is all necessary attempt to some particular relation existing between two elements. a move of building text understanding systems, but neither for resolving or is bridging our references aim Cfs We in merely and those here. need Cfs to in identify . Utilizing the level instead of semantic similarity measure U U i − 1 the over more closeness between of proposed i and general semantic-distance here, First, then, it has several which relation labels. avoids the need advantages to make the ex- plicit choice particular interlabeler or should or Poesio should not be included, as well choices about ficulties relations with can cannot, note, is no as the dif- in our approach. This is reliability that et al. since there actually tied another advantage: There is no labeling to is limit need the types of closely semantic our relationships into to of which the Cfs can enter. inferable restrict analysis to a subset the phenomenon; our Thus, need no there to of account centers semantic relationship to the in will handle having any kind centers the previous utterance. of 4.7.2 Overestimation Is of COMPLETE number SHIFT s. In examining Table of 5, we Of this an accurate estimation the (in)coherence see a high number COMPLETE SHIFT s. as we in The (5), when a of message messages makes in a shift in in this corpus? course, saw occur. writers topic, made we expect a COMPLETE (see Table SHIFT 2); to we the is no this corpus writers’ 146 paragraphs will although with know in that there number guarantee that paragraphing of coincide maximal shifts number cohesion, of this at least (we gives would us a general writers estimation on a possible of more COMPLETE SHIFT s more the side paragraphs than topics rather than on expect to err The number of is the side within of topics than paragraphs). COMPLETE SHIFT s comfortably that But range. appropriate there are two confounding nature of factors that The make first this is number inability higher of than is ac- tually approach for the (Fais the corpus. Yamura-Takei 2003). the Consider (9): a centering to handle event deixis and 9a. kyukyo urgent Cb=? Cf = 17th, U-san, match-up business, Fais Transitions and Coherence dou kahdo match-up ni henkoushitai want tonokotodesu same of U-san’s to to change thing “Because business, she would is urgent match-up on 17th.” like to COMPLETE SHIFT change to the same the 9b. Cb=? Cf = that ∅ 9c. RETAIN W/ ? Cb = Cf that ∅ = everyone, that ∅ teian Cb=? Cf = all, I-san suggestion COMPLETE SHIFT Cb=? Cf = first-base side, third-base side COMPLETE SHIFT 10c. sendagayanara Sendagaya if ∅ ∅ “If it’s Sendagaya, we late.” Cb=? Cf = Sendagaya, party ∅ , lack of caring Computational Linguistics Volume 30, Number 2 The COMPLETE of SHIFT s in this passage in are perfectly message. appropriate; among discourse entities this Why would there is no cohesion any was the completely, incoherent? In anyone send a mes- sage that outcomes of by this account, message in fact, the to discussions held the course of of statements in (10) refer the in This various occurs exchanging of the mes- sages this corpus. of of what toward the end the exchange is sports event This to of in writer go is to, where to messages, as resolution the questions sit, and what restaurant on to go to afterward sight. The simply adding his opinions each only in these apparently unrelated of topics. relationship we the understanding imagine the coparticipants in the message among them all holds might of mechanisms exchange. be required model Although able we to the sorts way from to this level of understanding, are a very long realizing them. watashi I mo ∅ terebi too ∅ TV Cb=? Cf = lacrosse ∅ , 20 TV 11b. namade watching kansen live game Cb=? Cf = watching live game “Since I have if it seen game, but is on the 25th, join us for the game, can (11d) Lacrosse in (11a), (some) live game watching in (11b) and (11c), and (a) game (on the 25th) in is important are actually semantically distinct elements, or and recognizing in full-text understanding summarizing. However, their maintaining distinctiveness for a account means these distinctions standard characterizing (It is not the transitions we would between the utterances containing them as NULL that game (on the 25th) was SHIFT inferable s. clear from that even want to Being say (a) an center (some) live game able these COHESIVE allows both maintain watching .) to designate transitions as us to the dato is if Cb = watching live game Cf ∅ = watching live game ∅ , 25th ga Cb=? M-san, H-san, game ∅ Fais Transitions and Coherence semantic distinctiveness of of the discourse elements and to capture the coherence in this portion the discourse. 4.7.4 of Lexical Cohesion notion of and Discourse Segmentation. As we saw is earlier, the some cohesion to delineate discourse structure fairly well re- use searched. ment is In lexical the lexical chain where approach (Morris and Hirst is, 1991), not the chain “breaks,” that where a new discourse seg- hypothesized subsequent bear a relationship to previous entities that would entities do semantic Kozima (1993) provides allow them the algorithm for determining where, to be added to of chain. semantic of words an in text, likely topic breaks occur on the graph cohesion values a We would say, once and validates that determination or against in human judgment. then, a breaks a significant dip the semantic cohesion value graph occurs, that chain utterance break is considered the first of new the following such a utterance a discourse segment. In sufficiently terms of semantic distance measure as might determined by the Gainen Base, we suggest that a of low new similarity Once characterize “sufficiently” both COMPLETE must SHIFT s and a segment. again, be defined; in the beginning of discourse light of the will preliminary results we measures saw previously, we not relative to the for similarity in conjecture the message that the definition “low” be (see Section 5.4). under scrutiny and In an absolute value addition, examination semantic might of the particular entities contributing also allow characterize the notion of to high levels of in similarity way. That is, us if we to not one “global topic,” albeit measure a differentiated of in determine just individual semantic distance for the sets of entities two adjacent each those (as utterances, but briefly described in the note 13), we distances for mine combination entities measure that are contributing the greatest amount of can deter- those entities identify of semantic similarity to the and a cluster entities that can be taken to represent a global topic. This concept of is worth theory. examining The more Cb of closely, since it bears on some of the very foundations centering (Walker, Joshi, U i most an utterance “represents Prince centrally 1998, concerns, similar to what is the discourse entity that the utterance 3). The of elsewhere Cb is called the ‘topic’” necessary and our a a sufficient page presence a and condition for topic coherence. However, taken to be both utterances nonetheless have a no coherent Cb; relationship is, to the immediate in of account, may that approach, that the presence of a Cb is, context discourse only nature sufficient have of condition. is maintained Our claim, in then, is that this more in our a not only through accurately reflects the of how coherence semantically. entity, In but our also through discourse: Cbs the successive use of the explicit repetition a central entities that are in closely related theory, account, are recognized and in function just as standard of centering not but their absence, a in common situation some does signal breakdown coherence. Coherence at least maintained kinds discourse, well a by similar which become Cbs in may be own as semantically entities, in (6). Using can individuated approach their right, as match-upinformation similarity, and we boxing identify do an to determining semantic can these particular entities.

5.1 There Refinement we a number of the of COHESIVE aspects of Transition are the proposal that need further Cf in scrutiny. in motivation transition by reference to any U i . It Note might that have defined the COHESIVE case, is be the fact, that there to define two different COHESIVE transitions: a Computational Linguistics Volume 30, Number 2 “ CONTINUE COHESIVE Cf( ), ,” in which the Cp( U) i has ,” in the highest to 1 and “ RETAIN COHESIVE which similarity other Cf measure with respect U Cf( i − ). a In our some implementation with has the highest similarity in to with U maximum i − 1 preliminary was this i to 1 Cp of corpus, 55% the the U the U of entity U Whether semantic is similarity necessary or meaningful i − one is i the time. such a distinction a a completely open question. In in Cfs a similar in vein, we might suggest that − as a type of the Cb Cf in (say, Cb U i with ′ i ). So, the highest similarity (2b) 1 be designated for example, if to the U of level of What would similarity to restaurant in (2a), soba the soba shop would shop Cb had highest of move be the ′ that utterance. 21 Certainly would be the ramifications with Cb this ?. In for of the theory? there be far fewer utterances = in if one of the major aims of of approach, terms is on the centering similarity, this is tracking focus However, the discourse, Cb a positive result. the ′ major insight of chosen theory, simply namely, the basis semantic is not we lose another (Hudson-D’Zmura centering Tanenhaus 1998) or that focus word order dependent Gordon, and upon (Gordon upon semantics 1999; Grosz, Gilliom 1993), et see and but upon the grammatical roles al. but also played Cfs by the Cfs. It strong is possible that for any given language, in speakers tend roles to place on that Cf have semantic is ties to the previous utterance we grammatical but question that do not high the template, this an empirical have the data to answer here.

douyara

tsuyuiri shitarashii rainy season beginning has happened toiu sengenmodeta youde declaration with Cb=? Cf = beginning of rainy season, declaration COHESIVE 12b. kyou mo shikkari ame ga futteimasu today heavily rain SUBJ is raining Cb=? Cf = rain , today “With the today.” declaration that the rainy season has come, it is raining heavily Fais Transitions and Coherence 13a. watashi me wa naniyorihitsuyounamono more important wa TOP than anything else TOP ki feeling 13b. 13c. “That IS MORE gave IMPORTANT a watched (that THAN ANYTHING me ELSE. If impressions.” someone else game), please let know your We have the intuition that ones the in COHESIVE (13). Both transition of in (12) can message, the these examples come from as “more” COHESIVE than and similarity measures for each of the same our intuitions: the The semantic measure these examples (13a) for COHESIVE 0.166; transition between (12a) confirms 0.583; the 0.011. that between and (13b), and for that between (13b) and (12b) and (13c), is for 22 It is not clear of to what of use we might put this more detailed however, information Fais strength connection among utterances; it about the observation varying levels marking in (2001) some e-mail authors make parallels the in that messages, hierarchical distinctions in paragraphs their using line breaks to separate utterance clusters having weak or some no semantic connection and full spaces to separate utterance clusters having Recall, semantic connection. of too, that Merging lexical cohesion, measure as measured with information by semantic provided distance, is only one aspect coherence. this by conjunctions, ga shitanodesu SUBJ got ga but kansou impression wo kikasetekudasai OBJ let me know me strong impression that PHYSICAL STRENGTH Cb=? Cf = most feeling, important thing, strength COHESIVE Cb=? Cf = person, game ∅ COHESIVE Cb=? Cf = impressions be characterized Computational Linguistics Volume 30, Number 2 referential of form, and other aspects of discourse would allow a more comprehensive account coherence. 5.3 Beyond The Simplex Up of Nominals ramifications we exploiting lexical cohesion only within a centering theory are exciting. But to this not point, only have (simplex) considered nominal the cohesion into exhibited by discourse entities. (14), in which arguments of enter the contribution of cohesive relationships. Consider recognition the more the verb to the cohesion excerpt allow to accurately for the coherence in of the could us account the passage: Cb=? Cf = tomorrow, Saturday COHESIVE Cb=? Cf = meeting ticket, place, discussion, Monday COMPLETE SHIFT (Cb = Cf working ? = people) The (14a) first transition shown in (14b). is But COHESIVE by virtue only of the Cfs tomorrow and Saturday in in Cf and Monday in (14c) considering of is not the in with discourse any of of entities the people cohesive message the other appearing lists, working natural. inclusion if we (14c) elements in this passage, although However, the this portion the seems quite coherent and with allow verbal elements , to contribute work,’ to cohesion, working people why then shows cohesion shusshashinai ‘won’t go to and we can explain would In the passage seems a similar the incorporation coherent. vein, our measurement of an of adequate analysis well. for Recall complex (2), nominals allow us to refine cohesion as in which lexical cohesion While exists among the entities of enseki ‘restaurant,’ sake ‘alcohol,’ and sobaya, ‘soba shop.’ in message, the recognition in we probably this cohesion allows us to characterize the our fact, underestimate the cohesion present if we co- herence base measures the not on the individual nominals, lexical items listed. nominals If we could well, calculate we would lexical cohesion just for simplex but for complex as calculate cohesion for enseki and for the entire phrase sake wo nomeru umai sobaya , ‘a Fais Transitions and Coherence good soba nection shop, which also serves alcohol,’ individual possibly items. a much stronger cohesive con- than that among the 23 5.4 We Improving Implementation demonstrated Base provides in Section 4.6 that semantic rigorous distance of as measured by the or a feasible a definition lexical cohesion. In Gainen basis for however, any implementation need of semantic distance The first measure is to order for this similar two major be fully of areas to be Gainen Base while effective, addressed. coverage; Gainen by the over does not the of prevent us from lack of complete coverage it of the corpus assessment the Base’s effectiveness a subset the data, it making an impossible needs over corpus as a whole. The does make to characterize cohesion be has do with the Throughout our to definitions of second area that to addressed we distance for each transition type. within messages; discussion it would have examined semantic distances whether as relative independent to determine empirically it is strengths be useful These might possible to set definitive, (e.g., levels for each transition type. levels what more SHIFT likely, s are those transitions having a semantic measure of 0.15– be absolute 0.3) or, SMOOTH in seems message relative, is such that a portion of the which defined from 0.005 0.875, type (e.g., range of a for each transition in semantic distance in particular measure a message semantic distances 15% 25% of to of SMOOTH to falling between and the total 0.87, SHIFT that is, s from correspond distances 0.22, message). range 0.13 to for that

Upon subjecting a corpus of Japanese of e-mail messages data to a centering analysis, a description these had to rely heavily on inferable it became clear that notion was centering and not to capture the coherence this corpus displays. We cen- ters of adequate inferable were intended in couched the more connectedness that of centers We to capture terms of the principled and explicit relation inventory of lexical relatedness. with well-defined used this relation, then, to supplement more the standard nature transitions types characterize the of coherence in transition that accurately of approach in this corpus imple- and demonstrated mentation. The the proposed computational feasibility in theory, provide this a preliminary transition types namely, a characterization inferable This is improvement the coherence over of of a previously unaccounted-for situation centering Cbs. sequences of utterances model containing of number of nonexplicit a crucial Cbs in (and the Passonneau high this corpus others; standard because 1998). the The 76% inclusion of of see a in COHESIVE which nonexplicit transition to the the Cbs and COMPLETE SHIFT allows us of characterize in corpus approach, while maintaining play a part, a portion operation the corpus Cbs of undescribed the standard transition states of centering theory in cases in which the standard the usual explicit It are may present. make be the other case that lexical cohesion and the notion well, of a COHESIVE transi- tion can contributions in to discourse us coherence a discourse while analysis as such as allowing to characterize Further, notions still recognizing referentially of dis- tinct discourse of elements. these of can augment information the definition in the topic a discourse Finally, segment by virtue the semantic provide contained a cohesion analysis. such an analysis can clues to discourse segment boundaries. Computational Linguistics Volume 30, Number 2 The incorporation of the of notion of lexical in cohesion has In been shown to the coherence this corpus. addition, it to be crucial the characterization of number further contributions to discourse analysis. And finally, it opens can make a variety of the door to a useful related strategies modeling that can in allow us to come closer to understanding and comprehensively coherence discourse. Acknowledgments I would like to thank Mitsuko observations Yamura-Takei for major critical in work, concerning Francis Bond the insightful points this for his and gentle comments, and an anonymous would translator for the translations. I also like of to Shigeaki acknowledge Amano, the unflagging Kasahara, Takashi aid Hamo, Tomoko Kaname Kawaguchi, providing and Gainen Yuuko Base Kanasugi in the analysis. References Boguraev, 2000. Lexical Branimir K. and Mary S. Neff. cohesion, discourse segmentation, and Paper document summarization. RIAO-2000, Paris. presented at Brennan, Carl Susan J. Pollard. E., Marilyn 1987. W. A Friedman, and approach pronouns. In Proceedings centering Annual to Meeting Linguistics the , Association of the 25th Computational of 155–162, for Stanford, CA. pages Chafe, Wallace. and flow of 1979. The flow language. In Talmy of thought editor, the Syntax Semantics: Discourse Givon, Syntax, and 12. Academic Press, New and York, volume Chafe, Wallace. pages 159–182. Cognitive, Cultural 1980. The Productions . Linguistic Pear Stories: Narrative and Ablex, Norwood, Aspects of Clark, NJ. Herbert H. 1977. In David Inferences LaBerge in comprehension. Jay Samuels, Basic Processes and S. Reading: Perception editors, Mahwah, NJ, Comprehension in Erlbaum, and Cote, 1998. Ranking pages 83–112. . Sharon. forward-looking Walker, Aravind centers. K. Joshi, In Marilyn Ellen F. A. Prince, Discourse . Clarendon, Centering Oxford, Theory and editors, in Fais, 55–69. pages Laurel. 2001. of Japanese Discourse issues In in the translation Proceedings 93–102, of PACLING email. Kitakyushu, Fukuoka, 2001 , pages Fais, Laurel. 2002. The Japanese Japan. In AnoB construction Eighth Annual and Language Meeting centering. of the (NLP Association Proceedings, Natural Processing for 603–606, Keihanna, Japan. 2002) , pages Fais, Laurel 2003. The and Mitsuko nature of Yamura-Takei. Japanese Discourse referent Processes resolution , in 36(3):167–204. email. Fujita, Sanae method of and Francis new Bond. 2002. A adding entries to a valency dictionary In by Proceedings exploiting existing lexical International resources. Conference Theoretical of the Ninth Methodological Issues Machine on Translation and Gordon, (TMI-2002) Peter , C., Barbara 45–52, in pages J. Keihanna, Grosz, Japan. Laura A. Gilliom. 1993. Pronouns, and of in names, and the centering Cognitive Science attention Gordon, discourse. Peter C., Randall Hendrick, , 17:311–347. Ledoux, Chin Lung Yang. 1999. Kerry Processing and of of An reference and of the structure noun grammar: Language analysis complex Processes phrases. , 14(4):353–379. and Cognitive Grosz, 1986. Barbara Attention, J. and intentions, Candace L. Sidner. of Computational and the structure Linguistics , 12(3):175–204. discourse. Grosz, 1998. Barbara Lost intuitions J. and Candace L. Sidner. intentions. In Marilyn and A. forgotten K. Joshi, Ellen F. Prince, Walker, Aravind Centering and Theory Discourse . editors, Oxford, 39–51. in Clarendon, Hahn, Udo, pages Strube. 1996. Katja A Markert, and Michael approach conceptual reasoning 12th to European textual Conference ellipsis. In Artificial Proceedings of Intelligence the , 572–576, Budapest. on Halliday, Functional Michael pages Grammar A. K. 1994. . Arnold, An London. Introduction Harabagiu, to inference Sanda of M. 1998. WordNet–based In textual Proceedings cohesion FLAIRS-98 and coherence. Sanibel Island, FL, 265–269. of , Harabagiu, Sanda M. pages 1999. From A lexical cohesion to textual coherence: Journal Pattern data driven Recognition perspective. Artificial Intelligence of 13(2):241–265. and , Hudson-D’Zmura, Tanenhaus. 1998. Susan Assigning and Michael K. to pronouns: The antecedents ambiguous of role of the center attention In Marilyn as the A. default assignment. Aravind K. Joshi, Ellen Walker, and F. Prince, Fais editors, Centering Clarendon, Theory Oxford, in Discourse Huls, Edwin Bos, pages 199–226. . Carla, 1995. Automatic and Wim Claassen. referent resolution of deictic and anaphoric Computational Linguistics expressions. Hurewitz, Felicia. 1998. A , 21(1):59–79. In quantitative Marilyn A. look at Walker, discourse Aravind coherence. Prince, K. Joshi, Centering and Ellen Discourse . Clarendon, Theory F. editors, Oxford, in 273–291. pages Hutchins, John. 1995. Reflections of on machine the history and In present The MT state translation. Proceedings , Summit 89–96, V Kameyama, Megumi. pages 1998. Luxembourg. A study. Intrasentential centering: Walker, Aravind case K. Joshi, In Marilyn Ellen A. Prince, Centering Theory and F. Discourse editors, . Clarendon, Oxford, in 89–112. pages Kasahara, Tsutomu Kaname, Ishikawa, Kazumitsu Tsukasa Matsuzawa, Kawaoka. 1996. Viewpoint-based and measurement words. of In semantic Hans Douglas similarity between Lenz, Learning H. Fisher Data: and Statistics editors, V . Springer-Verlag, from New AI and Kozima, York, pages Hideki. 433–442. on 1993. Text segmentation based Proceedings similarity Association Computational 31st between Annual Meeting words. In of the Linguistics of the 286–288, for Columbus, OH. , Kozima, pages Similarity Hideki and Teiji words Furugori. 1993. between on computed English by spreading dictionary. activation In Proceedings an Conference European Chapter, of the Sixth Association of the 232–239, for Computational Utrecht, Linguistics Netherlands. , Kuno, pages Susumu. Language 1973. MIT The the Japanese . Press, Structure Cambridge, of the Lotfipour-Saedi, MA. Kazem. 1997. Lexical cohesion Meta, 42(1):185–192. and translation equivalence. Matsui, Tomoko. in 1999. On the role of context of relevance-based accessibility ranking Bouquet, Luciano candidate Serafini, referents. Patrick In Paolo Brezillon, Francesca Massimo Castellani, Benerecetti, CONTEXT and (Springer Lecture editors, ‘99 Intelligence, no. 1688), Notes in Morris, Graeme pages Hirst. 228–241. Artificial Lexical Jane and 1991. cohesion indicator computed of by thesaural relations Computational as an Linguistics the , structure 17(1):21–48. of Murata, text. Masaki, Hitoshi Isahara, and Transitions and Coherence Makoto indirect Nagao. 1999. in Resolution Japanese of anaphora using Proceedings “X no Y (Y sentences examples ACL99 of X).” Workshop In Coreference of the Its Applications , College on Park, MD. and Nakaiwa, Hiromi, Anaphora and Satoshi of Shirai. Japanese 1996. pronouns with resolution zero Proceedings COLING-96 deictic reference. Shigeko. 2000. , In Nariyama, of Referent pages 812–817. Identification Japanese. Ph.D. for Ellipted Arguments University of in Melbourne, Melbourne, thesis, Nippon Telegraph Australia. Corporation, and Telephone Laboratories. Communication 1996. Reference Science Morphological Analysis Manual Program ALT-JAWS for Keihanna, Telephone. Japan: Nippon Telegraph and Okumura, Word Manabu, and Takeo Honda. 1994. sense disambiguation on and text segmentation In Proceedings based 15th lexical International cohesion. Conference Computational of the 755–761, on Kyoto, Japan. Linguistics , Passonneau, pages Rebecca J. with 1998. Interaction of of discourse structure noun explicitness discourse Marilyn A. anaphoric Walker, Aravind phrases. K. Joshi, In Ellen Discourse F. Prince, Clarendon, editors, Oxford, Centering Theory and . in 327–358. pages Paul, Michael method and Eiichiro pronominal Sumita 2001. A trainable for anaphora information. resolution Shizen Gengo using Natural Processing], Shori shallow Language 8(3):59–85. [Journal of Poesio, Henschel, Massimo, Janet Hua Hitzeman, Cheng, Rodger Renate Rosemary Stevenson. 2000. Kibble, and Specifying A the parameters of centering theory: from corpus-based evaluation using text In Proceedings application-oriented 38th Annual domains. Association of the Computational Linguistics Meeting of the Hong Kong, for 400–407. , Sack, Warren. 1999. pages In Proceedings Diagrams of 37th social cohesion. Meeting Association of the Linguistics the , Hong Kong. for Computational Annual of Sag, Bond, Ivan Ann A., Timothy Copestake, Baldwin, Dan Francis Flickinger. A 2002. neck Multiword and in NLP. expressions: Gelbuk, pain editor, the Computational for In Linguistics Alexander and International Intelligent Conference Text Processing: (CICLing-2002) Third Springer-Verlag, Heidelberg/Berlin, , 1–15. pages Shimazu, Akira, Shozo Naito, and Hirosato . Computational Linguistics Nomura. of 1987. Japanese Semantic noun structure with analysis In phrases Meeting Proceedings adnominal particles. Association of the 25th Annual of Linguistics the Strube, , for Michael, 123–130, Computational pages Udo Stanford, Hahn. 1999. CA. Functional and centering—Grounding referential coherence in Computational information structure. 25(3):309–344. Linguistics , Suri, Linda Z. RAFT/RAPR and Kathleen F. McCoy. A 1994. and centering: comparison processing and discussion of problems related Computational to Linguistics complex , 20(2):301–317. sentences. Tetreault, Joel R. of 2001. A corpus-based evaluation Computational centering and pronoun resolution. 27(4):507–520. Linguistics , Vieira, An Renata, and Massimo Poesio. 2001. processing empirically based system for Computational definite Linguistics descriptions. , 26(4):539–593. Volume 30, Number 2 Walker, Marilyn A. 1998. Centering anaphora resolution, In Marilyn and A. discourse structure. K. Joshi, Ellen F. Walker, Prince, Aravind Centering and Theory Discourse editors, Oxford, in Walker, pages 401–435. . Clarendon, Marilyn Sharon Cote. A., Masayo 1994. Iida, Japanese and process of discourse and Computational the Linguistics centering. Walker, Marilyn A., , Aravind 20(2):193–231. Ellen F. Prince. 1998. Joshi, Centering in and naturally overview. occurring In Marilyn discourse: Joshi, F. A. An K. Ellen Walker, Prince, Aravind Centering and Theory Discourse . editors, Oxford, 1–28. in Clarendon, Yamura-Takei, pages Makoto Yoshie, Mitsuko, Teruaki Miho Fujiwara, Automatic and Aizawa. 2002. The linguistic of analysis zeros. In for Proceedings language teachers: Computational 19th International case of the Linguistics Conference 1114–1120, (COLING on Taipei, Taiwan. ‘2002) , pages