Shalom Lappin" SOAS, University of London Herbert J. Leass t Sietec Systemtechnik This paper presents an algorithm for identifying the noun phrase antecedents of third person pronouns and lexical anaphors (reflexives and reciprocals). The algorithm applies to the syntactic representations generated by McCord's Slot Grammar parser and relies on salience measures derived from syntactic structure and a simple dynamic model of attentional state. Like the parser, the algorithm is implemented in Prolog. The authors have tested it extensively on computer manual texts and conducted ablind test on manual text containing 360 pronoun occurrences. The algorithm successfully identifies the antecedent of the pronoun for 86% of these pronoun occurrences. The relative contributions of the algorithm's components to its overall success rate in this blind test are examined. Experiments were conducted with an enhancement of the algorithm that contributes tatistically modelled information concerning semantic and real-world relations to the algorithm's decision procedure. Interestingly, this enhancement only marginally improves the algorithm's performance (by 2%). The algorithm is compared with other approaches to anaphora resolution that have been proposed in the literature. In particular, the search procedure of Hobbs'algorithm was implemented in the Slot Grammar framework and applied to the sentences in the blind test set. The authors" algorithm achieves a higher rate of success (4%) than Hobbs' algorithm. The relation of the algorithm to the centering approach is discussed, as well as to models of anaphora resolution that invoke a variety of informational factors in ranking antecedent candidates. 1. Introduction We present an algorithm for identifying both intrasentential and intersentential antecedents of pronouns in text. We refer to this algorithm as RAP (Resolution of Anaphora Procedure). RAP applies to the syntactic structures of McCord's (1990, 1993, in press) Slot Grammar parser, and like the parser, it is implemented in Prolog. It relies on measures of salience derived from syntactic structure and a simple dynamic model of attentional state to select the antecedent noun phrase (NP) of a pronoun from a list of candidates. It does not employ semantic conditions (beyond those implicit in grammatical number and gender agreement) or real-world knowledge in evaluating candidate antecedents; nor does it model intentional or global discourse structure (as in Grosz and Sidner 1986). * School of Oriental and African Studies, University of London, London WCIH OXG, UK. E-mail:
slappin@clusl .ulcc.ac.uk
Most of the first author's work on this paper was done while he was a Research Staff Member in the
Computer Science Department ofthe IBM T.J. Watson Research Center. t Sietec Systemtechnik (Siemens AG), D-13623 Berlin, Germany. E-mail: leass@sietec.de
The second author's work on this paper was done while he was a visiting scientist at the IBM
Germany Scientific Center. @ 1994 Association for Computational Linguistics Computational Linguistics Volume 20, Number 4
In Section 2 we present RAP and discuss its main properties. We provide examples of its output for different sorts of cases in Section 3. Most of these examples are taken from the computer manual texts on which we trained the algorithm. We give the results of a blind test in Section 4, as well as an analysis of the relative contributions of the algorithm's components to the overall success rate. In Section 5 we discuss a procedure developed by Dagan (1992) for using statistically measured lexical preference patterns to reevaluate RAP's salience rankings of antecedent candidates. We present he results of a comparative blind test of RAP and this procedure. Finally, in Section 6 we compare RAP to several other approaches to anaphora resolution that have been proposed in the computational literature.
RAP contains the following main components. • An intrasentential syntactic filter for ruling out anaphoric dependence of a pronoun on an NP on syntactic grounds (This filter is presented in Lappin and McCord 1990a.) • A morphological filter for ruling out anaphoric dependence of a pronoun on an NP due to non-agreement of person, number, or gender features • A procedure for identifying pleonastic (semantically empty) pronouns • An anaphor binding algorithm for identifying the possible antecedent binder of a lexical anaphor (reciprocal or reflexive pronoun) within the same sentence (This algorithm is presented in Lappin and McCord 1990b.) • A procedure for assigning values to several salience parameters (grammatical role, parallelism of grammatical roles, frequency of mention, proximity, and sentence recency) for an NP. (Earlier versions of these procedures are presented in Leass and Schwall 1991.) This procedure employs a grammatical role hierarchy according to which the evaluation rules assign higher salience weights to (i) subject over non-subject NPs, (ii) direct objects over other complements, (iii) arguments of a verb over adjuncts and objects of prepositional phrase (PP) adjuncts of the verb, and (iv) head nouns over complements of head nouns. 1 • A procedure for identifying anaphorically linked NPs as an equivalence class for which a global salience value is computed as the sum of the salience values of its elements. • A decision procedure for selecting the preferred element of a list of antecedent candidates for a pronoun. 1 This hierarchy is more or less identical to the NP accessibility hierarchy proposed by Keenan and
Comrie (1977). Johnson (1977) uses a similar grammatical role hierarchy to specify a set of constraints
on syntactic relations, including reflexive binding. Lappin (1985) employs it as a salience hierarchy to
state a non-coreference constraint for pronouns. Guenthner and Lehmann (1983) use a similar salience
ranking of grammatical roles to formulate rules of anaphora resolution. Centering approaches to
anaphora resolution use similar hierarchies as well (Brennan, Friedman, and Pollard 1987; Walker, Iida,
and Cote 1990). Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution 2.1 Some Preliminary Details RAP has been implemented for both ESG and GSG (English and German Slot Grammars); we will limit ourselves here to a discussion of the English version. The differences between the two versions are at present minimal, primarily owing to the fact that we have devoted most of our attention to analysis of English. As with Slot Grammar systems in general (McCord 1989b, 1993, in press), an architecture was adopted that "factors out" language-specific elements of the algorithm.
We have integrated RAP into McCord's (1989a, 1989b) Logic-Based Machine Translation System (LMT). (We are grateful to Michael McCord and Ullrike Schwall for their help in implementing this integration.) When the algorithm identifies the antecedent of a pronoun in the source language, the agreement features of the head of the NP corresponding to the antecedent in the target language are used to generate the pronoun in the target language. Thus, for example, neuter third person pronouns in English are mapped into pronouns with the correct gender feature in German, in which inanimate nouns are marked for gender.
RAP operates primarily on a clausal representation f the Slot Grammar analysis of the current sentence in a text (McCord et al. 1992). The clausal representation consists of a set of Prolog unit clauses that provide information on the head-argument and head-adjunct relations of the phrase structure that the Slot Grammar assigns to a sentence (phrase). Clausal representations of the previous four sentences in the text are retained in the Prolog workspace. The discourse representation used by our algorithm consists of these clausal representations, together with additional unit clauses declaring discourse referents evoked by NPs in the text and specifying anaphoric links among discourse referents. 2 All information pertaining to a discourse referent or its evoking NP is accessed via an identifier (ID), a Prolog term containing two integers. The first integer identifies the sentence in which the evoking NP occurs, with the sentences in a text being numbered consecutively. The second integer indicates the position of the NP's head word in the sentence. 2.1.1 The Syntactic Filter on Pronoun-NP Coreference. The filter consists of six conditions for NP-pronoun non-coreference within a sentence. To state these conditions, we use the following terminology. The agreement features of an NP are its number, person, and gender features. We will say that a phrase P is in the argument domain of a phrase N iff P and N are both arguments of the same head. We will say that P is in the adjunct domain of N iff N is an argument of a head H, P is the object of a preposition PREP, and PREP is an adjunct of H. P is in the NP domain of N iff N is the determiner of a noun Q and (i) P is an argument of Q, or (ii) P is the object of a preposition PREP and PREP is an adjunct of Q. A phrase P is contained in a phrase Q iff (i) P is either an argument or an adjunct of Q, i.e., P is immediately contained in Q, or (ii) P is immediately contained in some phrase R, and R is contained in Q.
A pronoun P is non-coreferential with a (non-reflexive or non-reciprocal) noun phrase N if any of the following conditions hold: 1. P and N have incompatible agreement features. 2. P is in the argument domain of N. 3. P is in the adjunct domain of N. Computational Linguistics Volume 20, Number 4 . . P is an argument of a head H, N is not a pronoun, and N is contained in H. P is in the NP domain of N. 6. P is a determiner of a noun Q, and N is contained in Q.
Examples of coindexings that would be rejected by these conditions are given in Figure 1. Condition 1:
The womani said that he/is funny. Condition 2:
Shei likes her/.
John/seems to want to see himi. Condition 3:
She/sat near her/. Condition 4:
He/believes that the mani is amusing.
This is the man/hei said John/wrote about. Condition 5:
John/s portrait of himi is interesting. Condition 6:
Hisi portrait of John/is interesting.
Hisi description of the portrait by John/is interesting. Figure 1 Conditions on NP-pronoun non-coreference ( xamples). 2.1.2 Test for Pleonastic Pronouns. The tests are partly syntactic and partly lexical. A class of modal adjectives is specified. It includes the following items (and their corresponding morphological negations, as well as comparative and superlative forms). necessary good economical possible useful easy certain advisable desirable likely convenient difficult important sufficient legal A class of cognitive verbs with the following elements is also specified. recommend think believe know anticipate assume expect It appearing in the constructions of Figure 2 is considered pleonastic (Cogv-ed = passive participle of cognitive verb); syntactic variants of these constructions (It is not~may be Modaladj..., Wouldn't it be Modaladj .... etc.) are recognized as well.
To our knowledge, no other computational treatment of pronominal anaphora resolution has addressed the problem of pleonastic pronouns. It could be argued that recognizing pleonastic uses of pronouns is a task for levels of syntactic/semantic analysis that precede anaphora resolution. With the help of semantic lasses defined in the lexicon, it should be possible to include exhaustive tests for these constructions in Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution It is Modaladj that S It is Modaladj (for NP) to VP It is Cogv-ed that S It seems/appears/means/follows (that) S NP makes/finds it Modaladj (for NP) to VP It is time to VP It is thanks to NP that S Figure 2 Pleonastic uses of it. analysis grammars. 3
following formulation of the binding algorithm is defined by the following hierarchy of argument slots: subj > agent > obj > (iobjlpobj) Here subj is the surface subject slot, agent is the deep subject slot of a verb heading a passive VP, obj is the direct object slot, iobj is the indirect object slot, and pobj is the object of a PP complement of a verb, as in put NP on NP. We assume the definitions of argument domain, adjunct domain, and NP domain given above.
A noun phrase N is a possible antecedent binder for a lexical anaphor (i.e., reciprocal or reflexive pronoun) A iff N and A do not have incompatible agreement features, and one of the following five conditions holds. 1. A is in the argument domain of N, and N fills a higher argument slot than A. , A is in the adjunct domain of N. 3. A is in the NP domain of N. 4. N is an argument of a verb V, there is an NP Q in the argument domain or the adjunct domain of N such that Q has no noun determiner, and (i) A is an argument of Q, or (ii) A is an argument of a preposition PREP and PREP is an adjunct of Q. . A is a determiner of a noun Q, and (i) Q is in the argument domain of N and N fills a higher argument slot than Q, or (ii) Q is in the adjunct domain of N. Examples of bindings licensed by these conditions are given in Figure 3. 2.1.4 Salience Weighting. Salience weighting is accomplished using salience factors. A given salience factor is associated with one or more discourse referents. These discourse referents are said to be in the factor's scope. A weight is associated with each Computational Linguistics Volume 20, Number 4 Condition 1:
They/wanted to see themselves/.
Mary knows the people/who John introduced to each other/. Condition 2:
Hei worked by himself/.
Which friends/plan to travel with each other/? Condition 3:
John likes Bill/s portrait of himselfi. Condition 4:
They/told stories about themselves/. Condition 5:
\[John and Mary\]/like each otheri's portraits. Figure 3 Conditions for antecedent NP-lexical anaphor binding. factor, reflecting its relative contribution to the total salience of individual discourse referents. Initial weights are degraded in the course of processing.
The use of salience factors in our algorithm is based on Alshawi's (1987) context mechanism. Other than sentence recency, the factors used in RAP differ from Alshawi's and are more specific to the task of pronominal anaphora resolution. Alshawi's framework is designed to deal with a broad class of language interpretation problems, including reference resolution, word sense disambiguation, and the interpretation of implicit relations. While Alshawi does propose emphasis factors for memory entities that are "referents for noun phrases playing syntactic roles regarded as foregrounding the referent" (Alshawi 1987, p. 17), only topics of sentences in the passive voice and the agents of certain be clauses receive such emphasis in his system. Our emphasis salience factors realize a much more detailed measure of structural salience.
Degradation of salience factors occurs as the first step in processing a new sentence in the text. All salience factors that have been assigned prior to the appearance of this sentence have their weights degraded by a factor of two. When the weight of a given salience factor reaches zero, the factor is removed.
A sentence recency salience factor is created for the current sentence. Its scope is all discourse referents introduced by the current sentence.
The discourse referents evoked by the current sentence are tested to see whether other salience factors should apply. If at least one discourse referent 4satisfies the conditions for a given factor type, a new salience factor of that type is created, with the appropriate discourse referents in its scope.
In addition to sentence recency, the algorithm employs the following salience factors: Subject emphasis Existential emphasis: predicate nominal in an existential construction, as in There are only a few restrictions on LQL query construction for WordSmith. 4 In this paper we do not distinguish between properties of a discourse referent and properties of the NP
that evokes it. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution Table 1 Salience factor types with initial weights Factor type Sentence recency Subject emphasis Existential emphasis Accusative emphasis Indirect object and oblique complement emphasis Head noun emphasis Non-adverbial emphasis Accusative emphasis: direct object (i.e., verbal complement in accusative case) Indirect object and oblique complement emphasis Head noun emphasis: any NP not contained in another NP, using the Slot Grammar notion of "containment within a phrase" (see Section 2.1.1). This factor increases the salience value of an NP that is not embedded within another NP (as its complement or adjunct). Examples of NPs not receiving head noun emphasis are Initial weight 100 80 70 50 40 80 50 the configuration information copied by Backup configuration the assembly in bay C the connector labeled P3 on theflatcable Non-adverbial emphasis: any NP not contained in an adverbial PP demarcated by a separator. Like head noun emphasis, this factor penalizes NPs in certain embedded constructions. Examples of NPs not receiving non-adverbial emphasis are Throughout thefirstsection of thisguide, these symbols are also used ... In the Panel definition panel, select the C~Specify'' action bar. option from the
The initial weights for each of the above factor types are given in Table 1. Note that the relative weighting of some of these factors realizes a hierarchy of grammatical roles. 5 2.1.5 Equivalence Classes. We treat the antecedent-anaphor relation in much the same way as the "equality" condition of Discourse Representation Theory (DRT) (Kamp 1981), as in u---y. This indicates that the discourse referent u, evoked by an anaphoric NP, is anaphorically linked to a previously introduced discourse referent y. To avoid confusion with 5 The specific values of the weights are arbitrary. The significance of the weighting procedure is in the
comparative relations among the factors as defined by the weights. We have determined the efficacy of
this relational structure of salience factors (and refined it) experimentally (see Section 4.2). Computational Linguistics Volume 20, Number 4 mathematical equality (which, unlike the relation discussed here, is symmetric), we represent he relation between an anaphor u and its antecedent y by y antecedes u. Two discourse referents u and y are said to be co-referential, 6written as coref(u~y) if any of the following holds: • y antecedes u • u antecedes y • z antecedes ufor some discourse referent z and coref(z.y) • z antecedes y for some z and coref(z.u) Also, coref(u,u) is true for any discourse referent u. The coref relation defines equivalence classes of discourse referents, with all discourse referents in an "anaphoric hain" forming one class: equiv(u) = {y I coref(u,y)}
Each equivalence class of discourse referents (some of which consist of only one member) has a salience weight associated with it. This weight is the sum of the current weight of all salience factors in whose scope at least one member of the equivalence class lies.
Equivalence classes, along with the sentence recency factor and the salience degradation mechanism, constitute adynamic system for computing the relative attentional prominence of denotational NPs in text. 2.2 The Resolution Procedure RAP's procedure for identifying antecedents of pronouns is as follows. . Create a list of IDs for all NPs in. the current sentence and classify them as to their type (definite NP, pleonastic pronoun, other pronoun, indefinite NP). . Examine all NPs occurring in the current sentence. a. Distinguish among NPs that evoke new discourse referents, those that evoke discourse referents which are presumably coreferential with already listed discourse referents, and NPs that are used non-referentially. Apply salience factors to the discourse referents evoked in the previous step as appropriate. Apply the syntactic filter and reflexive binding algorithm (first phase). b. C. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution (i) (ii) If the current sentence contains any personal or possessive pronouns, a list of pairs of IDs from the current sentence is generated. This list contains the pronoun-NP pairs in the sentence for which coreference can be ruled out on syntactic grounds (using the conditions tated above). If the current sentence contains any lexical anaphors (i.e., reciprocal or reflexive pronouns), a list of ID pairs is generated. Each lexical anaphor is paired with all of its possible antecedent binders. d. If any non-pleonastic pronouns are present in the current sentence, attempt o identify their antecedents. Resolution is attempted in the order of pronoun occurrence in the sentence.
In the case of lexical anaphors (reflexive or reciprocal pronouns), the possible antecedent binders were identified by the anaphor binding algorithm. If more than one candidate was found, the one with the highest salience weight was chosen (see second example of Section 3.1).
In the case of third person pronouns, resolution proceeds as follows: 1. A list of possible antecedent candidates is created. It contains the most recent discourse referent of each equivalence class. The salience weight of each candidate is calculated and included in the list. The salience weight of a candidate can be modified in several ways: a. b. If a candidate follows the pronoun, its salience weight is reduced substantially (i.e., cataphora is strongly penalized). If a candidate fills the same slot as the pronoun, its weight is increased slightly (i.e., parallelism of grammatical roles is rewarded). It is important o note that, unlike the salience factors described in Section 2.1.4, these modifications of the salience weights of candidates are local to the the resolution of a particular pronoun. 2. A salience threshold is applied; only those candidates whose salience weight is above the threshold are considered further. 3. The possible agreement features (number and gender) for the pronoun are determined. The possible sg (singular) and pl (plural) genders are determined; either of these can be a disjunction or nil. Pronominal forms in many languages are ambiguous as to number and gender; such ambiguities are taken into account by RAP's morphological filter and by the algorithm as a whole. The search splits to consider singular and plural antecedents separately (steps 4--6) to allow a general treatment of number ambiguity (as in the Spanish possessive pronoun su or the German pronoun sie occurring as an accusative object). 4. The best sg candidate (if any) is selected: a. b. If no sg genders were determined for the pronoun, proceed to Step 5. Otherwise, apply the morphological filter. Computational Linguistics Volume 20, Number 4 c. d. e. The syntactic filter is applied, using the list of disjoint pronoun-NP pairs generated earlier. The filter excludes any candidate paired in the list with the pronoun being resolved, as well as any candidate that is anaphorically linked to an NP paired with the pronoun. If more than one candidate remains, choose the candidate with the highest salience weight. If several candidates have (exactly) the highest weight, choose the candidate closest to the anaphor. Proximity is measured on the surface string and is not directional. The remaining candidate is considered the best sg candidate. 5. The best pl candidate (if any) is selected. The procedure parallels that
outlined above for the best sg candidate: a. b. c. d. e. If no pl gender is specified for the pronoun, proceed to Step 6. Otherwise, apply the morphological filter. Apply the syntactic filter. If more than one candidate remains, choose the candidate with the highest salience weight; if several candidates have the highest weight, choose the candidate closest to the anaphor. The remaining candidate is considered the best pl candidate. 6. Given the best sg and pl candidates, find the best overall candidate: a. b. If a sg candidate was found, but no pl candidate, or vice versa, choose that candidate as the antecedent. If both a sg and a pl candidate were found, choose the candidate with the greater salience weight (this will never arise in analysis of English text, as all English pronominal forms are unambiguous as to number). 7. The selected candidate is declared to be the antecedent of the pronoun.
The following properties of RAP are worth noting. First, it applies a powerful syntactic and morphological filter to lists of pronoun-NP pairs to reduce the set of possible NP antecedents for each pronoun. Second, NP salience measures are specified largely in terms of syntactic properties and relations (as well as frequency of occurrence). These include a hierarchy of grammatical roles, level of phrasal embedding, and parallelism of grammatical role. Semantic onstraints and real-world knowledge play no role in filtering or salience ranking. Third, proximity of an NP relative to a pronoun is used to select an antecedent in cases in which several candidates have equal salience weighting. Fourth, intrasentential ntecedents are preferred to intersentential candidates. This preference is achieved by three mechanisms: • An additional salience value is assigned to NPs in the current sentence. • The salience values of antecedent candidates in preceding sentences are progressively degraded relative to the salience values of NPs in the current sentence. • Proximity is used to resolve ties among antecedent candidates with equal salience values. The fifth property which we note is that anaphora is strongly preferred to cataphora. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution
RAP generates the list of non-coreferential pronoun-NP pairs for the current sentence, the list of pleonastic pronouns, if any, in the current sentence, the list of possible antecedent NP-lexical anaphor pairs, if any, for the current sentence, and the list of pronoun-antecedent NPpairs that it has identified, for which antecedents may appear in preceding sentences in the text. Each NP appearing in any of the first three lists is represented by its lexical head followed by the integer that corresponds to its position in the sequence of tokens in the input string of the current sentence. The NPs in the pairs of the pronoun-antecedent list are represented by their lexical heads followed by their IDs, displayed as a list of two integers. 7
After installation of the option, the backup copy of the Reference Diskette was started for the computer to automatically configure itself. Antecedent NP--lexical anaphor pairs. computer.18 - itself.22 Anaphor--Antecedent links. itself.(1.22) to computer.(1.18) John talked to Bill about himself. Antecedent NP--lexical anaphor pairs. John.1 - himself.6, Bill.4 - himself.6 Anaphor--Antecedent links. himself.(l.6) to John.(1.1) In the second example, \]ohn.(1.1) was preferred to 8i11.(1.4) owing to its higher salience weight.
Most of the copyright notices are embedded in the EXEC, but this keyword makes it possible for a user-supplied function to have its own copyright notice. Non-coreferential pronoun--NP pairs. it.16 most.l, it.16 it.16 function.23, it.16 copyright.28, its.26 - notice.29, -
- notice.5, it.16 - keyword.14,
it.16 - user.20, it.16 - notice.29,
its.26 - most.l, its.26 - notice.5, its.26 - copyright.28 7 Recall that the first integer identifies the sentence inwhich the NP appears, and the second indicates
the position of its head word in the sentence. Computational Linguistics Volume 20, Number 4 Pleonastic Pronouns. it. 16 Anaphor--Antecedent links. its.(l.26) to function. (1.23) function.(l.23) and keyword.(l.14) share the highest salience weight of all candidates that pass the morphological and syntactic filters; they are both subjects and therefore higher in salience than the third candidate, £XEC.(1.10). function.(1.23) is then selected as the antecedent owing to its proximity to the anaphor.
Because of this, MicroEMACS cannot process an incoming ESC until it knows what character follows it. Non-coreferential pronoun--NP pairs. it.12 - character.15, it.l? - character.15 Anaphor--Antecedent links. it.(l.12) it.(l.iT) to MicroEMACS.(l.4) to ESC.(I.10) MicroEMACS.(1.4) is preferred over ESC.(1.10) as an antecedent of it.(1.12)-MicroEMACS.(1.4) receives subject emphasis versus the lower object emphasis of ESC.(1.10). In addition, MicroEMACS.(1.4) is rewarded because it fills the same grammatical role as the anaphor being resolved.
In the case of it.(1.17), the parallelism reward works in favor of ESC.(1.10), causing it to be chosen, despite the general preference for subjects over objects.
At this point, emacs is waiting for a command. It is prepared to see if the variable keys are TRUE, and executes some lines if they are. Non-coreferential pronoun--NP pairs. it.1 - key. O, it.1 - line.16, it.1 - they.18, they.18 - it.1 Anaphor--Antecedent links. it.(2.1) they.(2.18) to emacs.(1.5)
to key.(2.0)
Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution discourse referents from earlier sentences that are not members of anaphoric chains extending into the current sentence rapidly become "uncompetitive." Salience weight Discourse referent(s) 465 emacs.(i.5) s(it,l).(2.1) 310 s(key,l).(2.9) s(they,l).(2.18) 280 s(line,l).(2.16) 135 s(command,2).(l.lO) 90 s(point,4).(l.3)
You have not waited for the file to close. You may have asked to print on the virtual printer, but it cannot print until the output file is closed. Non-coreferential pronoun--NP pairs: you.1 you.l printer.lO, you.l - it.13, you.l - output.19, file.20, it.13 - you.l, it.13 - output.19, it.13 file.20 Salience values:
printer.(2.10)
file.(1.7) - 190 - 270 Salience factor values:
printer.(2.10)
sentence_rec - i00
non_adverbial_emph
pobj_emph - 40
head_emph - 80 - 50 file.(l.7) sentence_rec - 50 non_adverbial_emph - 25 subj_emph - 40 head_emph - 40 Local salience factor values:
file. (I.7)
parallel_roles_reward - 35 Anaphor--Antecedent links: it.(2.13) to printer.(2.10)
This example illustrates the strong preference for intrasentential ntecedents, printer.(2.10) is selected, despite the fact that it is much lower on the hierarchy of grammatical roles than the other candidate, file.(1.7), which also benefits from the parallelism reward. Degradation of salience weight for the candidate from the previous sentence is substantial enough to offset these factors. The PARTNUM tag prints a part number on the document. &name.'s initial setting places it on the back cover.
Computational Linguistics Volume 20, Number 4 Non-coreferential pronoun--NP pairs: it.6 - setting.4, it.6 - cover.lO Salience values: number.(1.7) - 175 tag.(1.3) - 155 scsym(name).(2.1) document.(1.10) PARTNUM.(1.2) - 150 - 135 Salience factor values: number.(l.Z)
sentence_rec - 50
non_adverbial_emph
acc_emph - 25
head_emph - 40 tag.(l.3).
sentence_rec - 50
non_adverbial_emph - 25
subj_emph - 40
head_emph - 40 scsym(name).(2.1)
sentence_rec - I00
non_adverbial_emph PAKTNUM.(I.2)
sentence_rec - 50
non_adverbial_emph document.(l.lO)
sentence_rec - 50
non_adverbial_emph
pobj_emph - 20
head_emph - 40 - 25 Local salience factor values: number.(l,7) parallel_roles_reward - 35 Anaphor--Antecedent links: s(it,l).(2.6) to s(number,l).(l.7)
Four candidates receive a similar salience weighting in this example. Two potential intrasentential candidates that would have received a high salience ranking, setting.(2.4) and cover.(2.10), are ruled out by the syntactic filter. The remaining intrasentential candidate, scsym(name).(2.1) 8 ranks relatively low, as it is a possessive determiner--it scores lower than two candidates from the previous sentence. The parallelism reward causes number.Off ) to be preferred. 4. Testing of RAP on Manual Texts We tuned RAP on a corpus of five computer manuals containing a total of approximately 82,000 words. From this corpus we extracted sentences with 560 occurrences 8 &name. is a document formatting symbol: it is replaced by a predefined character string when the text
is formatted. ESG treats such symbols as being unspecified for number and gender; number may be
assigned uring parsing, owing to agreement constraints. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution Table 2 Results of training phase Total Intersentential cases Intrasentential cases Number of pronoun occurrences 560 Number of cases that the algorithm 475 (85%) resolves correctly 89 72 (81%) 471 403 (86%) of third person pronouns (including reflexives and reciprocals) and their antecedents. 9
In the training phase, we refined our tests for pleonastic pronouns and experimented extensively with salience weighting. Our goal was, of course, to optimize RAP's success rate with the training corpus. We proceeded heuristically, analyzing cases of failure and attempting to eliminate them in as general a manner as possible. The parallelism reward was introduced at this time, as it seemed to make a substantial contribution to the overall success rate. A salience factor that was originally present, viz. matrix emphasis, was revised to become the non-adverbial emphasis factor. In its original form, this factor contributed to the salience of any NP not contained in a subordinate clause or in an adverbial PP demarcated by a separator. This was found to be too general, especially since the relative positions of a given pronoun and its antecedent candidates are not taken into account. The revised factor could be thought of as an adverbial penalty factor, since it in effect penalizes NPS occurring in adverbial pps. 1°
We also experimented with the initial weights for the various factors and with the size of the parallelism reward and cataphora penalty, again attempting to optimize RAP's overall success rate. A value of 35 was chosen for the parallelism reward; this is just large enough to offset the preference for subjects over accusative objects. A much larger value (175) was found to be necessary for the cataphora penalty. The final results that we obtained for the training corpus are given in Table 2.
Interestingly, the syntactic-morphological fi ter reduces the set of possible antecedents to a single NP, or identifies the pronoun as pleonastic in 163 of the 475 cases (34%) that the algorithm resolves correctly. 11 It significantly restricts the size of the candidate list in most of the other cases, in which the antecedent isselected on the basis of salience ranking and proximity. This indicates the importance of a powerful syntactic-morphological filtering component in an anaphora resolution system.
We then performed a blind test of RAP on a test set of 345 sentences randomly selected from a corpus of 48 computer manuals containing 1.25 million words. 12 The results which we obtained for the test corpus (without any further modifications of RAP) are given in Table 3.13
This blind test provides the basis for a comparative valuation of RAP and Dagan's • 9 These sentences and those used in the blind test were edited slightly to overcome parse inaccuracies.
Rather than revise the lexicon, we made lexical substitutions to improve parses. In some cases
constructions had to be simplified. However, such changes did not alter the syntactic relations among
the pronoun and its possible antecedents.
For a discussion of ESG's parsing accuracy, see McCord (1993). 10 See comments at the end of Section 4 about refining RAP's measures of structural salience. 11 Forty-three of the pronoun occurrences in the training corpus (~ 8%) were pleonastic; arandom sample
of 245 pronoun occurrences extracted from our test corpus included 15 pleonastic pronouns (~ 6%). 12 The test set was filtered in order to satisfy the conditions of our experiments on the role of statistically
measured lexical preference in enhancing RAP's performance. See Section 5.1 for a discussion of these Computational Linguistics Volume 20, Number 4 Table 3 Results of blind test Total Intersentential cases Intrasentential cases Number of pronoun occurrences 360 70 Number of cases that the algorithm 310 (86%) 52 (74%) resolves correctly 290 258 (89%) (1992) system, RAPSTAT, which employs both RAP's salience weighting mechanism and statistically measured lexical preferences, as well as for a detailed analysis of the relative contributions of the various elements of RAP's salience weighting mechanism to its overall success rate. We will discuss the blind test in greater detail in the following sections. 4.1 Limitations of the Current Algorithm Several classes of errors that RAP makes are worthy of discussion. The first occurs with many cases of intersentential naphora, such as the following: This green indicator is lit when the controller is on. It shows that the DC power supply voltages are at the correct levels. Morphological and syntactic filtering exclude all possible intrasentential candidates. Because the level of sentential embedding does not contribute to RAP's salience weighting mechanism, indicator.(1.3) and controller.(1.8) are ranked equally, since both are subjects. RAP then erroneously chooses controller.(1.8) as the antecedent, since it is closer to the pronoun than the other candidate.
The next class of errors involves antecedents that receive a low salience weighting owing to the fact that the evoking NP is embedded in a matrix NP or is in another structurally nonprominent position (such as object of an adverbial PP). The users you enroll may not necessarily be new to the system and may already have a user profile and a system distribution directory entry. &ofc. checks for the existence of these objects and only creates them as necessary. Despite the general preference for intrasentential candidates, user.(1.2) is selected as the antecedent, since the only factor contributing to the salience weight of object.(2.8) is sentence recency. Selectional restrictions or statistically measured lexical preferences (see Section 5) could clearly help in at least some of these cases.
In another class of cases, RAP fails because semantic/pragmatic nformation is required to identify the correct antecedent.
conditions. 13 Proper resolution was determined by a consensus of three opinions, including that of the first author. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution Table 4 Relative contribution of elements of salience weighting mechanism I II III IV V VI VII VIII IX X Total correct 310 (86%) 308 (86%) 308 (86%) 302 (84%) 301 (84%) 297 (83%) 294 (82%) 231 (64%) 212 (59%) 184 (51%) Correctly disagrees with RAP Incorrectly disagrees with RAP 4 4 11 9 25 17 79 119 143 2 2 3 12 1 21 17 Again, the Migration Aid produces an exception report automatically at the end of every migration run. As you did with the function, use it to verify that the items have been restored to your system successfully. function.(2.6) is selected as the antecedent, rather than aid.(1.5). 4.2 The Relative Contributions of the Salience Weighting Mechanisms Using the test corpus of our blind test, we conducted experiments with modified versions of RAP, in which various elements of the salience weighting mechanism were switched off. We present he results in Table 4 and discuss their significance.
Ten variants are presented in Table 4; they are as follows: I II III IV V VI VII VIII IX X
The single most important element of the salience weighting mechanism is the recency preference (sentence recency factor and salience degradation; see VIII). This is not surprising, given the relative scarcity of intersentential naphora in our test corpus (less than 20% of the pronoun occurrences had antecedents inthe preceding sentence). Deactivating the equivalence class mechanism also led to a significant deterioration in RAP's performance; in this variant (VII), only the salience factors applying to a "standard" RAP (as used in the blind test) parallelism reward deactivated non-adverbial and head emphasis deactivated matrix emphasis used instead of non-adverbial emphasis cataphora penalty deactivated subject, existential, accusative, and indirect object/oblique complement emphasis (i.e., hierarchy of grammatical roles) deactivated equivalence classes deactivated sentence recency and salience degradation deactivated all "structural" salience weighting deactivated (II +III + V + VI) all salience weighting and degradation deactivated Computational Linguistics Volume 20, Number 4 particular NP contribute to its salience weight, without any contribution from other anaphorically linked NPs. The performance of the syntactic filter is degraded somewhat in this variant as well, since NPs that are anaphorically linked to an NP fulfilling the criteria for disjoint reference will no longer be rejected as antecedent candidates. The results for VII and VIII indicate that attentional state plays a significant role in pronominal anaphora resolution and that even a simple model of attentional state can be quite effective.
Deactivating the syntax-based elements of the salience weighting mechanism individually led to relatively small deteriorations in the overall success rate (II, III, IV, V, and VI). Eliminating the hierarchy of grammatical roles (VI), for example, led to a deterioration of less than 4%. Despite the comparatively small degradation in performance that resulted from turning off these elements individually, their combined effect is quite significant, as the results of IX show. This suggests that the syntactic salience factors operate in a complex and highly interdependent manner for anaphora resolution.
X relies solely on syntactic/morphological filtering and proximity to choose an antecedent. Note that the sentence pairs of the blind test set were selected so that, for each pronoun occurrence, at least two antecedent candidates remained after syntactic/morphological filtering (see Section 5.1). In the 17 cases in which X correctly disagreed with RAP, the proper antecedent happened to be the most proximate candidate.
We suspect that RAP's overall success rate can be improved (perhaps by 5% or more) by refining its measures of structural salience. Other measures of embeddedness, or perhaps of "distance" between anaphor and candidate measured in terms of clausal and NP boundaries, may be more effective than the current mechanisms for nonadverbial and head emphasis. 14 Empirical studies of patterns of pronominal anaphora in corpora (ideally in accurately and uniformly parsed corpora) could be helpful in defining the most effective measures of structural salience. One might use such studies to obtain statistical data for determining the reliability of each proposed measure as a predictor of the antecedent-anaphor relation and the orthogonality (independence) of all proposed measures. 5. Salience and Statistically Measured Lexical Preference Dagan (1992) constructs aprocedure, which he refers to as RAPSTAT, for using statistically measured lexical preference patterns to reevaluate RAP's salience rankings of antecedent candidates. RAPSTAT assigns a statistical score to each element of a candidate list that RAP generates; this score is intended to provide a measure (relative to a corpus) of the preference that lexical semantic/pragmatic fa tors impose upon the candidate as a possible antecedent for a given pronoun, is Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution
RAPSTAT reevaluates RAP's ranking of the elements of the antecedent candidate list L in a way that combines both the statistical scores and the salience values of the candidates. The elements of L appear in descending order of salience value. RAPSTAT processes Las follows. Initially, it considers the first two elements Cl and C2 of L. If (i) the difference in salience scores between C1 and C2 does not exceed a parametrically specified value (the salience difference threshold) and (ii) the statistical score of C2 is significantly greater than that of C1, then RAPSTAT will substitute the former for the latter as the currently preferred candidate. If conditions (i) and (ii) do not hold, RAPSTAT confirms RAP's selection of C1 as the preferred antecedent. If these conditions do hold, then RAPSTAT selects C2 as the currently preferred candidate and proceeds to compare it with the next element of L. It repeats this procedure for each successive pair of candidates in L until either (i) or (ii) fails or the list is completed. In either case, the last currently preferred candidate is selected as the antecedent.
An example of a case in which RAPSTAT overules RAP is the following. The Send Message display is shown, allowing you to enter your message and specify where it will be sent. The two top candidates in the list that RAP generates for it.(1.17) are display.(1.4) with a salience value of 345 and message.(1.13), which has a salience value of 315. In the corpus that we used for testing RAPSTAT, the verb-object pair send-display appears only once, whereas send-message occurs 289 times. As a result, message receives a considerably higher statistical score than display. The salience difference threshold that we used for the test is 100, and conditions (i) and (ii) hold for these two candidates. The difference between the salience value of message and the third element of the candidate list is greater than 100. Therefore, RAPSTAT correctly selects message as the antecedent of it. 5.1 A Blind Test of RAP and RAPSTAT Dagan et al. (in press) report a comparative blind test of RAP and RAPSTAT. To construct a database of grammatical relation counts for RAPSTAT, we applied the Slot Grammar parser to a corpus of 1.25 million words of text from 48 computer manuals. We automatically extracted all lexical tuples and recorded their frequencies in the parsed corpus. We then constructed atest set of pronouns by randomly selecting from the corpus sentences containing at least one non-pleonastic third person pronoun occurrence. For each such sentence in the set, we included the sentence that immediately precedes it in the text (when the preceding sentence does not contain a pronoun). 16 We filtered the test set so that for each pronoun occurrence in the set, (i) RAP generates a candidate list with at least two elements, (ii) the actual antecedent NP appears in the candidate list, and (iii) there is a total tuple frequency greater than 1 for the candidate
See Dagan 1992 and Dagan et al. (in press) for a discussion of this lexical statistical approach to
ranking antecedent candidates and possible alternatives. 16 In the interests of simplicity and uniformity, we discarded sentence pairs in which the first sentence
contains a pronoun. We decided to limit the text preceding the sentence containing the pronoun to one
sentence because we found that in the manuals which we used to tune the algorithm, almost all cases
of intersentential anaphora involved an antecedent inthe immediately preceding sentence. Moreover,
the progressive decline in the salience values of antecedent candidates inprevious entences ensures
that a candidate appearing in a sentence which is more than one sentence prior to the current one will
be selected only if no candidates exist in either the current or the preceding sentence. As such cases are
relatively rare in the type of text we studied, we limited our test set to textual units containing the
current and the preceding sentence. Computational Linguistics Volume 20, Number 4 list (in most cases, it was considerably larger). ~7 The test set contains 345 sentence pairs with a total of 360 pronoun occurrences. The results of the blind test for RAP and RAPSTAT are as follows. TM
RAP Total correct: 310 (86%) Total decided: 360 (100%) Correctly decided: 310 (86%) RAPSTAT Disagrees with RAP: Correctly disagrees with RAP: Incorrectly disagrees with RAP: RAP/RAPSTAT Both wrong: Either RAP or RAPSTAT is correct:
When we further analyzed the results of the blind test, we found that RAPSTAT's success depends in large part on its use of salience information. If RAPSTAT's statistically based lexical preference scores are used as the only criterion for selecting an antecedent, he statistical selection procedure disagrees with RAP in 151 out of 338 instances. RAP is correct in 120 (79%) of these cases and the statistical decision in 31 (21%) of the cases. When salience is factored into RAPSTAT's decision procedure, the rate of disagreement between RAP and RAPSTAT declines sharply, and RAPSTAT's performance slightly surpasses that of RAP, yielding the results that we obtained in the blind test.
In general, RAPSTAT is a conservative statistical extension of RAP. It permits statistically measured lexical preference to overturn salience-based decisions only in cases in which the difference between the salience values of two candidates is small and the statistical preference for the less salient candidate is comparatively large. ~9 The comparative blind test indicates that incorporating statistical information on lexical preference patterns into a salience-based anaphora resolution procedure can yield a modest improvement in performance relative to a system that relies only on syntactic salience for antecedent selection. Our analysis of these results also shows that statistically measured lexical preference patterns alone provide a far less efficient basis for anaphora resolution than an algorithm based on syntactic and attentional measures of salience. 2°
We will briefly compare our algorithm with several other approaches to anaphora resolution that have been suggested. RAPSTAT 319 (89%) 182 (51%) 144 (79%) 41 (22% of cases decided) 25 (61%) 16 (39%) 22 (12%) 335 (93%) Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution 6.1 Hobbs' Algorithm Hobbs' (1978) algorithm relies on a simple tree search procedure formulated in terms of depth of embedding and left-right order. By contrast, RAP uses a multi-dimensional measure of salience that invokes a variety of syntactic properties pecified in terms of the head-argument s ructures of Slot Grammar, as well as a model of attentional state.
Hobbs' tree search procedure selects the first candidate encountered by a leftright depth first search of the tree outside of a minimal path to the pronoun that satisfies certain configurational constraints. The algorithm chooses as the antecedent of a pronoun P the first NPi in the tree obtained by left-to-right breadth-first traversal of the branches to the left of the path T such that (i) T is the path from the NP dominating Pto the first NP or S dominating this NP, (ii) T contains an NP or S node N that contains the NP dominating P, and (iii) N does not contain NPi. If an antecedent satisfying this condition is not found in the sentence containing P, the algorithm selects the first NP obtained by a left-to-right breadth first search of the surface structures of preceding sentences in the text.
We have implemented a version of Hobbs' algorithm for Slot Grammar. The original formulation of the algorithm encodes yntactic onstraints on pronominal anaphora in the definition of the domain to which the search for an antecedent NP applies. In our implementation ofthe algorithm, we have factored out the search procedure and substituted RAP's syntactic-morphological filter for Hobbs' procedural filter. Let the Mods (modifiers) of a head H be the sisters of H in the Slot Grammar epresentation of the phrase that H heads. Our specification of Hobbs' algorithm for Slot Grammar is as follows: . Find a node N1 such that (i) N1 contains the pronoun P; (ii) N1 is an S or NP; and (iii) it is not the case that there is a node N1, such that N1 contains N1, and N1, satisfies (i) and (ii). . Check the list of Mods of N1 left to right for NPs that are not elements of the list of pairs <P-NP> identified by the syntactic-morphological filter as noncoreferential nd that occur to the left of R ° Select the leftmost NP in the filtered list of NP Mods of N1. 4. If this list is nil, then repeat steps 2 and 3 recursively for each Mod in the list of Mods of N1, each Mod in this second list of Mods, etc., until an NP antecedent isfound. . If no NP antecedent isfound by applying step 4, then identify a node N2 that is the first NP/S containing N1. . If N2 is an NP and is not an element of the list of pairs <P-NP> identified by the filter, propose it as the antecedent. , Otherwise, apply steps 2-4 to N2.
If no antecedent NP is found, continue to apply steps 5 and 6 and then steps 2-4 to progressively higher NP/S nodes. . If no antecedent NPs are found at the highest S of the sentence, then take N1 to be the highest S node of the immediately preceding sentence and apply steps 2-4 to N1. Computational Linguistics Table 5 Results of blind test (Hobbs' algorithm) Total Number of pronoun occurrences 360 Number of cases that the algorithm resolves correctly 295 (82%) Number of cases for which HOBBS correctly disagrees with RAP 22 Number of cases for which HOBBS incorrectly disagrees with RAP 38
We ran this version of Hobbs' algorithm on the test set that we used for the blind test of RAP and RAPSTAT; the results appear in Table 5.
It is important to note that the test set does not include pleonastic pronouns or lexical anaphors (reflexive or reciprocal pronouns), neither of which are dealt with by Hobbs' algorithm. Moreover, our Slot Grammar implementation of the algorithm gives it the full advantage of RAP's syntactic-morphological filter, which is more powerful than the configurational filter built into the original specification of the algorithm. Therefore, the test results provide a direct comparison of RAP's salience metric and Hobbs' search procedure.
Hobbs' algorithm was more successful than RAP in resolving intersentential naphora (87% versus 74% correct). 21 Because intersentential naphora is relatively rare in our corpus of computer manual texts and because RAP's success rate for intrasentential anaphora is higher than Hobbs' (89% versus 81%), RAP's overall success rate on the blind test set is 4% higher than that of our version of Hobbs' algorithm. This indicates that RAP's salience metric provides a more reliable basis for antecedent selection than Hobbs' search procedure for the text domain on which we tested both algorithms.
It is clear from the relatively high rate of agreement between RAP and Hobbs' algorithm on the test set (they agree in 83% of the cases) that there is a significant degree of convergence between salience as measured by RAP and the configurational prominence defined by Hobbs' search procedure. This is to be expected in English, in which grammatical roles are identified by means of phrase order. However, in languages in which grammatical roles are case marked and word order is relatively free, we expect that there will be greater divergence in the predictions of the two algorithms. The salience measures used by RAP have application to a wider class of languages than Hobbs' order-based search procedure. This procedure relies on a correspondence ofgrammatical roles and linear precedence relations that holds for a comparatively small class of languages. 6.2 Discourse Based Methods Most of the work in this area seeks to formulate general principles of discourse structure and interpretation and to integrate methods of anaphora resolution into a computational model of discourse interpretation (and sometimes of generation as well). Sidner (1981, 1983), Grosz, Joshi, and Weinstein (1983, 1986), Grosz and Sidner (1986), 21 The difficulty that RAP encounters with such cases was discussed in Section 4.1. We are experimenting
with refinements inRAP's scoring mechanism to improve its performance inthese and other cases. Volume 20, Number 4 Intersentential cases Intrasentential cases 70 290 61 (87%) 234 (81%) 9 4 13 34 Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution Brennan, Friedman, and Pollard (1987), and Webber (1988) present different versions of this approach. Dynamic properties of discourse, especially coherence and focusing, are invoked as the primary basis for identifying antecedence candidates; selecting a candidate as the antecedent of a pronoun in discourse involves additional constraints of a syntactic, semantic, and pragmatic nature.
In developing our algorithm, we have not attempted to consider elements of discourse structure beyond the simple model of attentional state realized by equivalence classes of discourse referents, salience degradation, and the sentence recency salience factor. The results of our experiments with computer manual texts (see Section 4.2) indicate that, at least for certain text domains, relatively simple models of discourse structure can be quite useful in pronominal anaphora resolution. We suspect hat many aspects of discourse models discussed in the literature will remain computationally intractable for quite some time, at least for broad-coverage systems.
A more extensive treatment of discourse structure would no doubt improve the performance of a structurally based algorithm such as RAP. At the very least, formatting information concerning paragraph and section boundaries, list elements, etc., should be taken into account. A treatment of definite NP resolution would also presumably lead to more accurate resolution of pronominal anaphora, since it would improve the reliability of the salience weighting mechanism.
However, some current discourse-based approaches to anaphora resolution assign too dominant a role to coherence and focus in antecedent selection. As a result, they establish a strong preference for intersentential over intrasentential naphora resolution. This is the case with the anaphora resolution algorithm described by Brennan, Friedman, and Pollard (1987). This algorithm is based on the centering approach to modeling attentional structure in discourse (Grosz, Joshi, and Weinstein 1983, 1986). 22 Constraints and rules for centering are applied by the algorithm as part of the selection procedure for identifying the antecedents of pronouns in a discourse. The algorithm strongly prefers intersentential ntecedents hat preserve the center or maximize continuity in center change, to intrasentential ntecedents hat cause radical center shifts. This strong preference for intersentential ntecedents i inappropriate for at least some text domains--in our corpus of computer manual texts, for example, we estimate that less than 20% of referentially used third person pronouns have intersentential antecedents. 23
There is a second difficulty with the Brennan et al. centering algorithm. It uses a hierarchy of grammatical roles quite similar to that of RAP, but this role hierarchy does not directly influence antecedent selection. Whereas th e hierarchy in RAP contributes to a multi-dimensional measure of the relative salience of all antecedent candidates, in Brennan et al. 1987, it is used only to constrain the choice of the backward-looking center, Cb, of an utterance. It does not serve as a general preference measure for antecedence. The items in the forward center list, Cf, are ranked according to the hierarchy of grammatical roles. For an utterance U,, Cb(Un) is required to be the highest ranked element of Cf(U~_I) that is realized in U,. If an element E in the list of possible Computational Linguistics Volume 20, Number 4 forward Cf(Un-1), centers, is identified as the antecedent of a pronoun in Un, then E is realized in Un. The Brennan et al. centering algorithm does not require that the highest ranked element of Cf(Un-1) actually be realized in Un, but only that Cb(Un) be the highest ranked element of Cf(Un-1) which is, in fact, realized in Un. Antecedent selection is constrained by rules that sustain cohesion in the relations between the backward centers of successive utterances in a discourse, but it is not determined irectly by the role hierarchy used to rank the forward centers of a previous utterance. Therefore, an NP in Un_~ that is relatively low in the hierarchy of grammatical roles can serve as an antecedent of a pronoun in Un, provided that no higher ranked NP in Un-1 is taken as the antecedent of some other pronoun or definite NP in Un.24 An example will serve to illustrate the problem with this approach. The display shows you the status of all the printers. It also provides options that control printers. The (ranked) forward center list for the first sentence is as follows: (\[DISPLAY\] \[STATUS\] \[YOU\] \[PRINTERS3). Applying the filters and ranking mechanism of Brennan, Friedman, and Pollard (1987) yields two possible anchors. 2s Each anchor determines a choice of Cb(Un) and the antecedent of it. One anchor identifies both with display, whereas the second takes both to be status. The hierarchy of grammatical roles is not used to select display over status. Nothing in the algorithm rules out the choice of status as the backward center for the second sentence and as the antecedent of it. If this selection is made, display is not realized in the second sentence, and so Cb(Un) is status, which is then the highest ranked element of Cf(Un-1) that is realized in Un, as required by constraint 3 of the Brennan et al. centering algorithm.
In general, we agree with Alshawi (1987, p. 62) that an algorithm/model relying on the relative salience of all entities evoked by a text, with a mechanism for removing or filtering entities whose salience falls below a threshold, is preferable to models that "make assumptions about a single (if shifting) focus of attention. "26 6.3 Mixed Models This approach seeks to combine a variety of syntactic, semantic, and discourse factors into a multi-dimensional metric for ranking antecedent candidates. On this view, the score of a candidate is a composite of several distinct scoring procedures, each of which reflects the prominence of the candidate with respect to a specific type of information or property. The systems described by Asher and Wada (1988), Carbonell and Brown (1988), and Rich and LuperFoy (1988) are examples of this mixed evaluation strategy.
In general, these systems use composite scoring procedures that assign a global rank to an antecedent candidate on the basis of the scores that it receives from several 24 Other factors, such as level of embedding, may also be considered in generating an ordering for the list
of forward-looking centers. Walker, Iida, and Cote (1990) discuss ordering conditions appropriate for
Japanese. 25 An anchor is an association between a backward-looking center, Cb, and a list of forward-looking
centers, Cf, for an utterance. An anchor establishes a link between a pronoun and its antecedent by
associating the reference marker of the antecedent with that of the pronoun in the Cf list of the
utterance. 26 See Walker (1989) for a comparison of the algorithm of Brennan, Friedman, and Pollard (1987) with
that of Hobbs (1978) based on a hand simulation. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution evaluation metrics. Each such metric scores the likelihood of the candidate relative to a distinct informational factor. Thus, for example, Rich and LuperFoy (1988) propose a system that computes the global preference value of a candidate from the scores provided by a set of constraint source modules, in which each module invokes different sorts of conditions for ranking the antecedent candidate. The set of modules includes (among others) syntactic and morphological filters for checking agreement and syntactic onditions on disjoint reference, aprocedure for applying semantic selection restrictions to a verb and its arguments, a component that uses contextual and real-world knowledge, and modules that represent both the local and global focus of discourse. The global ranking of an antecedent candidate is a function of the scores that it receives from each of the constraint source modules.
Our algorithm also uses a mixed evaluation strategy. We have taken inspiration from the discussions of scoring procedures in the works cited above, but we have avoided constraint sources involving complex inferencing mechanisms and real-world knowledge, typically required for evaluating the semantic/pragmatic suitability of antecedent candidates or for determining details of discourse structure. In general, it seems to us that reliable large scale modelling of real-world and contextual factors is beyond the capabilities of current computational systems. Even constructing acomprehensive, computationally viable system of semantic selection restrictions and an associated type hierarchy for a natural language is an exceedingly difficult problem, which, to our knowledge, has yet to be solved. Moreover, our experiments with statistically based lexical preference information casts doubt on the efficacy of relatively inexpensive (and superficial) methods for capturing semantic and pragmatic factors for purposes of anaphora resolution. Our results suggest hat scoring procedures which rely primarily on tractable syntactic and attentional (recency) properties can yield a broad coverage anaphora resolution system that achieves agood level of performance. 7. Conclusion We have designed and implemented an algorithm for pronominal anaphora resolution that employs measures of discourse salience derived from syntactic structure and a simple dynamic model of attentional state. We have performed a blind test of this algorithm on a substantial set of cases taken from a corpus of computer manual text and found it to provide good coverage for this set. It scored higher than a version of Hobbs' algorithm that we implemented for Slot Grammar.
Results of experiments with the test corpus show that the syntax-based elements of our salience weighting mechanism contribute in a complexly interdependent way to the overall effectiveness ofthe algorithm. The results also support the view that attentional state plays a significant role in pronominal anaphora resolution and demonstrate that even a simple model of attentional state can be quite effective.
The addition of statistically measured lexical preferences to the range of factors that the algorithm considers only marginally improved its performance on the blind test set. Analysis of the results indicates that lexical preference information can be useful in cases in which the syntactic salience ranking does not provide a clear decision among the top candidates, and there is a strong lexical preference for one of the less salient candidates.
The relatively high success rate of the algorithm suggests the viability of a computational model of anaphora resolution in which the relative salience of an NP in discourse is determined, in large part, by structural factors. In this model, semantic and real-world knowledge conditions apply to the output of an algorithm that resolves pronominal anaphora on the basis of syntactic measures of salience, recency, Computational Linguistics Volume 20, Number 4 and frequency of mention. These conditions are invoked only in cases in which salience does not provide a clear-cut decision and/or there is substantial semantic-pragmatic support for one of the less salient candidates. 27 Acknowledgments We would like to thank Martin Chodorow, Ido Dagan, John Justeson, Slava Katz, Michael McCord, Hubert Lehman, Amnon Ribak, Ulrike Schwall, and Marilyn Walker for helpful discussion of many of the ideas and proposals presented here. The blind test and evaluation of RAPSTAT reported here was done jointly with Ido Dagan, John Justeson, and Amnon Ribak. An early version of this paper was presented at the Cognitive Science Colloquium of the University of Pennsylvania, in January 1992, and we are grateful to the participants of the colloquium for their reactions and suggestions. We are also grateful to several anonymous reviewers of Computational Linguistics for helpful comments on earlier drafts of the paper. References Alshawi, Hiyan (1987). Memory and Context
for Language Interpretation. Cambridge:
Cambridge University Press. Asher, Nicholas, and Wada, Hajime (1988).
"A computational ccount of syntactic,
semantic and discourse principles for
anaphora resolution." Journal of Semantics
6:309-344. Bosch, Peter (1988). "Some good reasons for
shallow pronoun processing." In
Proceedings, IBM Conference on Natural
Language Processing. New York:
Thornwood. Brennan, Susan; Friedman, Marilyn; and
Pollard, Carl (1987). "A centering
approach to pronouns." In Proceedings,
25th Annual Meeting of the Association for
Computational Linguistics, 155-162. Carbonell, Jaime, and Brown, Ralf (1988).
"Anaphora resolution: A multi-strategy
approach." In Proceedings, 12th
International Conference on Computational
Linguistics, 96-101. Dagan, Ido (1992). "Multilingual statistical
approaches for natural language
disambiguation" (in Hebrew). Doctoral
dissertation, Israel Institute of Technology,
Haifa, Israel. Dagan, Ido; Justeson, John; Lappin, Shalom;
Leass, Herbert; and Ribak, Amnon (in 27 Bosch (1988) suggests apsychological processing model in which hearers rely on first pass syntactically
based strategies for initial linking of pronouns to antecedent NPs.
press). "Syntax and lexical statistics in
anaphora resolution." Applied Artificial
Intelligence. Grosz, Barbara; Joshi, Aravind; and
Weinstein, Scott (1983). "Providing a
unified account of definite noun phrases
in discourse." In Proceedings, 21st Annual
Meeting of the Association of Computational
Linguistics, 44-50. Grosz, Barbara; Joshi, Aravind; and
Weinstein, Scott (1986, unpublished).
"Towards a computational theory of
discourse interpretation." Harvard
University and University of
Pennsylvania. Grosz, Barbara, and Sidner, Candice (1986).
"Attention, intentions, and the structure
of discourse." Computational Linguistics
12:175-204. Guenthner, Franz, and Lehmann, Hubert
(1983). "Rules for pronominalization." In
Proceedings, First Annual Meeting of the
European Chapter of the ACL, 144-151. Hobbs, Jerry (1978). "Resolving pronoun
references." Lingua 44:311-338. Johnson, David (1977). "On relational
constraints on grammars." In Syntax and
Semantics 8, edited by P. Cole and
J. Sadock, 151-178. New York: Academic
Press. Kamp, Hans (1981). "A theory of truth and
semantic representation." InFormal
Methods in the Study of Language, edited by
J. Groenendijk, T. Janssen, and
M. Stokhof. Amsterdam: Mathematisch
Centrum Tracts. Keenan, Edward, and Comrie, Bernard
(1977). "Noun phrase accessibility and
universal grammar." Linguistic Inquiry
8:62-100. Lappin, Shalom (1985). "Pronominal
binding and coreference." Theoretical
Linguistics 12:241-263. Lappin, Shalom, and McCord, Michael
(1990a). "A syntactic filter on pronominal
anaphora in slot grammar." In Proceedings,
28th Annual Meeting of the Association for
Computational Linguistics, 135-142. Lappin, Shalom, and McCord, Michael
(1990b). "Anaphora resolution in slot
grammar." Computational Linguistics
16:197-212. Shalom Lappin and Herbert J. Leass An Algorithm for Pronominal Anaphora Resolution Leass, Herbert, and Schwall, Ulrike (1991).
"An anaphora resolution procedure for
machine translation." IWBS Report 172,
IBM Germany Scientific Center,
Heidelberg, Germany. McCord, Michael (1989a). "Design of LMT:
A prolog-based machine translation
system." Computational Linguistics
15:33-52. McCord, Michael (1989b). "A new version
of the machine translation system LMT."
Literary and Linguistic Computing 4:218-229. McCord, Michael (1990). "Slot grammar: A
system for simpler construction of
practical natural language grammars." In
Natural Language and Logic: International
Scientific Symposium, edited by R. Studer,
118-145. Lecture Notes in Computer
Science, Berlin: Springer Verlag. McCord, Michael (1993). "Heuristics for
broad-coverage natural language
parsing." In Proceedings, ARPA Human
Language Technology Workshop, University
of Pennsylvania. McCord, Michael (in press). "The slot
grammar system." In Unification in
Grammar, edited by Jiirgen Wedekin and
Christian Rohrer (also IBM Research
Report RC 17313). Cambridge, MA: MIT
Press. McCord, Michael; Bernth, Arendse; Lappin,
Shalom; and Zadrozny, Wlodek (1992).
"Natural language processing within a
slot grammar framework." International
Journal on Artificial Intelligence Tools
1:229-277. Rich, Elaine, and LuperFoy, Susann (1988).
"An architecture for anaphora resolution."
In Proccedings, ACL Conference on Applied
Natural Language Processing, 18-24. Sidner, Candice (1981). "Focusing for
interpretation of pronouns." American
Journal of Computational Linguistics
7:217-231. Sidner, Candice (1983). "Focusing in the
comprehension of definite anaphora." In
Computational Models of Discourse, edited
by Michael Brady and Robert Berwick,
267-330. Cambridge, MA: MIT Press. Walker, Marilyn (1989). "Evaluating
discourse processing algorithms." In
Proceedings, 27th Annual Meeting of the
Association for Computational Linguistics,
251-261. Walker, Marilyn; Iida, Masayo; and Cote,
Sharon (1990). "Centering in Japanese
discourse." In Proceedings, 13th
International Conference on Computational
Linguistics, 1-6. Webber, Bonnie (1988). "Discourse Deixis:
Reference to discourse segments." In
Proceedings, 26th Annual Meeting of the
Association for Computational Linguistics,
113-121. Williams, Edwin (1984). "Grammatical
relations." Linguistic Inquiry 15:639~73.