Traditionally researchers have defined the problem of comprehending anaphoric expressions as one of determining the antecedent of an anaphoric expression, that is, determining to which word or phrase an ana­phoric expression refers or "points". Recent studies in both artificial intelligence and linguistics have, dem­onstrated the need for a theory of the comprehension of anaphoric expressions, a theory that accounts for the role of syntactic and semantic effects, as well as inferential knowledge in explaining how anaphors are understood. In this paper a new theory, based on the concept of focusing in the discourse, is introduced to explain the interpretation of pronouns.

Before a theory can be given, and before even the difficulties in interpreting anaphors can be discussed, we must first re-consider what an antecedent is. The traditional definition encounters difficulty right from the start; it is founded on the notion that one word in a sentence refers or points back to another word or phrase in the (same or another) sentence. But words don't refer back to other words [Morgan 1978]; people use words to refer to entities in the world. In particu­lar they use pronouns to refer to entities which have already been mentioned in a discourse. Since an ana­phoric phrase does not refer to an antecedent, one might want to claim that both the antecedent and the anaphor co-refer to the same entity. That description is adequate for sentence si,

si I think green apples taste best and they make the best cooking apples too.

though not for discourse Dl, where there is no ante­cedent phrase in the discourse that co-refers with the pronoun "they."

Dl-1 My neighbor has a monster Harley 1200.

2 They are really huge but gas-efficient bikes.

Rather than view antecedence as co-reference, one might propose that antecedence is a kind of cognitive pointing, the kind of pointing that causes "they" and "green apples" to point (somehow) to the same class of entities in one's mind. This proposal is problematic for the same reason that co-reference is: people use pronouns when there is no other noun phrase in the discourse that points to the right mental entity. In Dl, "they" refers to bikes which are Harley 1200's as a group, while "a monster Harley 1200" mentions only some particular Harley 1200. "They" seems to be able to refer when used with the previously mentioned phrase "a monster Harley 1200" without the two phrases either co-referring or co-pointing.

If an anaphor does not refer to an antecedent phrase, and if it need not always co-refer with its ap­parent antecedent (as in Dl), then anaphor interpreta­tion is not simply finding the antecedent. Nevertheless the concept of antecedence as pointing back does seem to capture some aspect of the comprehension of anaphors, for when certain antecedent words are miss­ing from a discourse, people often fail to understand what is being said.

Let us define the problem of interpreting and un­derstanding an anaphor in the following way. The phrase "green apples" in si, when syntactically and semantically interpreted, is said to specify a cognitive element in the hearer's mind. In the computational model of that process, this element is a database item, which might be represented by Apples2 in by the sche­ma below:

string: "green apples" context: speakerl think * tastes best specifies: Apples2

Apples2:

super-concept: apples color: green used-for: cooking

The speaker uses the information in a cognitive repre­sentation like Apples2 above to choose the phrase "green apples" in si. The hearer then uses the phrase "green apples" plus the syntactic and semantic inter­pretation of the rest of the sentence to locate a similar cognitive element in his own mind; it may be slightly different because the hearer may not associate use in cooking with green apples. A cognitive element, such as Apples2, is called the specification of "green ap­ples." These elements, present in the memories of speaker and hearer, are of course related to other cog­nitive elements in their memories.

What is the relation of specifications to the real world? One might like to claim that a reference rela­tion exists between specified cognitive elements and objects in the world, but since referring is what people do with words, this relation is problematic for cogni­tive elements. Instead, specifications are said to represent the objects referred to; that is, they bear a well-structured correspondence to objects in the world. Apples2, the specification of "green apples," repre­sents the objects that are green apples. For phrases such as "Santa Claus," where there is no real world object to represent, a specification represents the men­tal schema to which are attached the properties nor­mally associated with this imaginary person.

The phrase "they" in si also specifies a cognitive element, namely the same one that "green apples" does. Since the two bear the same relation to the representation Apples2, I say that they "co-specify" that memory element, or alternatively, that the inter­pretation of "green apples" is the co-specifier of the interpretation of "they." Co-specification, unlike co-reference, allows one to construct abstract repre­sentations and define relationships between them which can be studied in a computational framework. With co-reference no such use is possible, since the object referred to exists in the world and is not availa­ble for examination by computational processes.

Even if a phrase and a pronoun do not co-specify, the specification of the phrase may be used to generate the specification of a pronoun. For example, in Dl "they" does not co-specify with the apparent anteced­ent phrase "a monster Harley 1200," but rather it refers to the class of Harley 1200's of which the ap­parent antecedent is an instance. Thus anaphor inter­pretation is not simply a matter of finding the corre­sponding cognitive element that serves as the specifi­cation of the anaphor; some additional process must generate a specification for the anaphor from the rela­ted phrase "a monster Harley 1200."

The concepts of specification and co-specification capture the "pointing back" quality of antecedence, and also permit us to formulate an explanation of ana­phor interpretation that avoids the pitfalls of the con­cept of antecedence. Anaphor interpretation can be studied as a computational process that uses the al­ready existing specification of a phrase to find the specification for an anaphor. The process uses a rep­resentation of the discourse preceding the anaphor to encode the syntactic and semantic relationships in each sentence as well as co-specification relationships be­tween phrases. These definitions in themselves do not constitute a theory of anaphor interpretation. They do, however, make possible a succinct statement of the problem: how does one determine the specifi­cation of a anaphor? Also, since we suspect that the specification of an apparent antecedent phrase plays some role in choosing an anaphor's specification, we may ask, just what is this role? We hope for a direct answer to these questions, but before one can be giv­en, let us consider how a theory of interpretation ought to address these questions. A brief look at the difficulties of choosing co-specifiers suggests which issues our theory should cover.

Determining the co-specifier of an anaphor is diffi­cult because there are a multitude of possible co-specifiers in a given discourse, and there is no simple way to choose the correct one. Yet human hearers and readers generally do recover the correct co-specifying phrase intended by the speaker. Human readers and hearers also fail to recover the co-specifying phrase in certain situations; this behavior is just as valuable an observation as garden path phe­nomena for theories of parsing. A theory of interpre­tation must predict the pattern of the hearer's and reader's correct and incorrect choices, as well as fail­ures to understand, by a rule-governed account. In addition, a taxonomy of the cases in which specifica­tions are used to generate other specifications must be given, as well as a means of predicting their distribu­tion.

Two other aspects of communication make it diffi­cult to find co-specifiers: the context of discussion and the inferences people make. People use the context surrounding an anaphor in understanding it. If a theo­ry of anaphor interpretation is to capture understand­ing, it must include a means of encoding discourse context and whatever structure it has; the context must be distilled into a form that preserves its richness without adding overwhelming complexity to the inter­pretation process. In addition, researchers have dis­covered that anaphor interpretation involves making inferences, some of which can be complex, each of which must be chosen from a large base of knowledge about objects, people and things. The practical de­ployment of inferential capabilities for any task re­quires control: knowing what to infer when, and knowing when to stop. Since the general control problem is poorly understood, a theory of anaphor interpretation must provide solutions to the more spe­cific problem of controlling inference in anaphor inter­pretation.

The major portion of this paper addresses the is­sues of determining the specifications of anaphors, with an emphasis on the role of context and inference. The theory of focusing on parts of the discourse is introduced, and a processor and its rules which rely on the focusing theory are discussed. Before turning to this discussion, the previous and extensive research on anaphora are reviewed.

The role of context and inference, as well as syntax and semantics, on anaphor interpretation have been explored extensively. A brief look at these explora­tions indicates the necessity of a new approach. Re­search on anaphora falls into four broad categories:

Rather than review each approach, I point out the contributions of each type to a theory of pronoun interpretation.

General heuristics, as a means of choosing anteced­ents, predict reliably in a large number of typical ex­amples. However, no simple characterization fits the wide variety of cases where they fail (see Winograd 1972 and Hobbs 1977); furthermore, the heuristic approach is not theoretically grounded and cannot offer a unified approach to the phenomena.

Semantic selectional restrictions, based on the Katz-Fodor theory of semantic markers, and used by many computational linguists, can reduce the space of possi­ble antecedents, but they cannot be used to eliminate all possibilities, as the example below illustrates (where feeling soft can be said of either a mud pack or one's face):

s2 Put the mud pack on your face. Notice how soft it feels. Syntactic restrictions, on logical form [Chomsky 1976] and on constituent structure [Lasnik 1976, Reinhart 1976], stipulate conditions in which a pronoun and a noun phrase must have disjoint reference, as shown below.

s3 * Near Dan, he saw a snake. s4 * The man whose house he bought went gold digging in Alaska.

These rules, however, do not stipulate the interpreta­tion of a pronoun; in a general theory they serve only as a filtering condition on the class of possible co-specifications. Furthermore, syntactic restrictions must also stipulate the disjoint reference conditions on reflexive pronouns although no adequate account of these conditions has yet appeared.

Work by researchers in artificial intelligence on inference led to methods for forward and backward chaining of inferences to bind the pronoun, represent­ed as a free variable, with some piece of knowledge; with this approach, the pronoun's interpretation was the value bound to the free variable. This approach revealed that inferences about world knowledge are often needed to interpret pronouns. However, these methods failed to control the inference process suffi­ciently. Charniak, attempting to resolve this problem, proposed demons that would "wake up" in the appro­priate situation (that is, processes which could them­selves notice when they were to begin processing). But a large cache of demons would be required, and no assurance could be given that demons would exist in every situation. Most significantly, this proposal said nothing about the situation where two or more demons might apply (who gets control? how are the decisions made?). Furthermore all of the inference-based approaches to pronoun interpretation fail to offer any theoretical approach because they rely on a simple mechanism, (simple variable binding between pronoun and some other phrase) which does not apply in many uses of anaphors, such as Dl.

Discourse approaches to anaphora include a techni­que similar to the inference method; one identifies sentence pairs and determines their semantic relation­ship as one of elaboration, similarity, contrast, parallel structure; the pronouns are interpreted by variable binding between items of the sentence pairs [Hobbs 1979]. Webber 1978, using a notion of "discourse identifications" (that contain certain semantic and discourse content) similar to the notion of specifica­tion, stipulates constraints on the representation of relations among items mentioned in a discourse.

Grosz [1977, 1978, 1981] illustrates how parts of a speaker's knowledge, relevant to a discourse segment, are highlighted via focusing, a process that reflects what a speaker says and the nature of the knowledge in the space. She shows that the structure of a task affects what items will be focused on in the discourse. Reichman has expanded this paradigm by describing context spaces, delineated by their topics. Her analysis shows that within a context space only certain items may be pronominalized. She leaves open the ques­tions: What is the recognition procedure for determin­ing a context space? How does one identify its topic? How does the hearer determine the interpretation of a pronoun, i.e., how does a hearer decide which highly focused items act as the co-specification of a pronoun? All these approaches support the view that since hear­ers do not have privileged access to a speaker's mind, other than through what a speaker says, imposing structure on the speaker's discourse will provide a framework for establishing the interpretation of pro­nouns.

The remainder of this paper defines the concept of speaker's foci and shows that they can be used to choose specifications for personal pronouns. The rules for choosing interpretations are stated within a frame­work that shows:

• how to control search in inferring by a new me­thod called constraint checking,

• how to take advantage of syntactic, semantic and discourse constraints on interpretation,

• how to generalize the treatment of personal pronouns, to serve as a framework for the theo­ry of interpretation for all anaphors.

A simple discourse can be used to illustrate how pronouns indicate what the speaker is talking about in the discourse. In the sample below, the speaker men­tions two dogs and tells us something about each. D2-1 I have two dogs.

3 the other is a cocker spaniel.

4 The poodle has some weird habits.

5 He eats plastic flowers and likes to sleep in a paper bag.

6 It's a real problem keeping him away from plastic flowers.

8 and he's a good watch dog.

9 I like having them as pets.

The speaker uses the pronouns "he" and "him" to indicate that the poodle is the actor of eating flowers and the object of an action of keeping. The speaker mentions his second dog with "the other" and then uses the pronoun "he" only in predicate adjective and predicate nominative constructions without reference to any events. Initially the speaker focuses attention on both dogs and then turns his attention to each for a part of the discourse. The process by which the speaker uses language to indicate his focus is called the focusing process. The items in focus are those that are talked about for a part of the discourse. Items in focus can be used in two ways, as actor focus and as discourse focus. The actor focus is a discourse item that is predicated as the agent in some event, so for example, the poodle is the agent in eating flowers, while the discourse focus is, roughly speaking, an item the speaker wishes to make several predications about.

The focusing process, when explained from the viewpoint of a hearer, can be described as a process of tracking the speaker's foci as they change over the discourse. A hearer does not have privileged informa­tion about what is in a particular speaker's head, so the hearer must decide what the speaker is talking about on the basis of what the speaker uses as initial referring expressions and subsequent co-specifying anaphoric expressions. The hearer follows a focus and checks to see if the anaphoric expressions which the speaker uses either co-specify with the hearer's repre­sentation of that focus or specify a representation related to it. The hearer is said to be tracking the speaker's focus because she or he can determine what is being talked about only after the speaker has said something. Focus, as it is used here, is akin to Grosz's notion [1977] of immediate focus.

A process model of focusing and focus tracking consists of three distinct processors. The first chooses foci based on what the speaker initially says. The algorithm for this choice depends on many phenomena - see Sidner 1979 for details. Then an interpreter (the pronoun interpreter) uses these foci and a set of rules of pronoun interpretation (hereafter "pi-rules") to interpret the anaphoric expressions in the discourse. This interpreter, like a human hearer, must keep in mind whatever other newly mentioned elements the speaker has introduced, since sometimes an anaphor may co-specify with one of those instead of the ele­ments in focus. A third process updates the foci by decisions that depend on anaphoric interpretations chosen by the pronoun interpreter. During this last phase, the updating process moves one of the foci to a new element of the discourse, if some anaphor co-specified with that new element while no anaphor co-specified with the phrase already in focus. The three processes taken together sketch a simple model of focus tracking; the model appears to behave like its human counterpart in the way it interprets anaphors and in the instances in which it fails to "understand."

So far, the process model looks circular. Foci are chosen, they are used to determine how anaphors co-specify with the foci and then the co-specifiers are used to determine what's in focus. In fact, the model is not circular, but its steps are cyclic. The processor cycles through the three processes for each sentence in the discourse. This cycling differs from logical circu­larity because it depends on new information present­ed over time, the time of each sentence of the dis­course.

For example, reconsider the first few sentences of D2, given previously. Initially the speaker mentions two dogs, which become the discourse focus, and then he extends the discussion of them using "one...the other" anaphors. The actor focus is the speaker. Us­ing a definite noun phrase anaphor "the poodle," the speaker turns his attention to the that dog, and it be­comes the discourse focus and the actor focus. Using another definite noun phrase anaphor "the cocker," the speaker changes the discourse focus to the cocker. The speaker's concluding sentence expands the dis­course focus back to both dogs with the use of "them."

In Sidner 1979 a machine for choosing and updat­ing the discourse focus and actor focus is given. The machine chooses items in the discourse to serve as "expected" foci after the first sentence of a discourse and uses the interpretation of pronouns as well as non-pronoun anaphors to determine whether its ex­pected choices were correct. The machine updates the discourse and actor focus after each sentence and changes foci when the anaphoric expressions no longer co-specify with the item in focus. This simple machine tracks the foci from the hearer's point of view and shows how the hearer can recognize foci and changes of foci. In the remaining discussion, it is assumed that the focus machine can determine a discourse and an actor focus. Our attention is directed at the following concerns. How can these two foci be used to interpret discourse pronouns? What rules are used by the pro­noun interpreter? How do these rules make use of the constraints on the theory of pronoun interpretation discussed earlier in this paper?

The focusing theory of pronoun interpretation can be outlined as follows. At any time in the discourse after the first sentence, there are foci that are the prime candidates for co-specifying with a pronoun. Also available is a list of alternate candidates, called potential foci, for each focus. Unless the focus candi­date is ruled out by certain criteria (to be discussed below), the pronoun interpreter uses it to determine the specification of the pronoun. Most of the discus­sion in this paper concerns the criteria that must be encoded in the rules for choosing among the foci as well as the criteria for rejecting a focus in favor of one of the potential foci.

The proposal made here contains two implicit proc­essing assumptions, (1) serial processing, and (2) end-of-sentence processing. By "serial processing", I mean that the interpreter checks a focus as a candidate in interpreting the pronoun, and then if that focus is unacceptable, checks alternate candidates in turn. By "end-of-sentence processing", I mean that pronouns are not interpreted until the entire sentence has been syntactically and semantically interpreted. Both of these criteria could be given up without undermining the focusing theory. One could envision processing in parallel by checking the foci and alternates and then determining the pronoun's specification from an order­ing of all those candidates that meet the criteria of choice. To interpret pronouns before the end of the sentence, the interpreter could decide using available information and then review its choice as more of the sentence is processed. These two implicit processing assumptions have been made because they simplify the account of focus and because they reflect an imple­mented version of one focus system. Further research will indicate whether these assumptions are too strong - if so, the focusing theory may be re-tested under parallel and mid-sentence interpretation assumptions.

The pronouns in the example discourse in the pre­ceding section can be interpreted with a simple rule:

6 while Zohar tried the flavor of the day cone.

8 the boys went home to study.

In D3, Alfred and Zohar are the initial actor focus while baseball is the initial discourse focus. D3-2 contains two pronouns, "they" and "it" which, ac­cording to R2, co-specify respectively with Alfred and Zohar, and baseball. D3-3 uses "their," which co-specifies with Alfred and Zohar, but is not accounted for by rule R2. Furthermore "they" in D3-4 does not co-specify with baseball but with ice cream cones. What has happened?

Suppose in addition to the ongoing actor and dis­course focus, the hearer can consider temporarily any new entities mentioned in the last sentence that the hearer has heard. These entities, the potential foci, are dropped out of processing use if the speaker fails to mention them a second time; such alternatives are elements the speaker may want to say more about, but their importance to the speaker cannot be determined from one sentence. Thus in D3-2, foci include the event of playing, and the times "everyday", "before dinner," and "after school". None of these potentials survives long into the discourse as D3-3 fails to pick up on any of them. D3-3 in turn introduces some other potential foci, and one of them, ice cream cones, is discussed in D3-4; notice also that D3-4 fails to mention baseball in any way.

Focusing theory with the syntactic and semantic representations discussed initially does not include an account of the representation of sentence pairs such as D6 where the sentence is ambiguous due to scope of quantifiers.

D6-1 Wendy showed each girl Bruce knows a cat. 2 a She had found it at Farmer John's, b They were all from the same litter. At present the focusing rules predict only that "a cat" is the expected discourse focus, but they say nothing about the scope relations for representing the seman­tics of the phrase as part of D6-1. Without scope relations, D6-2b is ruled out by the number agreement criteria when in fact most speakers say that "they" co-specifies with the set of cats that may be evoked in D6-1. Webber 1978 not only argues in detail for rep­resenting scope relations in the mechanisms for inter­preting anaphors, but also gives rules for determining scope in a representation of phrases that may be co-specifiers.

On Webber's analysis, D6-1 will have two repre­sentations for "a cat" and for "each girl Bruce knows." The one for "a cat" can be paraphrased as:

1. Rl: the cat associated with D6-1 such that Wendy showed it to each girl Bruce knows.

2. R2: the set of cats, the members of which are asso­ciated with D6-1 such that Wendy showed (one of) them to each girl Bruce knows.

Webber has suggested what the representation must be, and what remains to be determined is when it is used. Let us suppose that D6-1 is ambiguous, and no syntactic or semantic processing of it adjudicates the two readings Rl and R2 for "a cat." When the pro­noun interpreter seeks to co-specify "it" in D6-2a with the discourse focus, both readings will be availa­ble. The set reading R2 may be eliminated immediate­ly because of constraints that the co-specifier repre­sent a singleton, so Rl is left. In contrast, for D6-2b the reading Rl will be ruled out since "they" requires a plural co-specifier. This account both explains how the pronouns may be understood, and also is consist­ent with Van Lehn's findings [1978] that people do not normally disambiguate certain ambiguous quantifi­er scopes during sentence understanding for sentences such as D6-1. It is compatible with Van Lehn's find­ings because alternative interpretations of scope are not considered until additional discourse material be­yond the single sentence is presented.

Webber's rules also allow the phrase "each girl Bruce knows" to be interpreted as a prototype, with the result that "a cat" is also. Most speakers find this reading very odd, and one can ask how such a reading is eliminated. Possibly it is never generated, and Webber's rules need modification. Alternatively it is produced, and then is ruled out, either in understand­ing D6-1 because showing prototypic cats is bizarre, or because in interpreting "it" for D6-2a, the infer­ence mechanism balks at Wendy having found a proto­typic cat at Farmer Brown's. I think that such read­ings never occur in the first place, simply because most speakers, who do not report this reading themselves, have difficulty understanding the reading when they are told about it.

Another case of semantic ambiguity, similar to the one in D6, is illustrated in D7.

D7-1 Sally wanted to buy a vegomatic.

2 She had seen it advertised on TV.

"A vegomatic" may be interpreted specifically, to mean that there is one particular vegomatic or non-specifically, to mean that it is one of the many vego-matics. The focusing rules do not distinguish between the two after D7-1 because, like D6-1, D7-1 is ambi­guous, and neither interpretation can be chosen with certainty. When "it" is resolved for co-specification in D7-2, the inference mechanism must decide that Sally does not want the very one used in the advertise­ment on TV. "A vegomatic," whether understood as specific or non-specific, specifies a different represent­ation than "it" does. Therefore the pronoun's specifi­cation as a specific vegomatic must be generated from the ambiguous use.

Suppose, for a moment, that D7-1 is interpreted so that a representation that maintains ambiguity is avail­able. When the pronoun interpreter processes a sub­sequent sentence with a pronoun, it need only rule out readings if the inference machine discounts as contra­dictory one of the readings (Sally didn't want to buy the very one she saw on TV). If no reading is ruled out, the co-specifier would remain ambiguous, so that both the indefinite phrase and the pronoun would have ambiguous co-specifications. As the next example shows, there is some evidence for this behavior.

Consider the case shown in D8.

D8-1 Sally bought a vegomatic that had a broken cutting blade.

2 She had seen it advertised on TV. "A vegomatic that had a broken cutting blade" in the context of D8-1 usually means some particular vego­matic that Sally bought. However, "it" is ambiguous among the vegomatic Sally bought (the broken one), some vegomatic (possibly not broken), and a vegomat­ic that is an instance of prototypic vegomatic. Thus "it" is three ways ambiguous.

To understand D8-2, the pronoun interpreter does not distinguish among the three readings, since it ac­cesses the one provided by the specification in D8-1, which is the specific reading. To find the specification of "it," the inference mechanism must discover that it is slightly odd (1) for Sally to have seen the vegomatic with a broken blade which she bought being advertised on TV, and (2) for Sally to see any broken vegomatics on TV, and (3) for Sally to have seen the very one she bought on TV. Then if no other choices for co-specification are available, the specification of vego­matic from D8-1 must be used to generate an appro­priate specification for "it". Since only unbroken ones not bought by Sally are appropriate, the specification of "it" must be generated using only part of the phrase from D8-1.

This example seems problematic because it places much weight on the inference machine to decide that the reading is odd. However, this is likely to be just where the weight of the decision ought to be; many native speakers find D8 slightly bizarre because their first reading is that Sally had seen the vegomatic with the broken blade advertised on TV. In fact, it appears that when a specific indefinite noun phrase such as "a vegomatic" is introduced, and the speaker wants to turn attention to the non-specific reading, a plural pronoun is used as shown below:

slO She had seen them advertised on TV.

The plural non-specific reading as in s 10 is mentioned in the pronoun rules found in the appendix, but the generation of a specification as in D8-2 is not.

Examples such as D8 are perplexing for another reason; they are examples of what I call, following Fahlman 1977, the "copy phenomenon." The ambigu­ity centers around the fact that there can be many copies of an abstract prototype. Automobiles, com­puter programs, airplane flights and money are other common cases of entities that exhibit the copy phe­nomenon. In D9, the interpretation of "it" depends on whether the speaker is referring to a particular flight or the normal Sunday flight, a copy of which occurred on "this Sunday" because of the use of "usually".

D9-1 TWA 384 was so bumpy this Sunday I almost got sick.

2 It usually is a very smooth flight. Note that the "it" cannot co-specify with the particular flight on "this Sunday". However, it is possible that the speaker used "TWA 384" to refer to a particular flight; if this is so, the speaker mixed the specific and non-specific interpretations for the co-specifier of "it". Just as in D8, a specification for the pronoun must be generated — in this case, a non-specific one from the specific reading.

Another characteristic anaphora is the bound varia­ble case described by Partee [1978, 1972]. In D10 below, "him" co-specifies with Archibald, while if the reflexive "himself" were used, it would involve a vari­able bound to the quantifier from "every man."

D10-1 Archibald sat down on the floor.

2 Every man put a screen in front of him.

In linguistic theory, bound variables are assumed to be represented in sentence semantics; when used in con­junction with syntactic disjoint reference rules, pro­nouns within the scope of the quantifier can be distin­guished from non-scoped ones. Since the pronoun interpreter takes account of these conditions, it can easily choose a proper co-specifier for "him" in D10 in terms of the focus, but for "himself" it will recog­nize the bound relation to "every man." It is crucial to these cases that the representation of the interpreta­tion of a sentence includes scope of quantification, especially when the scope is unambiguous.

There are other restrictions on co-specification that result from the processing of the focus machine. The focus machine includes a stack on which old foci are stored when the focus changes. In addition to co-specifications with current actor and discourse foci, a speaker may use a pronoun to co-specify a discourse element that was once in focus but is no longer; Grosz 1977 described and illustrated this behavior for ana­phoric noun phrases in task-oriented dialogues.

A pi-rule that selects candidates from the focus stack can capture this behavior. However, the rule must be further constrained. An anaphor that is in­tended to co-specify with a stacked focus must not be acceptable as a co-specifier with either the foci or potential foci. An example from a literary text illus­trates how.

Was that old lady evil, the one Saul and I had seen sitting on the porch? I had dreamed about her. When the trolley car took me and Saul past her house again this morning, she was gone. Evil, it had a queer sound to it in Eng­lish.

[Here the narrative moves on to an incident in a school classroom. A discussion between the speaker and a male teacher ensues for five para­graphs. Then the following paragraph begins:]

She had worn an old brown coat and a green scarf over her head.

In this example, "she" co-specifies with the old lady discussed previously. If Potok had told of a discussion between the speaker and a female teacher, it would no longer be possible to tell that "she" was co-specifying with the old woman. While interpreting the reading of "she" as teacher might be a bit surprising because the teacher's clothing was not relevant to the previous conversation, no inference can rule out that reading because teachers may wear old brown coats. Hence the pronoun interpreter must reflect these facts, and it does so with the stacked focus constraint.

The stacked focus constraint is not stated directly within the pi-rules. Instead it is implicit in their func­tion. The condition is as follows: A pronoun cannot be used to co-specify with a stacked focus when a current focus is an acceptable co-specification since that current focus will be taken as the interpretation and the stacked focus will never come into considera­tion. The stacked focus constraint is a consequence of the movement of focus in the focus machinery.

The stacked focus constraint, however, may be overridden. Grosz (see Deutsch 1975, 1974) has identified several examples, involving pumps and bolt­ing operations, that show that a pronoun may co-specify with the stacked focus even when intervening material contains possible co-specifiers. Some compli­cated set of inferences about what can and cannot be bolted to what, or what can and cannot be loosened might be able to rule out the intervening possible co-specifications. However there appears to be too little delay in understanding for people who read her ex­cerpts (there is no means of determining whether the original speaker and hearer experienced any delay in understanding) to suggest that they are ruling out mul­tiple possibilities in interpreting the pronouns.Hence one may conclude that the speaker and hearer are taking advantage of their knowledge of the task to provide a discourse context in which the focusing ma­chinery can be applied.

How many such discourse interpretation mecha­nisms exist? While this paper does not address this question directly, some speculation is possible on the basis of research that has been reported elsewhere (see Sidner 1979, Robinson 1981). In general, it appears that discourses permitting violations of the stacked focus constraint must contain an implicit structure of task completion that guides the hearer to a context in which the foci for that context may be re-used. With­out this structure the hearer cannot decide that the speaker intended for the pronoun to co-specify with something other than the representation of the object currently under discussion.

The previous discussion has assumed that a pro­noun is always preceded by a co-specifying phrase. However, this is not always the case, and a complete theory of pronoun interpretation must address cases where the co-specifying phrase appears after the pro­noun, and where no co-specifier exists, but the discus­sion implies a specification for the pronoun.

Pronouns used with their co-specifiers appearing after the occurrence of the pronoun have been called backward anaphora in the linguistic literature; I refer to them as forward co-specifiers. Two such examples, sll and s 12, are given below.

sll If he comes before the show, give John these tickets and send him to the theatre.

si2 Near him, Dan saw a snake. In general the pronoun co-specifies with some noun phrase interpretation, but the phrase is placed forward in the discourse. The types of sentences in which this behavior can occur are limited. In general it seems to be permitted for fronted sentential prepositional phras­es (as in s 12), subordinate clauses fronted on another sentence (as in sll), and for sentences containing co-ordinating conjunctions. However, it is excluded, as far as I can tell, in the following cases: si3 * I heard about her job from Mary.

sl4 * I spoke about him with John's wife. Extensive research in linguistics on forward co-specifiers (Solan contains a good review) gives reliable evidence that structural constraints govern it; in par­ticular, syntactic rules can be stated that determine when forward co-specifiers are not permitted. The most recent formulation, by Solan, called the back­ward anaphora restriction, fails on the cases below,

si5 Penelope grabbed his cane and beat Peter with it.

but some working modification of it may yet be forth­coming.

sl6 In her room Mary saw a ghost.

sl7 I heard about Mary's job from her.

Whatever the best formulation of the syntactic rules for forward co-specifiers, they are preferable only in initial sentences of a discourse. For example, when sll occurs in mid-discourse, if a speaker has been talking about Henry, and just begun mention of Charles, native speakers will interpret "he" as co-specifying with Henry, or Charles (if Henry can be ruled out on basis of some special pragmatic knowl­edge). The pi-rules using focus behave in exactly this way. They will permit a forward co-specifier only if Henry and Charles can be ruled out by syntactic, se­mantic and world knowledge criteria. Were sll to occur when Lydia was in focus, the forward co-specifier would be possible, and pi-rules mirror this behavior.

When faced with a pronoun that has no preceding co-specifier and is not a forward co-specifier, the pro­noun interpreter relies on a condition that is called the missing co-specifier condition. In the remainder of this section, I describe that condition. The pi-rules include a rule for recognizing that a pronoun is missing its co-specifier, but they do not offer an interpretation for such pronouns.

In each of the cases below, the pronoun specifies some cognitive element that is related to one of the entities mentioned in either the previous sentence or the same sentence. These examples are different from the non-specific prototypic readings of pronouns discussed earlier because no phrase which can be used as a generator exists.

Dll-1 I saw Mr. Smith the other day; you know, she died last year.

2 John is an orphan. He misses them very much.

3 Pro-Castro people don't believe he is a monster.

4 I went to a concert last night. They played Beethoven's ninth.

5 I want to meet with Bruce next week. Please arrange it for us.

6 I used to be quite a tennis player. Now when I get together with the young guys to play, I can hardly get it over the net.

With the exception of Dll-1, most speaker-hearers are able to say which is the intended specification of the pronoun in the cases above. Dll-1 can be understood if the hearer is informed that Mr. Smith had a wife. However, some of these, especially 1 and 2, are so odd for most hearers that at first they fail to comprehend the pronoun. Hearers are divided on the acceptability of 3, and most hearers find 4 and 5 acceptable. Fur­thermore, such examples, as far as I can tell, do not occur naturally in written samples.

This paper does not give an account of just how such cases are understood. However, the focusing approach provides some basic structure that may be useful in generating an explanation for such situations. In all the multi-sentence cases, the pronoun specifies something which is closely associated with the focus. What is problematic is the fact that some speakers seem unable to understand a pronoun which specifies a database element that, while related to the item in focus, represents something that no longer exists, such as John's parents in light of John's orphanhood. Whatever the manner in which hearers recover specifi­cations for such pronouns, some principles are needed which govern why some uses of pronouns are accepta­ble and others are not.

The pi-rules give incorrect predictions for certain uses of pronouns, uses that are difficult to define. Intuitively, they may be characterized as occurring when there is a parallel structure between sentences of a discourse. In many of these cases the pi-rules pre­dict the wrong co-specifier. To understand what is meant by parallel structure, two simple cases, one in which the pi-rules do predict correctly, and another in which they fail, are discussed. In D12, the pronoun co-specifies with the mud pack, as the pi-rules would predict. The parallelism of these sentences is reflected in the semantics of "put on" and "pull off" as well as in the similarity of the syntactic structure of the two sentences, each being in imperative mood. D12-1 Put the mud pack on your face.

2 After 5 minutes, pull it off. The pi-rules predict the proper co-specifier in D12 because the thematic relations of the verb follow the similarity of structure. However, in D13 below, the pronoun "it" co-specifies with rose and not with the green Whitierleaf (the pi-rule choice). The initial focus after the first sentence is Whitierleaf, but the parallel syntactic structure of the sentences seems to govern a different choice of co-specifier. To summa­rize, between similarity of structure and the pi-rules, similarity is preferred as a means of choosing a cospecifier, so when each gives a different prediction, similarity of structure must be used.

D13-1 The green Whitierleaf is most commonly found near the wild rose. 2 The wild violet is found near it too.

At first glance it appears that the pi-rules could be "fixed" by simply observing that the initial focus is wrong and that a potential focus should be chosen. However, no such option is available, for such a "fix" requires that the inference machine reject the initial focus. To do so, the inference mechanism needs some knowledge about the world that indicates the unac-ceptability. For D13 no such knowledge could possi­bly be forthcoming since all the flora involved are found near one another. There is no knowledge to the effect that violets are found near wild roses and not near Whitierleafs.

Another example of parallel structures is shown in D14. The parallel structures again are reflected in the similarity of the syntactic forms as well as the seman­tics of "most" and "mine". After D14-1, the initial focus is the car radiator that is associated with each car of "most cars." Using the focusing rules, the pro­noun interpreter will take "it" to co-specify with that radiator. But this prediction is incorrect; "it" co-specifies with the radiator of the speaker's car.

D14-1 On most cars the radiator has a free bolt hook.

2 But on mine, it has a floating bolt hook.

The use of "it" here is similar to the instance of a prototype for "it" in D8. The two discourse examples differ because D14-2 has an underlying semantic form that parallels D14-1. D14-1 specifies a universal set of cars and says something about one of the parts for those cars; D14-2 specifies a set of one thing, the speaker's car, and says something about a part of it; the speaker's car is related to the universal car by instantiation. Thus "it" in D14-2 is not pointing to some instance of the prototypic radiator; it co-specifies with the radiator of the speaker's car, but "it" is related intensionally to the "radiator" in D14-1. The similarity in the underlying semantics of D14-1 and D14-2 must be used in interpreting the pronoun uses.

One might wish to construct some special-purpose mechanism that looks for similarities in structure be­tween two sentences. This method is doomed for two reasons. First, parallelism exists in many aspects of language, and it happens at arbitrary levels of structure rather than just syntactically. Second, at any given level, the problem of recognition of parallelism has plagued computational models of language since such models were first suggested. For example, pars­ing of English sentences containing conjunction is as yet an unsolved problem. Methods tried, such as those of Woods 1973 in LUNAR, fail because of over-generalization. Proper computational recognition of parallelism is still beyond the state of the art.

The fact that interpretation of parallelism has failed for other aspects of computational models of language only indicates that the problem is a deep one. The example below is especially surprising because it dem­onstrates the use of parallel constructions that may be found between whole paragraphs in a discourse. The interpretation of "the schedule," used anaphorically in D15-7, is not ambiguous between "SOL," which is a kind of schedule, and the transmission schedule; read­ers recognize that the anaphor is unambiguous presum­ably because the process described in lines 5-7 paral­lels the one in lines 2-4. Whatever the proper account of parallelism for pronominal anaphora, it must also be generalizable to account for this kind of example.

D15-1 The SOL is searched for an entry for the subscriber.

2 If one is found, the subscriber's relative transmission time is computed according to formula-1.

3 The subscriber's clock transmission time is computed according to formula-2.

4 When the transmission time has been computed, it is inserted as the pri­mary entry in a transmission schedule.

5 For each RATS entry, the RATS's rela­tive transmission time is computed according to formula-1,

6 and the RATS's clock transmission time is computed according to formula-2.

7 The RATS transmission times are entered into the schedule.

One possible consequence of these observations could be that the focus mechanism should be aban­doned in favor of some as yet unspecified mechanism that is able to determine parallel relations among sets of sentences in a discourse. However, methods for interpreting pronouns from parallel sentences and par­agraphs offer no constructive way of interpreting the pronouns in most of the examples presented in this paper. Many cases of co-specification occur where there is no similarity of structure other than the common subject-verb-object pattern typical of English sentences. Since what is being talked about appears in many constituent positions in sentences of a discourse, the S-V-O pattern is too gross a level to specify simi­larity. Hence, while parallelism is needed to deal with a certain set of cases for which the pi-rules predict incorrectly, the pi-rules are effective for many other cases of co-specification where parallelism would not be helpful. One may conclude that focus mechanisms account for one aspect of pragmatic anaphora, and that some different mechanism is needed to encode similarities in structure that sometimes occur in dis­course. This paper does not give an account of such a mechanism. Rather, the examples above provide some additional observations about the nature of parallelism in interpreting pronouns in natural languages.

Two claims have been substantiated in this paper. First, I have shown that focusing is compatible with linguistic rules for disjoint reference, semantic selec-tional restrictions, and with representations delimiting quantifier scope. Furthermore, these sources of infor­mation are necessary for the pronoun interpretation rules.

Second, I have demonstrated that focusing helps control the inference process needed to interpret pro­nouns. The pronoun interpreter predicts a co-specifier and then asks the inference machine for confirmation; when the machine finds a contradiction rather than consistency, the focusing process produces a new can­didate co-specifier. In effect, the focusing process and the inference machine collaborate on pronoun inter­pretation. In previous AI natural language systems, interpretation resulted from binding of free variables when making inference; the inference process could be characterized as one of proving a consequent from a set of premises. However, because the consequent of an inference rule contained free variables, many infer­ences had to be drawn and then "undone" during the search for a complete chain of inferences. The focus­ing approach eliminates this kind of blind binding and unbinding as well as shortens the inference chain search.

A part of this paper has described and illustrated how the pronoun interpretation rules predict the co-specifiers of pronouns with both the actor and the discourse focus, and indicated that both actor and discourse foci are necessary. The rules show how many constraints — syntactic, semantic, and pragmatic — affect the choice of specification for pronouns; they also show how many kinds of information about the world of the speaker and hearer play a part in distinguishing the co-specifications of pro­nouns. By means of the discourse and actor foci, the pi-rules differentially apply the constraints; the predic­tions which result are tested in the database represent­ation of the speaker and hearer's world. Focusing captures the effects of foregrounding (cf. Chafe 1975) since focusing accounts for the co-specifications of pronouns by means of the foci, and focus movement indicates how new entities may be foregrounded and pronominalized.

In this paper I have demonstrated the types of ana­phoric uses that cannot be explained by the focusing theory. These uses, the focus "popping" cases de­scribed by Grosz, and the parallelism cases, illustrate that in anaphor interpretation, other kinds of processes that are computationally realizable and controllable are needed. Furthermore, an adequate syntactic theo­ry of forward co-specifiers, which remains to be dis­covered, must be incorporated in the syntactic criteria for the pronoun interpretation rules.

This paper further specifies the nature of focusing as it relates to a theory of pronoun interpretation. A focus-based theory with stipulations for syntax, se­mantics and inferential knowledge, provides a pre­dictive and explanatory theory of pronoun interpreta­tion. The theory is predictive because it stipulates legal and illegal pronoun uses as well as their interpre­tations; the limits of its predictive power have been demonstrated as well. The theory is explanatory be­cause several observations about anaphora in English, all crucial to focusing theory, can account for each of the rules: that pronouns are signals of what is being discussed, that whenever the discussion changes, pro­nouns must signal the new element of discussion with­out confusion, and that hearers make use of syntactic, semantic, and pragmatic knowledge in a controlled way in understanding anaphors. A theory of anaphor interpretation different from the focusing theory needs either to use these observations in an account of the facts or to demonstrate why these observations are not relevant to the interpretation process.