"distributed architecture", but their ideas a) do not seem to capture enough discourse and heuristical knowledge and b) do not concentrate on and investigate a concrete domain, and thus risk being too general. We have tried nevertheless k) incorporate some of their ideas into our proposal. THE ANAPttORA RESOLUTION MODEL Our anaphora resolution model integrates modules containing different types of knowledge - syntactic, semantic, domain, discourse and heuristical knowledge. All the modules share a common representation ofthe cunent discourse. The syntactic module, for example, knows that the anaphor and antecedent must agree in number, gender and person. It checks if the c-command constraints hold and establishes disjoint reference. In cases of syntactic parallelism, it prefers the noun phrase with the same syntactic role as the anaphor, as the most probable antecedent. It knows when cataphora is possible and can indicate syntactically topicalized noun phrases, which are more likely to be antecedents han non-topicalized ones. The semantic module checks for semantic consistency between the anaphor and the possible antecedent. It filters out semantically incompatible candidates following the cun-ent verb semantics or the animacy of the candidate. In cases of semantic parallelism, it prefers the noun phrase, having the same semantic role as the anaphor, as a most likely antecedent. Finally, it generates a set of possible antecedents whenever necessary. The domain knowlcdge module is practically a knowlcdge basc of the concepts of the domain considered and thc discourse knowledge module knows how to track the center throughout the current discourse segment. The heuristical knowledge module can solnetimes bc helpful in assigning the antecedent. It has a set of useful rules (e.g. the antecedent is to be located preferably in thc current sentence or in the previous one) and can forestall certain impractical search procedures. The use of colnmon sense and world knowledge is in general commendable, but it requires a huge knowledge base and set of inference rules. The present version of the model does not have this mcxtule IIIiURISTICAI,

KNOWI ,I ';l X;t i l)omain lleuristics

Rating Rules

Recency

\[ Rl:ilqlilil iNTIA\], l ihJ ANAPIIOR-~--~.I ixptaisstoN SYNTACTIC KNOW\] ,t il )(}l i

Number Agrccmenl

Gender Agfeelncnl

PCI'SOll A~l'Celllellt

l)isjoim Reference

(~-(~ommaud Constraints

Cataphora

Syntactic Paralldislll

Syntactic Topicalization Although the syntactic and semantic criteria for the selection of an antecedent are already very strong, they are not always sufficient to discriminate alnong a set of possible candidates. Moreover, they serve more as filters to eliminate unsuitable candidates than as protx)sers of the most likely candidate. Additional Figure 1: Anaphora resolution model THE NEED CRITERIA FOR DISCOURSE

1)OMAIN KNOWI ,El X\]! i l)omain Concept Ktlowledgc 1 ~ase ANTI iCI il)ENT St,',MANTI( implementcd; its development, however, is envisaged for later stages of the project. The syntactic and semantic modules usually filter the possible candidates and do not propose an antecedent (with the exception of syntactic and semantic parallelism). Usually the proposal for an antecedent comes from the domain, heuristical, and discourse modules. The latter plays an important role in tracking the center and proposes it in many cases as the most probable candidate for an antecedent. Figurc 1 illustrates the general structure of our anaphom resolution model. I)|SCOURSI ~, KNOW I ,l ilX\]l'~ '1 'racking Center ANAPIIORA P,I {SOl ,VI {R

KNOW1,1 '~I)GE Semm~tic Consistency

Case Roles Semantic Parallelism

Animacy

Set Generalion criteria are therefore needed. As an illustration, considerthe following text. Chapter 3 discusses these additional or auxiliary storage devices, wlfieh mc similar to our own domestic tape cassettes and record discs. Figure 2 illustrates lheir connection to the main cenlral memory. In this discourse segment neither the syntactic, nor the semantic onstraints can eliminate the ambiguity between "storage devices", "tape cassettes" or "record discs" as antecedents for "their", and thus cannot urn up a plausible antecedent from among these candidates. A human reader would be in a better position since he would be able to identify the central concept, which is a primary candidate for pronominalization. Correct identification of the antecedent is possible on the basis of the pronominal reference hypothesis: in every sentence which contains one or more pronouns must have one of its pronouns refer to the center 1 of the previous sentence. Therefore, whenever we have to find a referent of a pronoun which is alone in the sentence, we have to look for the centered clement in the previous sentence. Following this hypothesis, and recognizing "storage devices" as the center, an anaphora resolution model would not have problems in picking up the center of the previous sentence ("storage devices") as antecedent for "their". We see now that the main problem which arises is the tracking of the center throughout the discourse segment. Certain ideas and algorithms for tracking focus or center (e.g. \[Brennan et al.87\]) have been proposed, provided that one knows the focus or center of the first sentence in the segment. However, they do not try to identify this center. Our approach determines the most probable center of the first sentence, and then tracks it all the way through the segment, correcting the proposed algorithm at each step. TRACKING THE CENTER IN "\['HE SUBLANGUAGE OF COMPUTER SCIENCE Identifying center can be very helpful in 1 Though "center" is ml uncrancc specific notion, we refer to "sentence center", because in many cases the centers of the uttermmes a senlence may consist of, coincide. In a complex sentence, however, we distinguish also "clause centers" anaphora resolution. Usually a center is the most likely candidate for pronominalization. There are different views in literature regarding the preferred candidate for a center (focus). Sidner's algorithm (\[Sidner 811), which is based on thematic roles, prefers the theme o\[ the previous sentence as the focus of the current sentence. This view, in general, is advocated also in (\[Allen87\]). PUNDIT, in its current implementation, considers the entire previous utterance to be the potential focus (\[Dahl&Ball 901). Finally, in the centering literature (\[Brennan et al. 87\]), the subject is generally considered to be preferred. We have found, however, that there are many additional interrelated factors which influence upon the location of the center. Wc studied the "behaviour" of center in various computer science texts (30 different sources totally exceeding 1000 pages) and the empirical observations enabled us to develop efficient sublanguage-dependent heuristics for tracking the center in the sublanguage of computer science. We summarize the most important conclusions as follows: 1) Consider the primary candidates for center from the priority list: subject, object, verb phrase. 2) Prefer the NP, representing a domain concept to the NPs, which are not domain concepts. 3) If the verb is a member of the Verb set = {discuss, present, illustrate, summarize, examine, describe, define, show, check, develop, review, report, outline, consider, investigate, explore, assess, analyze, synthesize, study, survey, deal, cover}, then consider the object as a most probable center. 4) If a verbal adjective is a member of" the Adj set = {defined, called, so-called}, then consider the NP they refer to as the probable center of the subsequent clause/current sentence. 5) If the subject is "chapter", "section", "table", or a personal pronoun - 'T', "we", "you", then consider the object as most likely center. 6) If a NP is repeated throughout the discourse section, then consider it as the most probable center. 7) Kf an NP occurs in the head of the section, part of which is the current discourse scgment, then consider it as the probable center. 8) If a NP is topicaKized, then consider it as a probable center. 9) Prefer definite NPs to indefinite ones. K0) Prefer the NPs in the main cKausc to NPs in the subordinate clauses. K 1) If the sentence is complex, then prefer for an antecedent a noun phrase from the previous chmse within the same sentence. As far as rule I is concerned, we found that the subject is a primary candidate for center in about 73% of the cases. The second most likely center would be the object (25%) and the third most likely one the verb phrase as a whole (2%). Therefore, the priority list \[subject, object, verb phrase\] is considered in terms of the apriori estimated probability. There are certain 'symptoms' which determine the subject or the object as a center with very high probability. Cases in point are 3) and 5). Other cases are not st) certain, but to some extent quite likely. For example, iK a non-concept NP is in subject position and if a repeated concept NP, which is also in a head, is in object position, it is ahnost certain that the latter is the unambiguous center. Moreover, certain preferences are stronger than others. For example an NP in subject position is preferred over an NP in a section head, but not in subject position. We have made use of our empirical results (with approximating probability lneasures) and AK techniques to develop a proposer module which identifies the most likely center. We must point out that even iK we do not need one for immediate antecedent disambiguation, a center must still be proposed for each sentence. O" else we will have to go all the way back to track it from the beginning of the segment when one is needed later on. The rules 1)- 11) should be ordered according to their priority - a problem, which is being currently investigated. Tracking the center in a discourse segment is very important since knowing the center of each current sentence helps in many cases to make correct decisions about an antecedent in the event that syntactic and semantic constraints cannot discriminate among the available candidates. AN ARTIFICIAL INTELLIGENCE APPROACH FOR CALCULATING THE PROBABILITY OF A NOUN (VERB) PHRASE.TO BE IN THE CENTER On the basis of the results described in the previous section, we use an artificial intelligence approach to determine the probability of a noun (verb) phrase to be the center of a sentence. Note that this approach allows us to calculate this probability in every discourse sentence, including the first one and to propose the most probable center. This approach, combined with the algorithm tbr tracking the center (\[Brcnnan et al. 87\]), is expected to yield improvcxl results. Our model incorporatcs an AI algorithm for calculating the probability of a noun (verb) phrase to be in the center of a discourse segment. The algorithm uses an inference engine based on Bayes' theorem:

P(HK) P(AII-IK) P(I t\[dA) ...................

P(I KOP(AIH~) forK = 1,2,... Under the conditions of our model Bayes' theorem allows the following 1173 interpretation: there are only two possible hypotheses for a certain noun (verb) phrase -that it is the center of the current sentence (clause) or that it is not. Let Hy be the positive, while HN - the negative hypothesis. If we call the presence of some of the pieces of evidence, described in the previous section, a "symptom", then let A denote the occurrence of that symptom with the examined phrase. P(AIHy) would be the apriori probability of the symptom A being observed with a noun (verb) phrase which is the center (we will henceforth refer to this factor as Py). By analogy P(AIHN) is the probability of the symptom being observed with a phrase which is not the center (henceforth referred to as PN). The aposteriori probability P(HKtA) is defined in the light of the new piece of evidence the presence of an empirically obtained symptom, indicating the higher probability the examined phrase to be in the center of the discourse segment. In other words, inference ngine based on Bayes' theorem draws an inference in the light of some new piece of evidence. This formula calculates the new probability, given the old probability plus some new piece of evidence. Consider the following situation. According to our investigation so far, the probability of the subject being a center is 73%. Additional evidence (symptom), e.g. if the subject represents adomain concept, will increase the initial probability. If this NP is also the head of the section, the probability is increased further. If the NP occurs more than once in the discourse segment, the probability gets even higher. An estimation of the probability of a subject, (direct or indirect) object or verb phrase (the only possible centers in our texts) to be centers, can be represented as a predicate with arguments: center (X, PI, \[symptoml (weight factorl 1' weight factorl2 ).... symptomN (weight factorN~, weight factolNz)\] ) where center (X, I, list) represents the estimated probability of X to be the center of a sentence (clause), X E {subjec t, objectl, object2 .... verb phrase} and Pl is the initial probability of X to be the center of the sentence (clause). Weight factorl is the probability of the symptom being observed with a noun (verb) phrase which is the center (Py). Weight factor2 is the probability of the symptom being observed with a noun (verb) phrase wiaich is not the center (PN). Following our preliminary results, wc can write in Prolog notation: The first fact means that the object is the center in approximately 25% of the cases. Moreover, it suggests that in 40% of the cases where the center is the object, the verb belongs to the set of verbs {discuss, illustrate, summarize, examine, describe, define...} and it is possible with 3% probability for the verb to be a member of this set while the center of the sentence is not the object. The above Prolog facts are part of a sublanguage knowledge base. The process of estimating the probability of a given phrase being the center of a sentence (clause), is repetitive, beginning with an initial estimate and gradually working towards a more accurate answer. More systematically, the "diagnostic" process is as follows: - start with the initial probability - consider the symptoms one at a time - for each symptom, update the current probability, taking into account: a) whether the sentence has the symptom and b) the weight factors Py and PN. center (object, 25, \[sylnptoln (verb_set, 40, 3), symptom (subject set, 40,2), symptom (domain concept (95, 80), symptom (repeated, 10, 5), symptom (headline, 10, 9)11, symptoln (topicalizexl, 6, 2), symptom (main chmsc (85, 30), symptom (definite.form (90, 7t))\]). center (subject, 73, Isymptonl (domain concept (95, 70), symptom (repeated, 10, 4), symptom (headline, 10, 8), symptom (topicalized, 10, 3), symptom (main_clause (85, 30), symptom (definite_form (85, 20)11). The probability for an NP to be the center is calculated by the inference engine represented as a Prolog program (left out here for reasons of space), which operates on the basis of the sublanguage knowledge base and the "local" knowledge base. The latter gives information on the current discourse segment. Initially, our program works with manual inputs. The local knowledge base can be represented as Prolog facts in the following way: observcd (lmadlinc). observed ( omailL conccl)O. obsmvcd (repeated). The inference engine's task is to match the expected symptoms of the possible syntactic function as center in the knowledge base of the sentence's actual symptoms, and produce a list of (reasonably) tx)ssible candidates. THE PROCEDURE: AN INTEGRATED KNOWLEDGE APPROA Ctt Our algorithm for assigning (proposing) an antecedent to an anaphor is sublanguage-oriented because it is based on rules resulting from studies of colnputer science texts. It is also knowledge-based because it uses at least syntactic, semantic and discourse knowledge. Discourse knowledge and especially knowing how to track the center play a decisive role in proposing the most likely antecedent. The initial version of our project handles only pronominal anaphors. However, not all pronomls may have specific reference (as in constructions like "it is necessary", "it should be pointed out", "it is clear", .... ). So be\[ore the input is given to the anaphor esolver, the pronoun is checked to ensure that it is not a part of such grammatical construction. This function is carried out by the "referential expression filter". The proccdurc for proposing an antecedent o an anaphor operates on discourse segments and can be described itffonnally in the lollowing way: l) Propose the center of the first sentence of the discourse segment using the method described. 2) Use the algorithm proposed in IBrennan et al. 871, ilnproved by an additional estimation of the cotTcct probability supplied by our method, in order to track the center throughout the discourse segment (in case the anaphor is in a complex sentence, identify clause centers too). 3) Use syntactic and semantic constraints to eliminate antecedent candidates. 4) Propose the noun phrase that has been filtered out as the antecedent in case no other candidates have come up; otherwise propose the center of the preceding sentence (clause) as the antecedent. SYSTI '~M PROGRAMS We should note that, unlike user progrants, system pmgran~s such as the supervisor and the language translator should not have to bc translated every lime they are used, olherwisc lhis would result ill a serious increase ill the time spent in processing a user's program. System t)rogr~|lns are usually written ill the assembly version of the machine langtmgc and are tnmslated once into Ihe nladlillC code itself, l;rom thCll oi1 they can be loaded into memory in machine code without the need for any immediate transhuion phases. They are written by specialist programmers, who arc "called system programmers and who know a great deal about the computer and the comlmtcr system 12)1" which their progrmns arc wrincn. They know Ihc exact ntmlber of location which each program will occupy mid in consequence can make use of these mmlbcrs in the supervisor and t l',qllslalor t)rograms. 117,5 The information obtained in 1) and 2) may not be used; however, it may be vital for proposing an antecedent in case of ambiguity. To illustrate how the algorithm works, consider the tollowing sample text: The proposed center of the first sentence is "system programs". The center remains the same in the second, third and forth sentences. Syntactic constraints are sufficient to establish the antecedent of "they" in the third sentence as "system programs". In the forth sentence, syntactic constraints only, however, are insufficient. Semantic constraints help here in assigning "system programs" as antecedent to "they". In the fifth sentence neither syntactic nor semantic constraints can resolve the ambiguity. The correct decision comes from proposing the center of the previous sentence, in this case "system programmers" (and not "programs"!), as the most likely antecedent. CONCLUSION The model proposed has two main advantages. First, it is an integrated model of different types of knowledge and uses existing techniques for anaphora resolution. Second, it incorporates anew approach for tracking the center, which proposes centers and subsequently antecedents with maximal likelihood. Since we regard our results still as preliminary, further research is necessary to confirm/improve the approach/model presented. ACKNOWLEDGEMENT I would like to express nay gratitude to Prof. Pieter Seuren for his useful comments and to the Machine Translation Unit, Universiti Sains Malaysia, Penang, where a considerable part of the described research has been carried out. REFERENCES \[Aonc & MeKee 93\] Ch. Aonc, D. McKee Language-independent a aphora resolution system for understanding multilingual texts. ProceeAings of the 31st Annual Meeting of the ACL, The Olfio State University, Colmnbus, Ohio, 1993 \[Allen87\] J. Allen Natural language understanding. The Benjamin/Cummings Publishing Company Inc., 1987 \[Brennan ct al. 87\] S. Brennml, M. Fridman, C. Pollard - A centering approach to pronouns. Proceedings of the 25th Annual Meeting of file ACL, Statfford, CA, 1987 \[Cmbonell &Brown 88\] J. Carbonell, R. 13town - Anaphora resolution: a multi-strategy approach. Proceedings of the 12. International Colfferencc on Computational Linguistics COLING'88, Budapest, 1988 \[Dahl & Ball 90\] 1). Dahl, C. Ball - Reference resolution in PUNDIT. Research Report CAITSLS-9004, March 1990. Center for Advanced Information Teclmology, Paoli, PA 9301 \[Frederking & Gehrkc 87\] R. Frederking, M. Gchrke - Resolving anaphoric references in a DRT-based dialogue system: Part 2: Focus at~l Taxonomic inference. Siemens AG, WlSBER, Bericht Nr. 17, 1987 \[Grosz & Sidncr 86\] B. Grosz, C. Sidncr Attention, Intention and the Structure of Discourse. Computalional Linguistics, Vol. 12, 1986 \[Hayes 8111 P.J. Hayes - Anaphorafor limited domain systems. Proceedings of the 7th IJCAI, V~mcouver, Canada, 1981 \[Hirst 81\] G. Hirst - Anaphora in natural language understanding. Berlin Springer Verlag, 1981 \[Hobbs 78\] J. IIobbs - Resolving pronoun refere~es. Lingua, Vol. 44, 1978 \[lngria & StaUm'd 8911 R. lngria, D. Stallard -A computational mechanism for pronominal reference. Proceedings of the 27th Annual Meeting of the ACL, Vancouver, British Columbia, 1989 \[Mitkov 93\] R. Mitkov - A knowledge-basedand sublanguage-oriented approach for anaphora resolution. Proceedings of the Pacific Asia Conference on Formal and Computational Linguistics, Taipei, 1993 \[Preug et al. 94l PrcuB S., Schmitz B., Hauenschild C., Umbach C. Anaphora Resolution in Machine Translation. In W. Ramm, P. Schmidt, J. Schiitz (eds.) Studies in Machine Translation and Natural I.~mguagc Processing, Volume on "l)iscoursc in Machine Trmlslation" \[Rich & LupcrFoy 88\] 1~. Rich, S. lalpcrFoy An architecture for anaphora resolution. Proceedings of the Second Conference on Applitxt Natnral l~anguagc Processing, Austin, Texas, 1988 \[Robert 8911 M. Robert - RSsolution de formes pronominales dans l'interface d'interogation d'une base de donndes. Th6sc de doctorat. Facult6 des sciences de Luminy, 1989 \[Sidner 8111 C.L. Sidncr - Focusing for Interpretation of Pronouns. Americau Journal of Computational Linguistics, 7, 1981 \[Walker 89\] M. Walker -- Evaluating discourse processing algorithms. Proceedings of the 27th Annual Meeting of the ACL, Vancouver, Colmnbia, 1989