Language use involves very complex interactions among very diverse types of information, not only syntactic one but also semantic, pragmatic, and so forth. It is hence inappropriate to assume any specific algorithm for syntactic parsing or generation, which prescribes particular processing directions (such as left-to-right, top-down and bottom-up) and is biased for specific types of domain knowledge (such as a context-free-grammar). To account for the whole language use, we will have to put many such algorithms together, ending up with an intractably complicated model.

A better strategy is to postulate no specific algo­rithms for parsing or generation or any particular task, but instead a single uniform computational method from which emerge various types of computation in­cluding parsing and generation depending upon vari­ous computational contexts.

For example, Earlcy deduction (Pereira & Warren, 1983) is a general procedure for dealing with Horn clauses which gives rise to Earley-like parsing when given a context-free grammar and a word string as the input. Shieber (1988) has generalized this method so as to adapt to sentence generation as well. Those methods fail to give rise to efficient computation for a wide variety of contexts, however, because they pre­scribe processing directions such as left-to-right for parsing and bottom-up for generation. They also lack a general way of efficient ambiguity packing unlimited to context-free grammars. Hasida (1994a) proposes a more general inference method for clausal form logic programs which accounts for efficient parsing and gen­eration as emergent phenomena. This method pre­scribes no fixed processing directions, and the way it packs ambiguity is not specific to context-free gram­mars. However, it is rather complicated and has greater computational complexity than standard algorithms do.

In this paper we propose another inference method for Horn logic programs based on Hasida (1994a), and show that efficient parsing and generation emerge from it. Like that of Hasida (1994a), this method is to­tally constraint-based in the sense that it presupposes no fixed directions of information flow, but it is more efficient owing to a parsimonious method of instanti­ation. In Section 2 wo define this inference method, which is a generalization of chart parsing, and may be also thought of as a connection method or a sort of program transformation. Section 3 illustrates how effi­cient parsing and generation emerge from this method without any procedural stipulation specific to the task and the domain knowledge (syntactic constraints). Sec­tion 4 introduces a parsimonious instantiation method for ambiguity packing. We will show that owing to this method the efficiency reaches that of the standard al­gorithms with regard to context-free parsing. Section 5 concludes the paper by touching upon further research directions.

A constraint is represented in terms of a Horn clause program such as below.

(a) -p(A,B) -A=a(C).

(b) p(X,Y) -X=a(Y).

(c) p(U,W)-p(U,V)-p(V,W).

Names beginning with capital letters represent vari­ables, and the other names predicates and functors. The atomic' formulae following the minus sign are neg­ative (body) literals, and the others are positive (head) literals. A clause without a positive literal is called a top clause, whose negation represents a goal (top-level hypothesis), which corresponds to a query in Pro­log. For instance, top clause (a) in the above program is regarded as goal 3A, B,C{p(A, 13) A A = a(C)}. In general, there may be several top clauses. The pur­pose of computation is to tell whether any goal is sat-isfiablc, and if so obtain an answer substitution for the terms (variables) in a satisfiable goal. We consider the minimal Herbrand models as usual. So the set of an­swer substitutions for A in the above program is (a(B), a(a(B)),a(a(a(B))),...}.

A graphical representation of this program is shown in Figure 1. Here each clause is the set of the literals enclosed in a dim closed curve. A link connecting ar­guments in a clause is the term (variable) filling in those arguments. (It is a hyperlink when there arc more than two arguments.) A transclausal link repre­sents the unifiability between two corresponding argu­ments of two unifiable literals. (Neglect the arrows for a while.)

A hypothesis is a conjunction of atomic formulas and bindings. The premise of a clause (i.e., the con­junction of the atomic formulas and bindings which appear as negative literals) is a hypothesis. An ex­pansion for a hypothesis is a way of combining (in­stances of) clauses by resolutions so as to translate the hypothesis to another hypothesis involving bindings only. We will refer to an expansion by the sequence of clauses in the order of leftmost application of res­olution using their instances. In the above program, for example, expansion (c, b, b) translates the top-level hypothesis s(A,B) A A=a(C) to a hypothesis A=a(C) A C=a(B). An expansion of a clause is an expansion of its premise. We will simply say 'an expansion' to mean an expansion of the top-level hypothesis. A program represents a set of expansions, and the computation as discussed later is to transform it so as to ligure out cor­rect hypotheses while discarding the wrong expansions (those entailing wrong hypotheses).

We say that there is a dependency between two terms when those terms are unified in some expansion, and the sequence of terms (including them) mediating this unification is called the dependency path of this dependency. In Figure 1, for instance, the dependency between A and X is mediated by dependency path A X, A-U'X, A'U-U-X, and so on. There is a dependency be­tween C and B, among others, because; of the unifia­bility of the two -«=a(«)s, though this unifiability is not explicitly shown in Figure 1. We say a dependency between two terms is consistent when they are not bound by inconsistent bindings. All the dependencies in Figure 1 are consistent.

A solution of the program is an expansion in which every dependency is consistent. So the computation wo propose in this paper is to transform the given pro­gram in such a way that every dependency be consis­tent. To figure out dependencies, we use a symbolic operation called subsumption, and delete the parts of the program which contributes to wrong expansions only. For example, suppose there is an inconsistent dependency between terms a and ß. We create an instance; ß' of ß by subsumption operations to be dis­cussed shortly, so that every expansion containing an instance of ß' contains an instance of a dependency path between a and ß. We can then delete the clause containing ß' and probably some more parts of the program without affecting the declarative semantics of the program. Below we will define a computational procedure in such a way that the set of the possible expansions eventually represent the set of all the solu­tions.

Subsumption operation is to create subsumption relationship. We regard each part (clause, atomic formula, term, etc.) of a program as the set of its instances, and say that a part £ of the program sub­sumes another part ij to mean that we explicitly know that £ D ij. We consider that a link is subsumed by S if and only if one of the terms it links is subsumed by S. We say term S is an origin of ij when ij is subsumed by S. In this paper we consider that every origin is a bound term (the term filling in the first argument of a binding). Let us say that two clauses (or two liter­als) are equivalent when they are of the same form and for each pair of corresponding terms the two terms have the same set of origins.

Subsumption relation restricts the possibility of ex­pansions so that if term ij is subsumed by another term Ä, then every expansion containing an instance of /; must also contain an instance of S. Subsumption relation is useful to encode structure sharing among expansions. In subsumpfion-based approaches, a term may subsume several non-umfiable terms and thus the first term is shared among the latters. However, that is impossible in unification-based approaches, where different expansions cannot share the same instance of a term or a clause.

A partially instantiated clause is a clause; some of whose terms is subsumed by another term in possi­bly another clause. For instance, (1) a(Ä7.Z) -b(ÄT.ÄJ) -c(Ä7,Z).

is a partial instantiation of the following clause:

(2) a(X.Z) -b(X,Y) -c(Y,Z).

A represents a term subsumed by term A. Hereafter wo say just 'clause' to refer to both uninstantiated clauses and partially instantiated clauses.

A program consisting of such clauses is a gener­alization of a chart (Kay, 1980). A chart is a graph whose nodes denote positions between words in a sen­tence and whose arcs are regarded as context-free rules each instantiated partially with respect to at most two such positions. For instance, an active arc from node i to node j labelled with [A —» • B • C] is an instance of rule A —> BC with both sides of B instantiated by positions i and j. This arc approximately corresponds to (l).'

^This notation is problematic because it is unclear whether two occurrences of A in a clause denote the- same term. In this paper they always do.

Here we show that chart-like parsing and semantic-head-driven generation emerge from the above compu­tational method. We discuss examples of parsing and generation both on the basis of the following grammar.

-vp(Sem,SbjSem,Y,Z).

-v(Sem,SbjSem,ObjSem,X,Y) -np(ObjSem,Y,Z).

(5) np(Sem,X,Y) -Sem=tom -X="Tom"(Y). (G) np(Sem,X,Y) -Sem=mary -X="Mary"(Y).

-Sem=love(Agt,Pat) -X=" loves" (Y).

Since we have already mentioned ambiguity packing in the previous section, below we do not explicitly deal with ambiguity but instead discuss just one sentence structure in both parsing and generation.

Let us first consider parsing of sentence 'Tom loves Mary'. The problem is encoded by the program in Figure 4. The input-driven computation proceeds as shown by the arrows, which represent subsumption op­erations taking place in the ordering indicated by the labelling numbers. A thick dependency path is pro­cessed by successive subsumptions with the same ori­gin. The only subsumption operations executable in the initial situation is the one numbered I and after that the one numbered 2, along the thick path between Ar, and X in (5). As the result of these unfoldings, we obtain the following clauses.

(9) np(Sem,A0,A1) -Sem=tom -Au = "Tom"(Ai).

Of course other partially instantiated clauses may be created here; from definition clauses of s e>tlu;r than (3) and those of np other than (5), but we omit them here and concentrate on just one solution.

Now the copy of link with the arrow numbered 3 connected to (9) can mediate subsumption operations. So the subsumption operation indicated that arrow is triggered, though that does not duplicate (9) because Ai already subsumes the target. The result is already reflected in (9). The subsequent subsumption oper­ations numbered 4, 5, and C will yield the following clauses.

(1.0) s(Sem,Ä7,Z) -np(SbjSernÄ,,ÄT) -vp(Sem,SbjSem,Ai ,Z).

-v(Sem,SbjSem,ObjSem,Ai,Y) -np(ObjSem,Y,Z).

-Ä7—' loves" (Ä7).

Now the subsumption operations by Aj are commenced, due te> the; creation of (12). Accordingly, the; folle>wing clauses are created, and the parsing is finished.

-vp(Sem,SbjSem,Ai ,A3).

-v(Sem,SbjSem,ObjSem,Ai ,A2) -np(ObjSem,Ä7,Ä7).

(15) np(Sem,A2,A.i) -Sem=mary -A2="Mary"(A3).

From the; earlier discussion, in the case e)f context-free parsing the; number of clauses created there; is 0(nM), where n is the number e>f the input wen-els and M the maximum number of the occurrences e>f non-terminal symbols in a context-free rule. This is larger than the; space complexity e>f the; standard pars­ing algorithms, but later we; will show how to improve the efficiency so as to be equivalent to the; standard algorithms.

Ne) particular order among the subsumption oper­ations is prescribed in the above computation, and so it is not ' inherently limited to top-down or bottom-up. Note also that the lefl-to-right processing order among the input words is derived from the definition strong link, rather than stipulated as in Earley deduc­tion, among others. We; can account for island-driven parsing as well, by allowing links between bindings te) trigger subsumptions more earlier.

Let us next take a look at sentence generation. Consider the; program shown in Figure; 5. Here the input is seiuautic structure love(tom,mary). Again the; computational process is indicated by the; numbered arrows. (>' take's place after (3, but the order among 0, 7, and 6' is arbitrary as long as G should be; before 7. So the only possible subsumption operation in the beginning is the ones by Love, which go through the thick curve connecting Love and the X in (4). This creates the following clause, among others.

~Love=love(Tom,Mary) -X=" loves"(Y).

The parsing process discussed above is computation­ally more complex than chart parsing. Here we im­prove our method by introducing a more efficient scheme for ambiguity packing and thus reduce the parsing complexity to that of chart parsing, which is 0(n) for space and 0(n) for time.

The present inefficiency is due to excessive multi­plication of clauses: much more partially instantiated clauses are created than arcs in a chart. So let us suppose that a subsumption operation does not dupli­cate a whole clause but only some part of it, so that a clause is copied incrementally, as shown in Figure (i. We assume that a subsumption to an argument of a

"JSo the semantic-lieael-etriven génération parallels better with left-to-right parsing than with syntactic-heael-driven parsing.

°The semantic monotonicity is practically same as the inpvit-boundness with regard to semantic structures.

literal copies the term filling in that argument, the lit­eral, and some other literals which mention that term, unless there have already been the terms and literals to be thus created. Subscript i of a literal indicates that it is created by the i-th subsumption operation.

We must ensure that this partial copying be se­mantically equivalent to the copying of whole clauses. That is a trivial business when there are just one or two literals in the original clause. The case where there are more than three literals reduces to the case where there are exactly three literals, by grouping several literals connected directly (through terms) and treat them as if they were one literal. So below let its consider the case where there are three; literals in a clause.

A non-trivial check must be done in such a case; as in the lower right of Figure C. Here you must copy -r(«,«)2 and -q(.,.)j but not -q(»,«), because -r(«,»)2 is compatible with -q(»,«)i but not with -q(.,.). We say that a set of literals are compatible when there is an instance of the clause which involves an instance of each of those literals. Also, two literals are said to be heterogeneous when they have different originals in the original uninstantiated (lause. (The; original of an original literal is itself.) In general, when a sub­sumption operation copies two heterogeneous, directly connected literals and creates two directly connected literals, the necessary and sufficient condition for this partial copy to 1»; semantically equivalent to the full-clause; ce>py is obviously that the; former two literals be; compatible.

When two of the; original literals are iK)t connected directly with each other, twe> heterogeneous literals which have; directly connected originals are; compat­ible iff theîy are; alse) directly connected; we; nee;el not consider two literals whose originals are not directly connected, because; one subsumption operation does ne>t copy such literals at a time. When all of the three original literals are connected directly with each other, two heterogeneous literals are compatible iff the;y are; connected not only directly but also through another literal heterogeneous to both. In fact, -r(«,«)2 anel -q(»,«)i are connected both through term £ and through p(«,«)2, but -r(»,«)2 anel -q(«,«) are; not con­nected through any instance; of the original p(«,»).

In the; e-ase; of context-free parsing, 0(u) literals are; created, whe;re n is the number of words in the in­put string, provided that the origins of subsumptions are; the positions between the input words only, elue; to the input-driven computation. Since; there are; just a constant times more links than literals, the space com­plexity of context-free parsing hence becomes 0(n) in our method. The time e'.e>mple:xity is 0(n), because there are 0(n) different ways e>f making each literal. Now the correspondence with chart parsing is more; ex-ae:t. An are: in the; chart corresponds te> an instantiated literal. For instance, are: [A —» • B • C] from node i to node j corresponds to instantiated literal -b(A;,A.,), and [A —> • B C •] from node i to node j corresponds to a(Ai,Aj). For a context-free ride with more: than two symbols in the right-hand side, we can group sev­eral literals to erne fis mentioned above and reduce it to a rule: with just two symbols in the right-hand side:.

We have proposed a flexible inference method for Horn logic programs. The computation based on it is a sort of program transformation, and chart parsing and semantic-head-driven generation are epiphenomena emer­gent thereof. The proposed method has nothing spe­cific to parsing, generation, context-free grammar, or the like. This indicates that there is no need for any special algorithms of parsing or generation, or perhaps any other aspect of natural language processing.

The idea reported above has already been partially implemented and applied to spoken language under­standing (Nagao, Hasida, k Miyata, 1993), and an account of how the roles of speaker and hearer may switch in the midst of a sentence (Hasida, Nagao, k Miyata, 1993). Although this line of work has incor­porated a notion of dynamics (Hasida, 1994b) as the declarative semantics to control context-sensitive com­putation, we arc planning to replace dynamics with probability. For input-bound programs together with input-driven computation, it is quite straightforward to define probabilistic semantics as a natural exten­sion of stochastic context-free grammars, among oth­ers, because all the body literals are probabilistically independent in that case. We would like to report soon on a general treatment of probabilistically dependent literals while preserving the efficient structure sharing, which will guarantee efficient computation and learn­ing.

Reference

Hasida, K., Nagao, K., k Miyata, T. (1993). Joint Utterance: Intrasentential Speaker/Hearer Switch as an Emergent Phenomenon. In Ba-jesy, R. (Ed.), Proceedings of the. 13th Interna­tional Joint Conference on Artificial Intelligence Chambéry.

with Incremental Copy

Ilasida, K. (1994a). Common Heuristics for Parsing, Generation, and Whatever .... In Str/.alkowski, T. (Ed.), Reversible Grammar in Natural Lan­guage Processing. Kluwer Academic Publisher, Dordrecht.

Hasida, K. (1994b). Dynamics of Symbol Systems. New Generation Computing, 12('i). to appear in May 1994.

Kay, M. (1980). Algorithm Schemata and Data Struc­tures in Syntactic Processing. Tech. rep., XE­ROX Palo Alto Research Center, Palo Alto, Cal­ifornia.

Nagao, K., Hasida, K„ k Miyata, T. (1993). Under­standing Spoken Natural Language with Omni­Directional Information Flow. In Proceedings of the l'Jth International Joint Conference on Ar­tificial Intelligence.

Pereira, F. C. N„ k Warren, D. II. D. (1983). Parsing as Deduction. In Proceedings of the 21sl Annual Meeting of ACL, pp. 137 144.

Shieber, S. M. (1988). A Uniform Architecture for Parsing and Generation. In Proceedings of the 12th International Conference on Computational Linguistics, pp. 014-619.

Shieber, S. M., van Noord, G., k Moore, R. C. (1989). A Semantic-Head-Driven Generation Algorithm for Unification-Based Formalisms. In Proceed­ings of the 27th Annual Meeting of the Associa­tion for Computational Linguistics, pp. 7 17.