R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms a basic commonly agreed formalization, a the issue of nonmonotonicity is not addressed in the work described in this article.

Knowledge representation languages are evolving toward more declarativity, as exemplified by the evolution from KL-ONE (Brachman and Schmolze 1985) to languages such as GLASSIC (Borgida et al. 1989) or LOOM (MacGregor 1988, 1990). The terminological component describing the objects (the data model of object-oriented database systems) has always been more declarative than the assertional component (procedural attachment or methods), and the current trend is to integrate those two components more closely, where the assertional component is some kind of rule-based system, as in LOOM (Yen, Neches, and MacGregor 1988).

Typed feature structures are very similar to structured objects of object-oriented languages and to conceptual structures of knowledge representation languages. Thus, typed feature structures have the potential to act as a lingua franca for both computational linguistics and artificial intelligence, and this should ease the communication between those two worlds. Since conceptual structures are used for example in text generation (Bourbeau et al. 1990) or knowledge-based machine translation (Nirenburg et al. 1992), typed feature structures provide an attractive alternative to current procedural implementations.

In Section 2, we present a language that combines the notions of partial information and inheritance in a fully declarative framework. It is based on feature structures augmented with the notion of types, which are organized into an inheritance network. Using types, it is possible to define structured domains of feature structures and to classify feature structures. Logical conditions are attached to types, akin to method attachment, but in a fully declarative framework. Recursivity is an integral part of the language, giving the necessary expressive power for describing complex recursive linguistic structures. We end the section by an overview of the TFS abstract rewrite machine used for computing descriptions of the meaning of typed feature structures. 3

Section 3 describes the use of inheritance in two examples of unification grammars using the TFS formalism: an HPSG grammar for a fragment of English and an LFGstyle transfer grammar for a small machine translation problem between English and French.

Assume the existence of an (abstract) informational domain U, for example, the set of linguistic objects. Feature structures describe objects of this universe by specifying values for attributes of objects and equality constraints between some values. More precisely, as feature structures can provide only partial information about the objects they describe, a feature structure denotes a set of objects in this universe. This set could be a singleton set, for example, in the case of atomic feature structures. Feature structures are ordered by a subsumption relation: a feature structure fl subsumes another feature structure f2 iff fl provides the same or less information than f2: fl _> f2. In our universe, this means that the set described by fl is a superset of the set described by f2. Note that there can be feature structures that cannot consistently describe the 2 The KR logic (Rounds and Kasper 1986), from which most of other logic formalizations of feature

structures are derived, plays this role in formal accounts of feature structures. 3 This formalism is fully implemented. An interpreter for the TFS rewrite machine has been

implemented at the University of Stuttgart by Martin Emele and the author and has been used to test

several linguistic models such as DCG, LFG, HPSG, and SFG \[Emele and Zajac 1990b; Emele et al.

1990; Zajac 1990a; Bateman and Momma 1991\]. Computational Linguistics Volume 18, Number 2 same objects. For example, a feature structure describing verb phrases and a feature structure describing noun phrases are not consistent: the intersection of the sets they denote is usually empty.

As different sets of attribute-value pairs make sense for different kinds of objects, we also divide our feature structures into different ypes. These types are ordered by a subtype relation: a type t2 is a subtype of another type tl if t2 provides at least as much information as tl. For example, assuming that a verb phrase is a phrase, then the set of verb phrases is included in the set of phrases. Using types to model this taxonomic hierarchy, the type symbol VP denotes the set of verb phrases, the symbol PH denotes the set of phrases, and we define VP as a subtype of I'H.

This description implies that, if we know that a linguistic object is a verb phrase, we can deduce that it is a phrase. This deduction mechanism is expressed in our type system as type inheritance. Furthermore, with each type we associate constraints expressed as feature structures, thereby defining an inheritance network of typed feature structures: ifa feature structure is of type t and tlhere exist supertypes of t, then t inherits all the attribute-value pairs and equality constraints of the feature structures associated with all the supertypes of t.

Computation isperformed by a typed feature structure machine capable of checking a set of type constraints defined as an inheritance network of typed feature structures. Given a typed feature structure inheritance network, we query the machine by asking if some feature structure satisfies the constraints defined by the network. To produce the answer, the system proceeds by gradually adding the constraints that should be satisfied by the query: an answer will be a set of feature structures where each feature structure is subsumed by the query and where all the type constraints of the network hold on all substructures ofthe elements of the answer. The answer is the empty set when the query does not satisfy the constraints defined by the network. Related work. The basic approach described in this section is based on original work by Ait-Kaci (1984, 1986) on the KBL language and has also been influenced by the work on HPSG by Pollard and Sag (1987) and Pollard and Moshier (1990). Among the growing literature on the semantics of feature structures, many relevant results and techniques have been published by Smolka (1988, 1989), Smolka and A'/t-Kaci (1988), and AitKaci and Podelski (1991). Based on Pollard and Sag (1987), Pollard (1990), and Pollard and Moshier (1990), a computational formalism, very close to the TFS formalism, is currently under design at CMU for implementing HPSG (Carpenter 1990, Franz 1990).

The early work presented in Emele and Zajac (1989a) was an attempt o directly implement he ideas presented in Ait-Kaci (1984, 1986); Emele and Zajac (1989b) was already a departure from Ait-Kaci's KBL language, implementing a different evaluation strategy that allowed the use of cyclic feature structures. Compared with the KBL language, the current TFS language • allows the use of cyclic feature structures, • has a simple operational semantics implementing an inheritance rule and a specialization rule, • uses a lazy evaluation scheme for the evaluation of constraints, • implements tatic coherence checks, • has a simple syntax distinguishing the definition of the partial order on type symbols, and the definition of constraints associated to types. R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms

ti <_ t, and 2. for all substructures u = \[XIA\[...\] of ti, type A is a minimal type and u ~ def(A).

The first condition says that all the elements of St satisfy the constraints expressed by t. The second condition says that all the elements of ,St satisfy the constraints defined Computational Linguistics Volume 18, Number 2 LIST LIST APPEND \[!: APPEND : LIST\] : LIST.\] : LIST\[ : LIST\] NIL CONSIfirst: T

Lrest: LIS'\[ APPEND0/2: m L: ~T

rst:N CO lSLrest: \[\] \[\] LIST : CONS\[ first:N

\[rest:IN APPEND1 Figure 3 Rewrite rules for LIST and APPEND. by the network. If St is empty, the feature structure t is inconsistent (modulo the constraints of the inheritance network). St can be finite, e.g. in the case of a dictionary, but it can also be infinite in the case of recursive types: for example, the set of feature structures subsumed by LIST is the (infinite) set of all possible lists represented as feature structures,

In this section, we describe an abstract rewrite machine for computing the representation of the denotation of typed feature structures given an inheritance network. The rewrite mechanism is based on a variant of narrowing 9adapted to feature structures.

An inheritance network of feature structures is compiled into a rewriting system as follows: each direct link between a type A and a subtype B generates arewrite rule of the form A\[a\] --* B\[b\] where A\[a\] and B\[b\] are the definitions of A and B, respectively. Figure 3 shows the rewrite rules corresponding to the network of Figure 2.

The interpreter is given a "query" (formulated as a typed feature structure) to evaluate. The first step is to check that the feature structure respects the two typing rules (Section 2.3.1). The idea is then to try to satisfy all the constraints defined by the inheritance network by incrementally adding more constraints to the query using the rewrite rules (nondeterministically) toget closer to the solution step by step. The rewriting process stops when conditions 1 and 2 described above hold.

A rewrite step for a structure t is defined as follows: if u is a substructure of t at path p and u is of type A, and there exists a rewrite rule A\[a\] --, B\[b\] such that • A\[a\] A u 3_, and • A\[a\] A u < A\[a\] then the right-hand side B\[b\] is unified with the substructure uat path p, giving a new structure t' that is more specific than t (Figure 4). R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms Figure 4 A rewrite step.

The first condition checks that the rule is applicable: the 1.h.s. has to be consistent with the substructure. The implementation factorizes common 1.h.s., avoiding the DNF expansion: if a 1.h.s. l of a rule l --* r is not consistent with the substructure, this computation branch is a failure branch, and all rules u ~ v where u < I are discarded in one step without further computation.

The second condition implements a lazy rewriting strategy: if A\[aJ A u is equal to A\[a\], all rules A\[a\] --* B\[b\] could be applied with success, and failure could come only from the rewriting of some other substructures after the exploration of all choices for u. To avoid the exploration of failure branches as much as possible, the evaluation of the substructure u is suspended until the evaluation of some other substructure having some part in common with u makes u more specific, narrowing the set of potential choices for the subtypes of A for u. Thus, the search space is explored "intelligently," postponing the evaluation of branches of computation that would correspond for example to uninstantiated PROLOC goals (see for example van Hentenryck and Dincbas \[1987\], van Hentenryck \[1989\] on evaluation techniques in constraint logic programming).

Rewrite steps are applied nondeterministically everywhere in the structure until no further rule is applicable. 1°

The choice of which substructure to rewrite is only partly determined by the availability of information (using the lazy rewriting rule). When there are several substructures that could be rewritten, the computation rule is to choose one of the outermost ones, i.e., one closest to the root of the feature structure (innermost strategies are usually nonterminating). This outermost rewriting strategy is similar to hyperresolution in logic programming. In comparison, PROLOG uses a leftmost computation rule.

For a given substructure, the choice of which rule to apply is done nondeterministically, and the search space is explored depth-first using a backtracking scheme. Although this strategy is not complete (a complete breadth-first search strategy could be used for debugging purposes), the use of the outermost rule has favorable termination 10 Conditions do not change this general scheme (they are evaluated using the same rewriting

mechanism) and are omitted from the presentation here for the sake of simplicity. See for example

Dershowitz and Plaisted (1988) and Klop (1990) for a survey on rewriting. Computational Linguistics Volume 18, Number 2 properties when compared to PROLOG'S leftmost rule: there are problems where a TFS computation will terminate when the corresponding logic program implemented in PROLOG will not; for example, for left-recursive rules in naive PROLOG implementations of DCGs. 3. Inheritance and Constraint-Based Grammars

• A unique general constraint solving mechanism is used: grammars define constraints on the set of acceptable linguistic structures. • As a consequence, there is no formal distinction between "input" and "output." For example, the same kind of data structure could be used to encode both the string and the structural description, and, as for the HPSG sign (Pollard and Sag 1987), they could be embedded into a single data structure that represents the relation between the string and the associated linguistic structure. • Specific mapping properties, based on constituency, linear precedence, or functional composition, are not part of the formalism itself, but can be encoded explicitly in the formalism.

An approach that uses a unique deductive mechanism for parsing and generation is described in Dymetman and Isabelle (1988). Within this approach, a lazy evaluation mechanism based on the specification of input/output arguments is implemented (in PROLOG), and the evaluation is completely data-driven: the same program parses or generates, depending only on the form of the input structure.

A constraint-based grammar does not need a context-free mechanism to build up constituent structures for parsing or generation: Dymetman, Isabelle, and Perrault (1990) describe a class of reversible grammars ("Lexical Grammars") based on a few composition rules that are very reminiscent of categorial grammars. Other kinds of approaches have been proposed, e.g., using a dependency structure and linear precedence relations (Reape 1990; see also Pollard and Sag \[1987\]). In Saint-Dizier (1991), linear precedence rules are defined as constraints in a language based on typed feature structures and SLD-resolution, which is used to experiment with GB theory.

In the following sections, we describe two examples of constraint-based grammars: an HPSG grammar for a fragment of English, and an LFG-style transfer grammar for a small machine translation problem between English and French.

R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms

\[sYH I Loc f SuBcAT ~ \] \[DTRS headed..~trueture\[ \] \]:~'\[DTRs\[HEAD-DTR I SYN \] LOC l SUBCAT append(ID,l~)\] \[ \[COMP-DTRS ~\] J PHRASE < SUBCAT-FP. t dtrs: \[comp-dtm: ~\[~ \[3: ~J Figure 5 The HSPG subcategorization principle and its TFS encoding. of definitions as in Figure 2.11 HPSG is so far the only formal linguistic theory based on the notion of typed feature structures (Pollard 1990), and is thus a good candidate to illustrate the possibilities of the TFS formalism. The following presentation isbased on Emele (1988) and Emele and Zajac (1990b).

The basic linguistic object in HPSG (Pollard and Sag 1987) is a complex linguistic structure, the "sign," with four levels of description: phonology, constituent structure, syntax, and semantics. In HPSG, there is no distinction between "input" and "output:" the relation between a string and a linguistic structure is encoded as a single feature structure representing the "sign."

HPSG "principles" are encoded using inheritance: a feature structure of type PHRASE inherits the constraints associated with types SUBCAT-FP, HEAD-FP, and SEM-FP. For example, the HPSG subcategorization principle is encoded in TFS using inheritance to model implication, and the TFS APPEND relation to encode the functional constraint on concatenation (Figure 5). In a similar way, HEAD-FP encodes the HPSG Head Feature Principle and SEM-FP encodes the Semantics Principle.

The type SIGN is divided into several subtypes corresponding to different mappings between a string and a linguistic structure. We first have the basic distinction between phrases and words. The definition of a phrase recursively relates subphrases and substrings, and defines the phrase as a composition of subphrases and the string as the concatenation of substrings. Since the formalism itself does not impose any constraints on how the relations between phrases and strings are defined, the grammar writer has to define them explicitly. In HPSG (Pollard and Sag 1987), the ordering of phrases is defined using linear precedence relations: the order in which the substrings associated with subphrases are concatenated to give the string associated with a phrase are guided by these linear precedence relations (Reape 1990).

In the example given below (Figure 6), we make simplifying assumptions: the LOCAL feature is not used and there are two possible orderings for complements. The type IDP1 encodes Grammar Rule 1 (Pollard and Sag 1987, pp. 149-155), which says that a "saturated phrasal sign," i.e., a feature structure with \[syn : \[subcat :(>\]\], is the combination of an unsaturated phrasal head with one phrasal complement on the left. For example, for structures like S ~ NP VP, S is the "saturated phrasal sign," NP is 11 We will use a more condensed notation for lists with angle brackets provided by the TFS language: a

list C0NS\[first: Mary, rest: CONS \[first : sings, rest: NIL\]\] is writtenas <Mary sings>.

Computational Linguistics Volume 18, Number 2 SUBCAT-FP HEAD-FP SEM-:FP \[phon: \[sl~Dubcat: 0 \]

Fphon: B \]

/syn: \[subcat: (SIGN)\] \], J \] IDP2/. \[hea(l-dtz: WOl~D\[phon: m \] L Lcomp-dt~s: (SXGN\[phon: El\]) JJ .'~ APPEND : :- APPEND : L3: ~J Figure 6 Part of the HPSG PHRASE hierarchy: PHRASE inherits from "principles" (given here without their definitions) and is subdivided into two subtypes corresponding to different complement orderings. the left phrasal complement and VP is the unsaturated phrasal head. Furthermore, the string (the value of the phon feature) of the IDP1 phrase is the concatenation of the string of the complement with the string of the head.

The type IDP2 encodes Grammar Rule 2 and states that an "unsaturated phrasal sign," i.e., a feature structure with \[syn:\[subcat:lSIGNI\] \], is the combination of a lexical head with any number of complements on the r~ht (e.g., for VP --* V XP*): the string associated with IDP2 is the concatenation of the string of the head with the concatenation of the strings of the complements, where the relation ORDER-COMP defines in which order the complements strings are concatenated.

The difference between the parsing and the generation problem is then only in the form of the structure given to the interpreter for evaluation. A query for the parsing problem is an underspecified structure where only the string is given; conversely, a query for the generation problem is an underspecified structure where only the semantic form is given (Figure 7).

In both cases, the interpreter uses the same set of rewrite rules to fill in "missing information" according to the type definitions. The result in both cases is exactly the same: a fully specified structure containing the string, the full semantic form, and also all other syntactic information such as the constituent structure (Figure 8). 3.3 Bi-directional Transfer in Machine Translation We have sketched above a very general framework for specifying mappings between a linguistic structure, encoded as a feature structure, and a string, also encoded as a feature structure. We apply a similar technique for specifying transfer ules for machine translation, which we prefer to call "contrastive rules" since there is no directionality involved (Zajac 1989; 1990a). R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms Query for parsing PHILASE\[phon: (Elm ate every cookie)\] Query for generation Paa.asz : |~gl:~,-d:\[r,~tr:\[n~e: KXM\]\]\] _\]

\[ / g2:|spec: Li.a: \[r--tr:\[ra": EVERY COOKm\]\]\] Figure 7 Queries for parsing and generation. phon: (Kim ate every cookie)

L subcat: 0

__\[comp-dtrs: (...) dtrs: TR~:~\[head.dtr: IDP2..,

'zeln: EAT J IDP1 sem: 2ARG-REL \[m h°r rein: NAMING m lr,.,r" • 111 / / / \[ kname: KIM J J J Figure 8 The common solution to the parsing and generation problems.

The idea is rather simple: assume we are working with linguistic structures imilar to LFG's functional structures for English and French as proposed in Kaplan et al. (1989). We define a translation relation as a type TAU-LEX with two features, eng for the English structure and fr for the French structure. This "bilingual sign" is defined on the lexical structure: each subtype of TAU-LEX defines a lexical correspondence between a partial English structure and a partial French structure for a given lexical equivalence. Such a lexical contrastive definition for a verb also has to pair the arguments recursively. This is expressed in the condition part of the definition (Figure 9) by a recursive condition TAU-LEX on the arguments. The translation of syntactic and semantic features, like tense or determination, isalso specified in the condition part, and these contrastive definitions can and should be defined separately from the lexical definitions as different subnetworks.

Inheritance is used here to capture generalizations over relations: all the information in the example of Figure 9 can be unpacked and redefined as follows. We have Computational Linguistics Volume 18, Number 2 TAU-LEX-FALL ~ TAU-LEX-ITV.

/eng:/subj: l \[tense: TAU-LEX-FALL \] ipred: TOMBEI 1 . I~: \[subj: ~ L L tense: Figure 9 A transfer ule. /\ TAU TAU-LEX... TAU-TENSE... U L /Feng:rtense le'tea*el U eng ~reng subj ~ eng ~reng pred FALL\] Figure 10 Part of an inheritance network of transfer elations for verbs. three levels of generalization for the translation of verbs: TAU-LEX-V is subdivided into several relations for translating, for example, transitive verbs (TAU-LEX-TV), intransitive verbs (TAU-LEX-ITV), etc. The condition that specifies the translation of tenses is defined for the whole class of verbs TAU-LEX-V. The condition that specifies the translation relation between subjects is defined for intransitive verbs (TAU-LEX-ITV), but cannot be specified for all verbs since it does not apply to, e.g., impersonal verbs. This leaves only the translation of predicates to be defined at the leaves of the hierarchy (Figure 10). Thus, at each level of generalization, we specify the minimal amount of information needed for translation. The same kind of organization can be used for nouns, where syntactic features uch as determination are associated with the higher types, and where the minimal types define the equivalences between lexical forms of nouns themselves. R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms Query FA '. / rPre(l: STUDENT \] / / TAU eng: isubj: / B,ea: A 1/// / Ln"": s GJJ Ltense: E-PRES 11 \[ / J J Answer TAU-LEX-FALL PrecI: FALL \] \] rPre(i: STUDENT 11 o-s:/,.bj: ! \[p,~: A \]/! / L*~:: L,,,,m: ~_s~JJ /

Ltense: F-PRES J

rp,ed: TOMBER \]

\[ rp,ed: ETUDIANT \] \[ ~: i \[subj: \[ \[pred:UN "1// L='>~:: L"u": F'S~GJ J / Ltense: F-PRES J Figure 11 Query and answer for the translation of "A student falls."

The transfer problem for each direction is then stated in the same way as for parsing or generation: the input structure is an underspecified "bilingual sign" where only the structure for the source language is given. Using the contrastive grammar, the interpreter fills in missing information and builds a bilingual sign 12 (Figure 11).

It is not necessary to specify in the contrastive definitions all monolingual constraints that have to be satisfied by the English structure and by the French structure. We can assume that we have monolingual grammars that define the appropriate mappings between the set of English sentences and the set of associated English structures, and similarily for French. Using these monolingual constraints in addition to the contrastive grammar, the TFS interpreter would build the fully specified monolingual structures, implementing a constraint-based translation system. 3.4 Termination Problems For parsing and generation, since no constraints are imposed on the kind of mapping between the string and the semantic form, termination has to be proved for each class of grammar and for the particular mechanism used for either parsing or generation with this grammar. If we restrict ourselves to classes of grammars for which terminating evaluation algorithms are known, we can implement hose directly in TFS. However, the TFS evaluation strategy allows more naive implementations ofgrammars, and the outermost rewriting of "sub-goals" terminates on a strictly larger class of programs than for corresponding logic programs implemented inPROLO~. Furthermore, the grammar writer does not need, and actually should not, be aware of the control that follows the shape of the input rather than a fixed strategy, thanks to the lazy evaluation mechanism. Computational Linguistics Volume 18, Number 2

For HPSG-style grammars, completeness and coherence as defined for LFG, and extended to the general case by Wedekind (1988), are implemented in HPSG using the "subcategorization feature principle" (Johnson 1987): for the TFS implementation of HPSG, termination is guaranteed, at least for the simplified version containing only head-complement structures, described in Section 3.2. Termination conditions for parsing are well understood in the framework of context-free grammars. For generation using feature structures, one of the problems is that the input could be "extended" during processing, i.e., arbitrary feature structures could be introduced in the semantic part of the input by unification with the semantic part of a rule. However, if the semantic part of the input is fully specified according to a set of type definitions describing the set of well-formed semantic structures (and this condition is easy to check), this cannot arise in a type-based system since it is not possible to add arbitrary features to a typed feature structure.

A more general approach is described in Dymetman, Isabelle, and Perrault (1990), who define sufficient termination properties for parsing and generation for the class of "Lexical Grammars." These termination properties are conditions on the existence of "conservative guides" for parsing and generation and seem generalizable to other classes of grammars as well, and are also applicable to TFS implementations. Since Lexical Grammars are implemented in PROLOG, lefbrecursion must be eliminated for parsing and for generation, but this does not apply to TFS implementations. The idea of conservative guides is relatively simple and says that for parsing, each rule must consume a nonempty part of the string, and for generation, each rule must consume a nonempty part of the semantic form. These conditions seem to be equivalent as to require the existence of a well-founded relation on strings (for parsing) and of a well-founded relation on semantic forms (for generation). The existence of such well-founded relations is actually a necessary condition for proving the termination of parsing and generation (see Deville \[1990\] for a more general discussion on wellfounded relations in the context of logic programming).

Termination for reversible transfer grammars is discussed in van Noord (1990). One of the problems mentioned there is the extension of the "input," as in generation, and the answer is similar (see above). However, properties imilar to the "conservative guides" of Dymetman, Isabelle, and Perrault (1990) have to hold in order to ensure termination. 4. Conclusion The TFS system has been developed to provide a computational environment for the design and the implementation of formal models of natural language. The TFS formalism is designed as a specification language that can be used to design and implement formal linguistic models. It is not a programming language: it does not offer means of defining control information that would make execution more efficient (but less general), as it would be needed if it would be envisaged to use the system in an application-oriented environment (e.g., as a parser in a natural language interface to a database system). From formal linguistic models developed in TFS, it could be envisaged to develop programs, i.e., parsers or generators, that would efficiently implement the declarative knowledge contained in the formal specifications. 13

R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms

The TFS system is implemented using rewriting techniques in a constraint-based architecture adapted to feature structures: • • • The language is a logical language directly based on typed feature structures, and supports an object-oriented style based on multiple inheritance. Grammars are expressed as inheritance networks of typed feature structures. They define constraints on the set of acceptable linguistic structures. As a consequence, there is no formal distinction between "input" and "output." A unique general constraint solving mechanism is used. Specific mapping properties, based on constituency, linear precedence or functional composition, are not part of the formalism itself, but can be encoded explicitly using the formalism.

Although the current implementation isvery much at the level of an experimental prototype, and is still evolving, it has validated the basic concepts of the language and of the implementation, and has been used to test different linguistic models and formalisms such as LFG, DCG, HPSG, and SFG on small examples. From these various experimentations, we have defined extensions and improvements, both on the language and on the implementation, that are needed for scaling up the system.

On the language side, more expressivity is needed. For example, disjunctions over feature structures, various kinds of negation (Ait-Kaci 1986), and sets of feature structures (Pollard and Moshier 1990) are necessary to formalize, e.g., nontrivial semantic structures. Some types together with a specific syntax and associated operations could be conveniently added to the system as libraries of built-in types, e.g., characters, strings, and trees.

On the implementation side, the use of implementation techniques adapted from PROLOG implementations, constraint satisfaction languages, and object-oriented languages can be beneficial to the implementation of typed feature structure-based systems and have to be more thoroughly explored.14One of the major efficiency issues in the current implementation isthe lack of an efficient indexing scheme for typed feature structures. For example, since the dictionaries are accessed using unification only, each entry is tried one after the other, leading to an extremely inefficient behavior with large dictionaries. Thus, the use of a general indexing scheme based on a combination of methods used in PROLOG implementations and in object-oriented atabase systems is necessary and will be implemented in a future version of the system. Acknowledgments The design and the implementation ofthe TFS system have been carried out in cooperation with Martin Emele. I would like to thank Stefan Momma and Ulrich Heid; their numerous comments and advices helped to make this article more readable. I would also like to thank anonymous referees for their many detailed and helpful comments. All remaining errors are of course the sole responsibility of the author. 14 For example, the use in the current implementation f several techniques adapted from PROLOG

implementations such as structure sharing, chronological dereferencing (Emele 1991) and last call

optimization, have improved efficiency by several orders of magnitude over previous "naive"

implementations. Computational Linguistics References Ait-Kaci, Hassan (1984). "A lattice theoretic

approach to computation based on a

calculus of partially ordered type

structures." Doctoral dissertation,

University of Pennsylvania, Philadelphia,

PA. A'/t-Kaci, Hassan (1986). "An algebraic

semantics approach to the effective

resolution of type equations." Theoretical

Computer Science, 45, 293-351. Ait-Kaci, Hassan, and Meyer, Richard

(1990). Wild_LIFE, a user manual.

DEC-PRL Technical Note PRL-TN-1,

Rueil-Malmaison, France. Ait-Kaci, Hassan, and Podelski, Andreas

(1991). Towards a meaning of LIFE.

DEC-PRL Research Report PRL-RR-11,

Rueil-Malmaison, France. Bateman, John A., and Momma, Stefan

(1991). The nondirectional representation

of Systemic Functional Grammars and

Semantics as Typed Feature Structures.

IMS Technical Report, University of

Stuttgart, Germany. Biggerstaff, Ted J., and Perlis, Alan J. (eds).

(1989). Software Reusability. ACM

Press-Addison-Wesley. Birkoff, Garrett (1940). Lattice Theory.

American Mathematical Society, third

edition, 1973. Borgida, Alexander; Brachman, Ronald J.;

McGuinness, Deborah L.; and Resnick,

Lori Alperin (1989). "CLASSIC: a

structural data model for objects." In

Proceedings, 1989 ACM SIGMOD

International Conference on Management of

Data, Portland, Oregon. Bouma, Gosse (1990). "Non-monotonic

inheritance and unification." In

Proceedings, Workshop on Inheritance in

Natural Language Processing, Institute for

Language Technology and AI, Tilburg

University, The Netherlands. Bourbeau, L.; Carcagno, D.; Goldberg, E.;

Kittredge, R.; and Polgu6re, A. (1990).

"Bilingual generation of weather forecasts

in an operation environment." In

Proceedings, 13th International Conference on

Computational Linguistics (COLING'90).

Helsinki. Brachman, Ronald J., and Schmolze, James

G. (1985). "An overview of the KL-ONE

knowledge representation language."

Cognitive Science 9, 171-216. Carpenter, Bob (1990). "Typed feature

structures: inheritance, (in)equality and

extensionality." InProceedings, Workshop on

Inheritance in Natural Language Processing,

Institute for Language Technology and

AI, Tilburg University, The Netherlands. Volume 18, Number 2 Daelemans, Walter (1990). "Inheritance and

object-oriented natural anguage

processing." In Proceedings, Workshop on

Inheritance in Natural Language Processing,

Institute for Language Technology and

AI, Tilburg University, The Netherlands. Dershowitz, N., and Plaisted, D. A. (1988).

"Equational programming." In Machine

Intelligence 11, edited by Hayes, Michie,

and Richards. Oxford: Clarendon Press. De Smedt, Koenraad, and de Graaf, Josje

(1990). "Structured inheritance in

frame-based representation oflinguistic

categories." In Proceedings, Workshop on

Inheritance in Natural Language Processing.

Institute for Language Technology and

AI, Tilburg University, The Netherlands. Deville, Yves (1990). Logic programming.

Systematic Program Development. Reading,

MA: Addison-Wesley. Dymetman, Marc, and Isabelle, Pierre

(1988). "Reversible logic grammars for

machine translation." In Proceedings, 2nd

International Conference on Theoretical and

Methodological Issues in Machine Translation

of Natural Language. Pittsburgh, PA. Dymetman, Marc; Isabelle, Pierre; and

Perrault, Francois (1990). "A symmetrical

approach to parsing and generation." In

Proceedings, 13th International Conference on

Computational Linguistics (COLING'90).

Helsinki. Emele, Martin (1988). "A typed feature

structure unification-based approach to

generation." In Proceedings, WGNLC of the

IECE. Oiso University, Japan. Emele, Martin (1991). "Unification with lazy

non-redundant copying." In Proceedings,

29th Annual Meeting of the ACL. Berkeley,

CA. Emele, Martin, and Zajac, R6mi (1989).

"RETIF: A rewriting system for typed

feature structures." ATR Technical report

TR-I-0071, ATR, Kyoto. Emele, Martin, and Zajac, R6mi (1989).

"Multiple inheritance in RETIE" ATR

Technical report TR-I-0114, ATR, Kyoto. Emele, Martin, and Zajac, R6mi (1990a). "A

fixed-point semantics for feature type

systems." In Proceedings, 2nd Workshop on

Conditional and Typed Rewriting Systems

(CTRS'90). Montr6al, Qu6bec. Emele, Martin, and Zajac, R6mi (1990b).

"Typed unification grammars." In

Proceedings, 13th International Conference on

Computational Linguistics (COLING'90).

Helsinki. Emele, Martin; Heid, Ulrich; Momma,

Stefan; and Zajac, R6mi (1990).

"Organizing linguistic knowledge for

multilingual generation." In Proceedings, R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms

13th International Conference on

Computational Linguistics (COLING'90).

Helsinki. Etherington, David W.; Forbus, Kenneth D.;

Ginsberg, Matthew L.; Israel, David; and

Lifschitz, Vladimir (1989). "Critical issues

in nonmonotonic reasoning." In

Proceedings, 1st International Conference on

Principles of Knowledge Representation ad

Reasoning. Toronto, Ontario. Evans, Roger, and Gazdar, Gerald (1989).

"Inference in DATR." In Proceedings, 4th

European ACL Conference. Manchester, U.K. Franz, Alex (1990). "A parser for HPSG."

CMU report CMU-LCL-90-3, Laboratory

for Computational Linguistics, Carnegie

Mellon University. Fraser, Norman M., and Hudson, Richard

A. (1990). "Word grammar: An

inheritance-based theory of language." In

Proceedings, Workshop on Inheritance in

Natural Language Processing. Institute for

Language Technology and AI, Tilburg

University, The Netherlands. Halliday, M. A. K. (1985). Introduction to

Functional Grammar. London: Edward

Arnold. Huet, G6rard (1976). Rdsolution d'dquations

dans les langages d'ordre 1, 2,..., w. Doctoral

dissertation, Universit6 de Paris VII. Johnson, Mark (1987). "Grammatical

relations in attribute-value grammars." In

Proceedings, West Coast Conference on Formal

Linguistics, Vol. 6. Stanford, CA. Kaplan, Ronald M.; Netter, Klaus;

Wedekind, J~irgen; and Zaenen, Annie

(1989). "Translation by structural

correspondences." In Proceedings, 4th

European ACL Conference. Manchester, U.K. Kay, Martin (1984). "Functional unification

grammar: A formalism for machine

translation." In Proceedings, l Oth

International Conference on Computational

Linguistics (COLING-84). Stanford, CA. Klop, Jan Willem (1990). "Term rewriting

systems." To appear in Handbook of Logic

in Computer Science, Volume 1, edited by

S. Abramsky, D. Gabbay and T. Maibaum.

Oxford University Press. MacGregor, Robert M. (1988). "A deductive

pattern matcher." In Proceedings, 7th

National Conference on Artificial Intelligence

(AAAI'88). St. Paul, MN, 403-408. MacGregor, Robert M. (1990). "LOOM user

manual." USC/ISI Technical Report,

Marina del Rey, CA. Mann, William C., and Matthiessen,

Christian I. M. I. (1985). "Demonstration

of the Nigel text generation computer

program." In Systemic Perspectives on

Discourse, Volume 1, edited by James

D. Benson and William S. Greaves.

Norwood, NJ: Ablex. Nirenburg, Sergei, Carbonele, Jaime, Tomita,

Masaru, and Goodman, Kenneth (1992).

Machine Translation. AKnowledge-Based

Approach. San Mateo, CA: Morgan

Kaufmann. Pollard, Carl J. (In press). "Sorts in

unification-based grammar and what they

mean." In Unification in Natural Language

Analysis, edited by M. Pinkal and

B. Gregor. Cambridge, MA: The MIT

Press. Pollard, Carl J., and Moshier, Drew (1989).

"Unifying partial descriptions of sets." In

Information, Language and Cognition, edited

by P. Hanson, Vancouver Studies in

Cognitive Science 1. Vancouver:

University of British Columbia Press. Pollard, Carl J., and Sag, Ivan A. (1987).

Information-Based Syntax and Semantics.

Volume 1: Syntax. CSLI Lecture Notes 13,

Chicago University Press. Pollard, Carl J., and Sag, Ivan A.

(Unpublished research). Information-Based

Syntax and Semantics. Volume 2: Agreement,

Binding and Control. Reape, Mike (1990). "Parsing semi-free

word order and bounded discontinuous

constituency and "shake 'n' bake"

machine translation (or 'generation as

parsing')." International Workshop on

Constraint Based Formalisms for Natural

Language Generation, Bad Teinach,

Germany. Rounds, Williams C., and Kasper, Robert T.

(1986). "A Complete Logical Calculus for

Record Structures Representing Linguistic

Information." IEEE Symposium on Logic in

Computer Science. Saint-Dizier, Patrick (1991). "Processing

language with logical types and actives

constraints." In Proceedings, 5th Conference

of the European Chapter of the ACL. Berlin,

Germany. Shieber, Stuart (1986). An Introduction to

Unification-based Grammar Formalisms. CSLI

Lectures Notes 4, Chicago University

Press. Smolka, Gert (1988). "A feature logic with

subsorts." LILOG Report 33, IBM

Deutschland GmbH, Stuttgart. Smolka, Gert (1989). "Feature constraint

logics for unification grammars." IWBS

Report 93, IBM Deutschland GmbH,

Stuttgart. Smolka, Gert, and Ait-Kaci, Hassan (1988).

"Inheritance hierarchies: Semantics and

unification." J.Symbolic Computation, 7,

343-370. Stoy, Joseph E. (1977). Denotational Semantics: Computational Linguistics

The Scott-Strachey Approach to Programming

Language Theory. Cambridge, MA: The

MIT Press. van Hentenryck, Pascal (1989). Constraint

Satisfaction i Logic Programming,

Cambridge, MA: The MIT Press. van Hentenryck, Pascal, and Dincbas,

Mehmet (1987). "Forward checking in

logic programming." In Proceedings, 4th

International Conference on Logic

Programming. Melbourne, Australia. van Noord, Gertjan (1990). "Reversible

unification-based machine translation." In

Proceedings, 13th International Conference on

Computational Linguistics (COLING'90).

Helsinki. Wedekind, Jiirgen (1988). "Generation as

structure driven generation." In

Proceedings, 12th International Conference on

Computational Linguistics (COLING'88).

Budapest. Whitelock, Pete (1990). "Shake-and-bake Volume 18, Number 2

translation." Ms. Sharp Laboratories of

Europe, Oxford. Yen, John; Neches, Robert; and MacGregor,

Robert (1988). "Classification-based

programming: Adeep integration of

frames and rules." Technical Report

ISI/RR-88-213, USC/Information Science

Institute. Zajac, R6mi (1989). "A transfer model using

a typed feature structure rewriting system

with inheritance." In Proceedings, 27th

Annual Meeting of the ACL. Vancouver, B.C. Zajac, R6mi (1990a). "A relational approach

to translation." In Proceedings, 3rd

International Conference on Theoretical nd

Methodological Issues in Machine Translation

of Natural Language. Austin, Texas. Zajac, R6mi (1990). "Semantics of typed

feature structures." International Workshop

on Constraint Based Formalisms for Natural

Language Generation. Bad Teinach,

Germany. R6mi Zajac Inheritance and Constraint-Based Grammar Formalisms Appendix: Syntax of the TFS Language In the TFS language, there are two kinds of comments. In-line comments begin with a semicolon and end with the end of line. These comments can appear anywhere where a white space character is allowed in the syntax. They are skipped during reading. Syntactic comments begin and end with the % character. These comments can appear only where specified in the BNF syntax specification. They are attached to the structure produced by the reader and can be displayed if the appropriate printer option is set. Identifiers are case-sensitive. Macros are expanded statically by the compiler. The operator ~ is interpreted as the meet (unification).

The following extensions of the BNF notation are used: • \[X\] denotes the optional element X (zero or one occurrence). • X* denotes the free iteration of element X (zero, one or more occurrences). • X + denotes the iteration of element X (one or more occurrences). • Symbols in typewriter font denote symbols of the TFS syntax. <entity> ::= <query> I<definition> <query> ::-- .9 <expression> . <definition> ::= <po-definition> I <macro-definition> \] <type-definition> <po-definition> ::= <type-symbol> <<type-symbol> . <macro-definition> ::= <identifier> "= <expression> . <type-definition> ::-- <type-symbol> <expression> :- <conditions> . <conditions> ::= <expression> \[, <conditions> \] <expression> ::= <feature-structure> \[ &<expression> \] <feature-structure> ::=<comments> <tagged-feature-structure> <tagged-feature-structure> ::= <tag> \[ = <typed-feature-structure> \] I

<typed-feature-structure> <typed-feature-structure> ::-- <type-symbol> \[ <attribute-value-matrix> \] I

<attribute-value-matrix > I

<list> <attribute-value-matrix> ::= \[ \[ <attribute-value-pairs> \] \] <attribute-value-pairs> ::=<attribute-value-pair> \[ , <attribute-value-pairs> \] <attribute-value-pair> ::=<attribute> : <expression> <list> ::= < <expression>* \[ . <expression> \] > <tag> ::= #<identifier> <type-symbol> ::= <identifier> <attribute> ::= <identifier> Example 1: the textual definitmns ~r F~u~ 2. NIL < LIST. CONS < LIST. CONS\[first: T, rest: LIST\]. APPEND\[I: LIST, 2: LIST, 3: LIST\]. Computational Linguistics Volume 18, Number 2 APPENDO< APPEND. APPENDO\[I: NIL, 2: #I=LIST, 3: #i\]. APPEND1 < APPEND. APPEND1\[1: <#x #ii>, 2: #12, 3: <#x . #13>\]

• -APPEND\[I: #ii, 2: #12, 3: #13\]. Example 2: the textual definitions ~r F~ure 10. TAU-LEX < TAU. TAU-LEX-V < TAU-LEX. TAU-LEX-V\[eng: \[tense: #e-tense\],

fr: \[tense: #f-tense\]\]

:- TAU-TENSE\[eng: #e-tense, fr: #f-tense\]. TAU-LEX-ITV < TAU-LEX-V. TAU-LEX-V\[eng: \[subj: #e-subj\],

fr: \[subj: #f-subj\]\]

• - TAU-LEX\[eng: #e-subj, fr: #f-subj\]. TAU-LEX-FALL < TAU-LEX-ITV. TAU-LEX-FALL\[eng: \[pred: FALL\],

fr: \[pred: TOMBER\]\]