Rigid Lambek Grammars Are Not Learnable from Strings

Rigid Lambek grammars are not learnable from strings

Annie Foret and Yannick Le Nir

IRISA Rennes, FRANCE

e-mail: foret@irisa.fr and ylenir@irisa.fr

This paper is concerned with learning catego rial grammars in Gold's model (Gold, 1967). Recently, learning algorithms in this model have been proposed for some particular classes of classical categorial grammars (Kanazawa, 1998). We show that in contrast to classical categorial grammars, rigid and k-valued Lambek grammars are not learnable from strings. This result holds for variants of Lambek calculus ; our proof consists in the construction of limit points in each class. Such a result aims at clarifying the possible directions for future learning algorithms.

Categorial grammars have been studied in the field of natural language processing, classical (or basic) categorial grammars were introduced in (Bar-Hillel, 1953) ; here we focus on Lambek categorial grammars (Lambek, 1958) to which linear logic introduced by Girard (Girard, 1995) is closely connected. These grammars are lex-icalized grammars that assign types (or categories) to the lexicon; they are called k-valued, when each symbol in the lexicon is assigned to at most k types; they are also called rigid when 1-valued. Such k-valued grammars are of particular interest in recent works on learnability (Kanazawa, 1998) (Nicolas, 1999). The issue of extending Kanazawa's work to other categorial grammars has been raised and has become an active area of research. In this context, it is important to acquire a good understanding of the properties of the class of grammars in question.

In this paper we consider the following problem, is the class of rigid Lambek grammars learnable from strings. Learning (in the sense of Gold (Gold, 1967)) in our context is a symbolic issue that may be described as follows. Let Q be a class of grammars, that we wish to learn from examples. The issue is to define an algorithm, that when applied to a finite set of sentences, yields a grammar in the class that generates the examples; the algorithm is also required to converge. Formally, let C(G) denote the language associated with grammar G. and let V be a given alphabet, a learning algorithm is a function <fi from finite sets of words in V* to Q, such that for G € Q with C(G) = (ei)iGjv there exists a grammar G' € Q and there exists no € N such that : Vn > no <j>{{ei,..., e„}) = G' € Q with jC(G') = C(G).

One good reason to use categorial grammars in a learning perspective is that they are fully lexicalized : the rules are already known, only types assigned to words have to be derived from examples.

The paper is organized as follows. Section 2 addresses background definition and known results. We then proceed from one version of Lambek calculus to the other. Section 3 gives the initial construction and proof for Lambek calculus allowing empty sequences. Section 4 adresses the construction for Lambek calculus without empty sequence including products. Section 5 concludes.

In this section, we introduce basic definitions concerning categorial grammars. The interested reader may also consult (Casadio, 1988; Retore, 2000; Buszkowski, 1997; Moortgat, 1997) for an introduction or for further details. Let S be a fixed alphabet.

Types. Types are constructed from Pr (set of primitive types) and three binary connectives

/ , \ and • for products. Tp denotes the set of types. Pr contains a distinguished type, written S, also called the principal type.

Categorial grammar. A categorial grammar over £ is a finite relation G between £ and Tp. If < a. A >C G, we say that G assigns A to c, and we write G :ci4i.

We give a formulation of Lambek calculus, written L, including products consisting in introduction rules on the left and on the right of a sequent. For Lambek calculus without products, one simply drops the rules for •.

Lambek Derivation h l- The relation is the smallest relation h between Tp+ and Tp, such that for all F,F' C Tp. A. A' C Tp* and for all A,B (E Tp :

AhAA.YhBT,A\-B\r-frThA\BThB / Ar hAA,B,A'\-CI"hAA,B,A'\-C-v -nA.Y.A\B.A!h CA.B / A.V.A'hCA.A.B.A'hCA,(A• B),A'hC•II-hAT'h B T,T' h(A»B)

When we replace Tp by Tp* in F C Tp+ in the definition above, we get another version of Lambek calculus, without the non-empty left hand-side requirement, which we refer to as L§ with derivation relation h£0 ■ Note. We recall that the cut rule is satisfied by hi and hl0 . Note also that Fh^C implies Language. Let G be a categorial grammar over X. G generates a string c\ ... cn C £+ iff there are types A\,..., An € Tp such that : G : q h4- (1 < i < n) and A\,... ,An h l S . The language of G, written £l(G) is the set of strings generated by G. We define similarly CLfl(G) replacing h L with h i0.

Notation. In some sections, we may write simply h instead of \~l or l-£0 . We may simply write £(0) accordingly.

Rigid and ft-valued grammars. Categorial grammars that assign at most k types to each symbol in the alphabet are called k-valued grammars; 1-valued grammars are also called rigid grammars.

Example 1 Let Si = {John, Mary,likes} and let Pr = {S, N} for sentences and nouns respectively. Let G\ = {John N, Mary ^ N, likes ^ N \ (S / N)}. We get (John likes Mary) C Cl(G\) since (N, N\(S I N), N hLS). G\ is a rigid (or 1-valued) grammar.

For ease of proof, in next section we use models of L (or L$) that we now recall : powerset residuated semi-groups (or monoids), a special case of residuated semi-groups (see (Buszkows-ki, 1997) for details).

Powerset residuated semi-groups and monoids. Let (M,.) be a semi-group (. is associative). Let V(M) denote the powerset of M. A powerset residuated semi-group over (M,.) is the structure (V(M), o, =^, ^=, C) such that for X,Y CM :

X oY = {x.y : x (E X,yGF}X = {y (E M : (\/x (E X)x.yCY}Y<=X = {y€M : (Vx€X)y.xCY}

If (M, o) is a monoid (. is associative, I is a unit that is : Va; € M : I.x = x.I = x), then the above structure is a powerset residuated monoid (it has {/} as unit).

Interpretation. Given a powerset residuated semi-group (V{M), o, =^, -4=, C), an interpretation is a map from primitive types p to elements [\p\] in V(M) that is extended to types and sequences in the natural way :

lit'. \C2]] =[[d]]=►[[C2]]llC, /C2]] =[[Ci]]4=[[C2]][[Cl.C2]] = [[Cl]]o[[C2]][[C1,C2,...,Cn}} = [[C1}}o[[C2]}. ... o[[Cn}}

The following known property states that such structures are models for L : if F \~l C then [[F]] C [[C]]

If(M,.) is a monoid with an identityI,we add [[A]] ={1}for the empty sequence A and get a similar property forL§ :if Fl-£0Cthen [[F]]C[[Cj]

We now recall some useful definitions and known properties on learning.

Limit points. A class CC of languages has a limit point iff there exists an infinite sequence < Ln >n£N of languages in CC and a language L G CC such that : Lq C L\ ... Ç ... C Ln C ... and L = U„c,\ Ln (L is a limit point of CC).

Limit points imply unlearnability. The following property is important for our purpose. If the languages of the grammars in a class Q have a limit point then the class Q is unlearnable.

Definition. We define the following grammars where p and q are primitive types : G(i,n) = {a P I p; q I q; c ^ £>(!,„>} Language. We get (see proof) C(G(i „)) = c(b*a*)n and C(G{1^) = c(b*a*)* = c{a,b}*.

where -D{i,o)=SandD(i>n)=(-D(i,n-i> /(p / p)) / (q / q)G(i,*) ={a^PIP;p I p;cSj(pjp)}

Notation. Let (and t^^) denote the type assignment by (by respectively) on {a, b, c} extended on {a, b, c}* to sequences of types in the natural way; we write r= T(itn) on {a, b}* (indépendant of n > 0).

Key points. We use two main key ideas : tautologies of the Lambek calculus allowing empty sequences that ensure one way of type-derivability (-D{ilTl) l~ D(i,n-i)) ; alternation of two such tautologies that are unrelated (non-interderivable : t(o), t(6) \f t(o) or t(o),t(6) \f r(b) although we have T(a),T(a) h t(o)), this alternation prevents derivabilities in the other direction (-D{i,n-i) 1/ D^^). We thus provide a strictly infinite chain of types. Note that n denotes a bound of these alternations.

This implies that the class has infinite elasticity. A class CC of languages has infinite elasticity iff 3 < e, >,v ; \ sentences 3 < Li >,\- ; \ languages in CCWi eN :e,^Liand{ei,...,en}C

Ln+i (see (Kanazawa, 1998) for this notion and a use of it).

For the class of rigid ^-grammars.

- This yields a strictly increasing chain of language of rigid grammars.

- This shows that the class of rigid grammars has infinite elasticity (cf (Kanazawa, 1998) for details).

- This class also has a limit point as follows c{o, b}* which entails that this class is not learnable from strings.

Other restricted subclasses

- The same results hold if we restrict to a bounded order, where the order o(A) is :

o(p) =0 whenpis a primitive typeo(Ci \ C2) =max(o(Ci)+ l,o(C2))o(C2/C1)=o(d \ C2) ;

- this also holds for unidirectional grammars (we do not use \ ).

Our proof is based both on a syntactic reasoning on derivations, and on models.

Proposition 1 (Language description)

£(£(!,„)) = c(b*a*)n and C(G{i^}) = c{a,b}*.

proof ofc(b*a*)nCC{G{hn))

For n = 0 this is an axiom t^1i0)(c) = S h S.

Supposen>0 andw'= c.wCE£(G^li„_1^),

- we first show that c.b.a.w G C(G(iiTlA :

Dnt„-i),r(w)h5p/php/p-IID(l,n-l)I(PIP)>(P /p),t(w)hSg/ghg/g-II(g(i,n-i)/ (P / P))/ (g / g).(Pj Phr{w)h5= -D(l,n>=T(l,n>(<:)=r(b) =r(a)- we easily getc.wGC(G^i^)fromD(iin)l~D^iin forn >0 ; (more generallyCi/ (C2IC2)h CiinL0);- we also getc.a.wGC(G(iiTlAfrom D{i,n),p/phD{i,„-i\(since(D{hn)h-D(iiTl_i) /(p/ p)and -D{i,„-i)/ (p/ p),p / pi"£>(i,„-i>)

- we then get c.b.w G C(G/iiTlA since - finally, this is extended to repetitions of each letter a or b separately since T(a),T(a) h t(o) and r(b), r(b) h r(b) (if we replace each occurrence of a with a repetition of a or each occurrence of b with a repetition of b we still get an element of the language of (?(i)7l)).

D(ii7l),q/ q= {D(i,n-i)I(pIP))I(qIq),QIQi"D(l,n-l)I(PIP)i"D{l,n-1);

proof of £((?(!,„)) C c(6*o*)n (main part) We consider the standard linguistic interpretation (Buszkowski, 1997) : the powerset residu-ated monoid (T(V*), o, =^, -4=, C) over the free monoid (V*,.) where . is the concatenation operation and V* is the set of strings over the alphabet V = {a, b, c}.

Let us fixn(arbitrary), we define an interpretation as follows :[[S]]= c(b*a*)n,[[p]] = a*,[[q]} = b*.Let us supposeT(i>n)(w)hS.By models, we have [[t(i)71)(iw)]]C[[5]]. We first remark that [[p /p}} =a*and[[q j q}} =b*(sincel\p/p}}={z€ V* :€ [[p]wC[[p]]} =

£ 7* : £ a*,z.x C a*}).

We now show by induction on i that :V*(0<i<n):[[-D(M)]] =c{b*a*)n-1- casei=0<nholds since [[D^0)]] =[[S]}=c(b*a*)n- case (0 <i<n):[[(0<U-i>/(P/P))]]= {z£r:V3;£[[p/p]]I«£[[%I„1)]]}=md{z £ F* : Vi £ o*,z.a;Cc^a*)^"1))}= P{M-i>]](= c^o*)^-^1))) [[%,,)]] = [[(%,,-!)/(P/P))/ (9/ 9)]]={z£P:V|/£[[g/g]],^£[[%,-ii/(p/p)]]}={z€ : Vy €&*,s.yCc(6*o*)(n+1-1)} = {z £ 7*: Vn £Cc(6*o*)^n~i\b*.a*}

(from above)

= c(6*a*)("_I)(as desired)

We have thus shown that = c.

Therefore if [[t(i)71)(iw)]] C [[5]] this also means that iu = cu/ with w' € {a, 6}* ; we get c.[[T(lin)(«/)]] C c(b*a*)n = \[S}}, that is HT{i,7i)(w')]] ^= (b*o*)" that corresponds to w' C (6*o*)n as well (since [[t^ „)(o)]] = [[p / p]] = o* and[[r(1,n)(6)]] = [[9/g]] = 6*).

proof ofc{o,6}*C£(C?(i^))

- We have c C £(G(i^)) since 5 / (p / p) S.

- We now get coC£((7^^), since :p/p\-p/pSh 5

S/ (p/p),p/pi-S The other cases are straightforward since p / p,p / p^ p / p.

proof of £((7^^) C c{o, 6}* We consider the powerset residuated monoid {V{V*),o,=>, ^=,C) as above but with the following (similar) interpretation :

[[S]]=c{a,by,[[p]]=a\([[<?]] = &*)•Let us supposet^^^(w) h 5. By models, we have [[t^^(«;)]]C[[S]}.We have : [[p /p]] =a*We here get :[[S/(p/ p)]] = c.{o,b}*( = {z£: G[[p / p]],z.xC[[5]]} = {z £ F* : £o*, z.xCc.{o,6}*})Therefore if [[r(M)(«;)]]C[[5]] = c{o,6}*, this also means thatw=cw'withw'C{o, 6}* as desired ■4 Rigid limit points forLKey points:We haveD{1;7l)hi0-D{i,„-i)but

D(ii7l) Vl We then transform the type of c from T(1)r!^ and construct a limit point for L (with products).

Construction for L

We define the following types and assignmentst(2)7l),whereA = p / p, B = q / qand p, (? are primitive types :6—>■B;c—>■D(2,n)where D^2o)=^andD(2)n>' = (((D(2in-i>/ a) • a) /b)• b ]£n >0

Let G(2,n) denote the grammar defined by T(2,n) with alphabet {a, b, c}.

Let (7(2,*)denote the grammar, with type assignmentT(2l*}defined by :G{2,*) ={a4A;6-)• A;c -)• (57/l) •/l}

We get (see proof) £((7(2,n}) = c(6*o*)n and £(G{2,*}) = c{a,b}*.

Proposition 2 (Language description) £(G{2,n}) = c(6*o*)n and jC(G(2,*>) = c{o,6}*.

proof of c( - finally, this is extended to repetitions of each letter a or b separately since T(a),T(a) h t(o) and t(6),t(6) h t(6)

c£(G{2,„))For n = 0,t(2i0)(c) =Sh 5. Suppose n > 0 and «/ = c.wC£(G^2,n-i})- we first show thatc.b.a.wC£(G^2,n})(^2,n-l>/i)-A,i,rM(w)hSB,B,h B{{{D(2,n-i)IA). A)IB).B;^ ' 4^'T<2.«> (w)h5' =!>(2,„>=r(2,„>(c) '=T(t>)=T(<l3- we easily getc.wc£(G^2,n})sinceD(2,n)I"D(2,n-l)in^for" > °-- we also getc.a.w€ £(G^2,n))from^(2,7i>)^I"-D{2,n-1)(since D(2)„) h(£>(2,n-i> / ^) •Aand (£>(2ln-i> /-4) •a,AhD/2,n-i>)- we then getc.b.wc£(G(2n}) since£>(2,n>, B = (((D(2ln-1> /4) '•a)/b)»b,b^(D{2,n-i) /A)•AhD(2tn_1};proof of£(G(2,n))Cc(6*o*)n

We first show that I~l0 -D{2,n) by induction on n :

- case n = 0 is obvious since -0(1,0)=-^{2,0)=S- casen> 0 : we recall thatA= p / p,B = q j qare tautologies ofL$;by induction :D{ltn_1}hL0£>(2,n-i>we then get inL$ :D{l,n-l) ^D{2,n-l)AhAf(1,71-1) /A,Ah£»(2,71-1)D(i,.-i)/^l-D(2,,-i)/^«hB(■^(1,71-1)IA)IB, BV-£»(2,7i-i) / ^(£>(l,n-l>IA)IB, BV-(£»(2,«-!)M) • ^(£>(l,n-l>/ A) / B\-((£»(2,n-l)I A)* A)IBhB(£»(l,n-l)/ A) / B\-((£»(2,n-i)/ A)mA) / B)mB

We may now end the proof : suppose that T(2,n>(w) l-£ 5, we get T(iiTl)(«;) hi0 5 from previous property, therefore w c c(6*o*)n from proposition 1 as desired.

proof ofc{a,b}*C£(C?{2^))

- We have c c jC(G(2i*>) since (5 / A) • A -L S.

- We also getcac£((7^^), and similarly for co* orc{a,b}*since ^4,^4/l(whereA = PI»•proof of£(G(2^)Cc{a,b}*

This part follows from the construction for

L%sincet^^c)hi0t^,*) (c) as follows :S/jll-i,(SM) ^>S/^i-i,{SI A).Ahence^^^(w)hi0T(2)*)(w)andt(2,*)(«0\~l5 impliesT(i^)(w)hi05 ■

5.1 Non-learnability for subclasses.

From the constructions in previous sections we get the following proposition as a corollary :

Proposition 3 (non-learnability) The class of languages of rigid (or k-valued for an arbitrary k) Lambek grammars admits a limit point; the class of rigid (or k-valued for an arbitrary k) Lambek grammars is not learnable from strings.

Note. Our result has shown that in contrast to classical categorial grammars, rigid and k-valued Lambek grammars are not learnable from strings. This result holds for variants of Lambek calculus : (i) rigid Lambek grammars for L§ (allowing empty sequence) using only / (unidirectional) ; (ii) rigid Lambek grammars for L (with the non-empty left requirement) using ( / , •) . Our proof consists in the construction of limit points in each class : for (i) and (ii) we have c{o, b}* as limit point that are indeed rigid lambek grammars as already shown. In the first construction, we may also consider grammars of a bounded order.

5.2 Further remarks.

A stricly infinite chain of types. In each construction, the types Dn = D(ilTl) for c are such that : ... h Dn h D„_i h ... V D0. This is the key point for the inclusion of languages. Moreover Dn_i \f Dn, which is reflected by the strictness of inclusion of languages. Our construction has thus exhibited a stricly infinite chain of types with respect to deduction for each version of Lambek calculus.

Future work. We are also interested in other variants of categorial grammars, such as the non-associative version of Lambek calculus or Multi-modal categorial grammars for which the construction of rigid limit points could be extended.

We also plan to consider not only the string languages but some structure languages in a grammatical inference perspective. In (Bona-to and Retore, 2001), it has been shown that rigid Lambek grammars are learnable in the limit from stuctured sentences. Such structures, that are studied in (Tiede, 1999), are complete proof tree structures in normal forms. In our result, we show that, without any structure, some classes of Lambek grammars are not learnable in the limit from positive examples. It will now be very interesting to find an intermediate structure, that is more linguistically realistic than complete proof tree structures but that preserves the learnability in the limit of Lambek grammars.

Y. Bar-Hillel. 1953. A quasi arithmetical notation for syntactic description. Language, 29:47-58.

Roberto Bonato and Christian Retore. 2001. Learning rigid lambek grammars and minimalist grammars from structured sentences. Third workshop on Learning Language in Logic, Strasbourg.

W. Buszkowski. 1997. Mathematical linguistics and proof theory. In van Benthem and ter Meulen (van Benthem and ter Meulen, 1997), chapter 12, pages 683-736.

Claudia Casadio. 1988. Semantic categories and the development of categorial grammars. In R. Oehrle, E. Bach, and D. Wheeler, editors, Categorial Grammars and Natural Language Structures, pages 95-124. Reidel, Dordrecht.

Jean-Yves Girard. 1995. Linear logic: its syntax and semantics. In Jean-Yves Girard, Yves Lafont, and Laurent Regnier, editors, Advances in Linear Logic, volume 222 of London Mathematical Society Lecture Notes, pages 1-42. Cambridge University Press.

E.M. Gold. 1967. Language identification in the limit. Information and control, 10:447-474.

Makoto Kanazawa. 1998. Learnable classes of categorial grammars. Studies in Logic, Language and Information. FoLLI & CSLI. distributed by Cambridge University Press.

Joachim Lambek. 1958. The mathematics of sentence structure. American mathematical monthly, 65:154-169.

Michael Moortgat. 1997. Categorial type logic. In van Benthem and ter Meulen (van Benthem and ter Meulen, 1997), chapter 2, pages 93-177.

Jacques Nicolas. 1999. Grammatical inference as unification. Rapport de Recherche RR-3632, INRIA.

http: / / www.inria.fr/RRRT / publications-eng.html.

Christian Retoré. 2000. Systèmes déductifs et traitement des langues:un panorama des grammaires catégorielles. Technical Report RR-3917 2000, INRIA, Rennes, France. A revised version to appear in Traitement automatique du langage naturel, TSI.

Hans-Jörg Tiède. 1999. Deductive Systems and Grammars: Proofs as Grammatical Structures. Ph.D. thesis, Illinois Wesleyan University.

J. van Benthem and A. ter Meulen, editors. 1997. Handbook of Logic and Language. North-Holland Elsevier, Amsterdam.