2.1 The Core Language Engine, CLE The Core Language Engine is a general purpose natural language processing system for English developed by SRI Cambridge. It is intended to be used as a building block in a broad range of applications, e.g. data-b~.se query systems, machine translation systems, text-to-speecb/speech-to-text sysems, etc. The object of the CLE is to map certain natural language xpressions into appropriate predicates in logical form (or Quasi-Logical Form \[Alshawi ,(.: van Eijck 89\]). The system is based completely on tmi lication and facilitates a reversible phrase-structure type grammar.

The Swedish Institute of Computer ,'qci(m(e has with support from 8RI generalized the fi'anwwork and developed all equivahmt system for Swedish (the S-CLE, \[Gamback & Rayner 92\]). The two copies of the CLE have been used together to form a machine translation system \[Alshawi et a191\]. The S-('LE has a fairly large gramnmr covering most of the common constructions in Swedish. There is a good treatment of inflectional morphology, covering all main inflectional closes of nouns, verbs and adjectives.

The wide range of l)ossihle applications have put severe restrictions on the type of lexicon that can be used. The S-CLE h~ a function-word lexico~J containing about 400 words, including most Swedish pronouns, conjllnctlous, prepositions, determiners, particles and "special" verbs. In addition, there is a "core" content-word lexicon (with common nouns, verbs and adjectives) and domain specitic h'xica. This part of tbe system is still under development and all these content-word lexica together haw, about 750 entries.

The lexical entries contain information about il~flectional morphology, syntactic and semantic subcategorization, anti sortal (selectional) restrictions. Information abont the linguistic properties of an entry is represented by complex categories that include a principal category symbol and specifications of constraints on the values of syntactic/semantic features. Such categories also appear in the C.LF,'s grammar and matching and merging of the information encoded in them is carried out by unification during parsing. Two categories can be unified if the constraints on their feature values are compatible

In the actual "core" and domain Icxica, this information is kept implicit and represented as pointers to entries in a "paradigm" lexicon with a number of words representing basic word usages and inflections. ACTES DE COLING-92, NANTES, 23-28 AO~r 1992 1 1 7 3 I'gOC. OF COLING-92, NAN'rES, AUG. 23-28, 1992 For these "paradigm words" only, the complete set of feature vahles is explicitly specified. 2.2 The Vocabulary EXpander, VEX In the English CLE, new lexicon entries can be added by tile users with a tool developed for the purpose. q'his lexicon acquisition tool, the Vocabulary EXpander, is fully described in \[Carter 89\]. In parallel with the development of the S-CLE, a Swedish version of the VEX system was designed \[Gamback 92\].

VEX allows for the creation of lexical entries by users with knowledge both of a natural language and of a Sl)ecilic application domain, but not of linguistic theory or of tile way lexical entries are represented in the CLE. It presents examl)le sentences to the user and asks lor information on tile grammaticality of the sentences, and for selcctional restrictions on arguments of predicates VEX adopts a copy and edit strategy in colmtrnctiug Icxical entries. It builds on the "paradigm" lexicon and sentence patterns, that is, declarative knowledge of the range of sentential contexts ill which the word usages in that lexicon Call OCCUI'.

In the present work we want to investigate to what extent snch creation of lexicon entries can be performed with a minimum of user interaction, lnstead of presenting exaruple sentences to the user we are allowing the program to use a very large text where hopefully unknown words will occur in several ditlbrenl sentence patterns. This strategy will he filrther described i~, the following sections.

First, however, we will define what we mean by the notion of (subcategorization) "paradigm". Tile definition we adopt here is based on the one used in \[Carter 89\], namely that Definition 1 a paradigm zs any minimal non.empty intersection of Icxical entries. Every category in a pa,'adlgm will occur in czaclly the same set of entries in the lexicon as every other category Of auy) in that paradigm. Every ent,y consists of a dis3o2ul union of paradigms.

lh're, we assume that a lexicon can be described in terms of (a small set of) sucb paradigms, relying on ttle fact. that the open-class words exhibit at least approximate regularities) 2.3 The Lexicon Learning system, L 2 Previous experiments in automatic lexical acquistilion at. S1CS (L ~ - Lexicon Learning) used a set of

1 The system does not attempt to cope with c|oaed-categc)ry words. '\['hey have to be entered into a apecific function-word lexicon by a skilled linguist. sentences and a formal grammar to infer the lexical categorit.'-s of the words in the sentences. The original idea wa.q to start with an empty lexicon, assuming that the grammar would place restrictions on the words in the sentences ufficient to determine an assignment of lexical categories to them \[Rayner el al 88\]. This can I)e viewed as solving a set of equations where the words are variables that are Io be assigned lexical categories and the constraints that all sentences parse with respect to the grammar are the equations.

Unfortunately, it proved almost impossit,le to parse sentenees containing several nnknown words. For this reason the scheme was revised in several ways \[tlgrmander 88\]; instead of starting with an eu/pty lexicon, the starting point bccanw, a lexicon coutaining clnsed-cl;kss words snell ;L~ l)FOllOIlnS~ prepositions and determiners. The system would then at each stage only process sentences that coil rained exactly one unknown word, the hop,, I)eing that tlie words learned from these sentences would reduce the number of unknown words in the other ones. In addition to this, a rnorphologicat component w~s included to guide the assignments. Although the project proved the femsibility of the scheme, it also revealed some of its inherent problems, especially the need for fa.ster parsing methods. 2.4 Explanation-based learning, EBL A problem with all natural language grammars is that they allow a vemt number of possible constructions that very rarely, if ever, occur in real sentences. The application of explanation-based learning ~ (EBL) to natural language processing allows us to reduce tim set of possible analyses and provides a solution to the parsing inefficiency problem mentioned above (Subsection 2.3).

The original idea \[Rayner 88\] was t.o bypass llOl'lna\] processing and instead use a set of learlled rules that perh)rmed the t.~qks of the normal parsing component, l:ly indexing the learned rules eflicieutly, analysing an input sentence using the learned rules is w~ry much faster than normal processing \[Samuelsson & Rayner 9t\]. The learned rules can be viewed as templates for grammatically correct phrases which are extracted from the. granmmr and a set of training sentences using explanatiou-bmqed learning, llere, we assume the following definition: Definition 2 a ten'tplate ts a generalization constrvcted from lhe parse tree for a successfidly processed phrase, .,1 template is a tree spanning the parse with a mother category as root and a collection of its ancestor nodes

2t~xplanation-lmsed l arning is n machine learning techIllqlle closely related to tllaCro-operator learllil|g, chtlllkillg, and parliM evaluation and is described in e.g.. \[I)e.long & Mooney 8~';, Mitchell et at 86\]. AcrEs nl~ COTING-92, NAt,rl~s, 23-28 Ao(rr 1992 1 ! 7 4 (at arbitrary, but pre-defined, deep levels of nesting) as I~a~les.

The fact that the templates are derived from the original gramlnar guarantees that they represent correct phrlLses and the fact that they are extracted from real senteuces ensnres Ihat they represent constructions that actually occur. 3 Explanation-based

lexical learning, EBL 2 The basic algorithm goes ,xs follows: 1. Using a large corpus from the domain, extract

teUll)lates from the sentences contaiuing uo 1.111-

known words. 2. Analyse the remaining sentences (the ones con-

taiuing unknown words) using the templates,

while maintaining an interim lexicon for the un-

known words. 3. Compare the restrictions placed on the unknown words by the analyses obtained with other handcoded phrase templates pecific for the paradigms m the lexicon d. (2reate "rear' lexical entries from the mforma-

ti<m m the intcrhn lcxicon when a full set of

such templates \[covering a paradigm) has been

found.

In the following subsections, we will address these issues in turn. 3.1 Extracting templates from a domain-specific corpus A typical situation where we think that this method is well suited is when a general purpose NL system with a core lexicon (such as the S-CLE) is to be customized to a specific application domain. Tile vocabulary used in the domain will include e.g. technical terms that are not present in the core lexicon. Also, the use of the words in the core lexicon may differ between domains. In addition to this, some types of gralnmatieal coustrilcts may be more eonllnon ill one domain than ill allother. We will try to "get the flavour of the language" in a particular application euviromnenl from domain-specific texts.

The corpus is divided into two parts: one with seatellces containing ilnknown words, all(\] another where all the words are known, The latter group is used to extract plmme templates that capture tile grammatical constructions occurring in tile domain. rFhe process of extracting phrase templates from training sentences i outlined in Subsection 2.4. PRec. OF COLING-92, NAmV:s, AUG. 23-28, 1992

Assuming that a partieular set of phrase tenlplales is applicable to a sentence containing an unknown word will associate aset of constraints with the word. Naturally, the constraints Oil I\[le kBowlt words of the sentence should be satisfied if this tcmplatv is to be e(msidered. 3 This will correspond to a partic+ular parse or analysis of the seutenee. Thus a sol of constraints is a.ssociated with each different pm'se

A number of entries in the prot.otype iexicou will matcll the set of constraints associated with a senteuce. \['\]aeh prototyI)e is all illCal'llatioIl of il para digtn, Thus we can a.ssociate a word with a set of paradigms. (Note thai the paradigms may be nonexclusive.) All such +msociatious (corresponding to different parses of the same sentence) are collected, and used to update the+ interim h'xieon.

'\['h(! IllOSt obvious conslraiuts colnt! frolll syllt{ic tie considerations. If, in Ihe sentence John loves a ca( the word loves were unknown, while the other words did indeed have the obvious lexicai entries, the gram mar will require loves to be a transitive verb of third person singular agreement. Since the prototYl)eS of verbs are iu tl,e imperative form, we nmst associate a finite verb form with the imperatiw~, This is done by applying a omrphologieal rule that strips the '-s' from the word loves, reinforcing the hypothesis and gaining the tense information in the process.

Now, this ntorphological information lnay seem uniml)ortant in Fnglish, but it definitely is +lol it, Swedish: a word with more that+ one sy\]lal,h+ ending with '-or' has to be an in(h.finite common gel,der noun. If it is not of latin origin it lnusl, be a phi ral form an(I thus ils entire morl)hology is kJvm, n The odds that it is a countabh" noun (like d.ck), as (}\[)posed tO 1t lllaSS IIOIln (such {IS walev), ;ll'C ()vet" whehning.

During tile analysis of the set of sentences conlaining unknown words, an interim lexicon for these unknown words is kept. The interim lexieon is imlexed on word sterns and updated each titlie a IWW Sell fence is i)roeessed. \["or each word sI, eul+ t'e/o pieces of information are retained in this lexicon: a hypothesis about which paradigm or set. of paradigms lhe word is assumed to belong to, and a justifieat.ion Ihat encodes all evidence relevant o the word. The jnstifieation is used to make the hypothesis aml is main tained so that the entry may be Ul)(lat, ed whett new inlbrmation about tim word arrives. When all the l)hrase templates (sentence patterns) for lhlfilhnent Aeries nE COLINGO2. N,~t~"nis, 23-28 ̂ot~rl 1992 1 l 7 5 of a Sl)ecilic para(ligm have been found, an entry for the word is made in the domaimspecifie l xicon that is bcmg constructed. This is done while still keeping the justilication reformation, since this might contaht evidence indicating other word-senses or holnographs

A prelimiuary versi(~u of the lexieal acquisition sys tern has been implemented in Prolog. "File mealtile extracting telnplates froln Selltences with knowll words is \[uily operational. The parser for sentences witil unkuown words has also been tested, while tile iaterim lexicon still is subject to experimentatiolL Presenl.ly, a w'ry siml)le strategy for the interiln lexicon has been tesled. This version uses the set of all hypotheses ns the justification and use their dis.itmetion as the era'rent hypothesis. We are currently working Oll extending this sdlenle to one incorporating the full algorithm deseril)ed above.

Unknowu wor(l~ are matched with tim subcalegorizatiou paradigms of the S-CLE. In total 62 differenl synl.aet.ic/semantic paradigms are known by the present systmn: 5 for Swedish nmms, l0 for adjectives, aud all tim others for verbs. Tim morphological inflections are subdivided into 14 different inflectional cbLsses of nouns, 3 classes of adjectives, and 24 classes of verbs.

We have (mt.lin<'d amethod for autonlatic lexieal ae(luisilion. An existing lexicon built on the usage of i)rolotypica\] entries for l)aradigms of opemela.ss words, is ext.end~'d b5 infi~rring new lexical entries fl'OIII texts containing Dnkl/own words. The COllstraints placed on these words by the gramnlar arc compared with the prototypes and a hypothesis is made al)ouI what paradigm the word is most likely to l)olong to.

The hy\]lotheses ai)otlt, the ilnknown words are kept+ m an interim lexicon until a suflicient level of confidence is reached. Phrase templat<~s are both hand-cod<+d aud extracted front the grammar and donlaiu-spt!citic texts using an explanation-based h,arning method.

The work reported here was fimded by the Foundation tot the Swedish Institute of Computer Science aud the Swedish National Board for Industrial and T<,ch nical l)evelol)mel\]t ( N UTEK). I)roc. OF COLlNG-92, NANTES, AU(). 23-28, 1992

We would like to thank Manny Rayner and David Carter (SRI Cambridge) and Seif llaridi (SICS) for helpful discussions and suggestions, and Pierre Gander (Stockholm University) for valuable supl)ort. References Alshawi, 11. and J. van Eijek (1989). "Logical t"orms

in the Core Language I';ngine", the 271h An-

nual Meeling of the Association for Coalpala-

tional Linguistics, Vancouver, llritish Columbia,

pp. 25- 32. Alshawi, tl., D. ('.after, B. Gaml)iiek and M. Ray-

net 11991 ). "Translation by Quasi Logical Form

Transfer", the ~9lh Annual Meeting of th~ Asso-

ciation for Computational Li)tgaisltes, Univer-

sity of California, Berkeley. California, pp. 161..

168. Alshawi, I1. (ed.) (1992). 7'h~ Core Langttage

Engine, C'ambridge. Massachusetts: The MIT

Press. Boguraev, B., T. tlriscoe, J. Carroll, D. Carter

and C. Grovcr (1987). "The Derivation of a

Grammatically Indexed Lexicon from the Long-

lnan Dictionary of Contemporary English", the

251h Annual Meeting of the Associattoa for

Computational Linguistics, Stanford, Califor-

nia, pp. 193-200. C.arter, D. 11989). "Lexical Acquisition in the (_lore

Language Engine", the 4th Conference of the

European Chapter of the Association Jbr Com-

putational Liaguisttcs, Manchester, Eugland,

pp. 137- 144. Also available as SRI lnternat ioaa\[

Report CCSRC-012, Cambridge, England Calzolari, N. and R.. Bindi (1990). "Aequisilion

of Lexical Information from a Large Textual

Italian Corpus", Ihe 131h International Con-

ference on Computational Linguistics, lh'lsinki,

Finland, Vol. 3, pp. 54 59. DeJong, G and R. Mooney 11986). "Explanation

Based Learning: An Alternative View". Ma-

chine Learning, 1:145-- 176. Ganfl)~ck I:L and M. ll.ayner (1992). "The Swedish

Core Language Engine", the 2rd Nordic Confer-

ence of Tex! Comprehension in Man aad Ma-

chine, LinkSping, Sweden. Gamb/iek B. (1992). "I,exieal Acquisition: The

Swedish VEX System", the 2rd Nordic Confer-

ence of Tert Comprehension in Man and Ma-

chine. LinkiSpiag, Sweden. Granger, R.It. (1977). "FOlrl,-UP: A program that

figures Otlt meallings of words from colltex(', ACTXS DE COLING-92, NAN'W.S, 23-28 AO(Yr 1992 1 I 7 6 PROC. OF COLING-92, NANTES, AUG. 23-28, 1992 the 5tb lnter~*atio)ml Joint Conference on Artificial Intelligence, Cambridge, Massachusetts, pp. 172- 178. Grosz, B.J., D.E. Appelt, P. Martin, and

F.C.N. Pereira 11!)87). "TEAM: An Experi-

ment in the Design of Transportable Natural-

Language Interfaces", Artificial Intelligence,

32:173 243. tl6rmander, S. 11988). "The Problems of Learning

a 1,exicou with a Formal Grammar", SICS R,e-

search l/eport R88019, Stockholm, Sweden. Jacob.~. P. a~l¢l U. Zernik 11988). "Acquiring Lexieal

Knowledge from Text: A Case Study", the 7th

National Conference on Artificial Intelligence,

Saint Paul, Minnesota, pp. 739 744. Mitchell, T.M., ItM. Keller and S.T. Kedar-Cabelli

(1986). "Explanation-ltased Generalization: A

Unifying View". Machine Learning, 1:47 80. Rayner, M. 11988). "Applying Explanation-Based

Generalization to Natural-Language Process-

ing", the lntcrnalional Conference on Fifth

Geaeration C'ompuler Systems, 'lk)kyo, Japan,

pp. 1267- 1274 Rayner, M., /~. Hugosson and G. Ilagert (1988).

-lIsmg a Logic Grammar to Learn a Lexicon",

the 12th International Conference on Computa-

tional Linguistics Budapest, llungary, pp. 524-

529 Also available ,as SICS Research Report -

R88001, Stockhohu, Sweden. Samuelsson, C. and M. Rayner (1991). "Quantita-

tive Evaluation of Explanation-Based Learning

a~ an Optimization Tool for a Large-Scale Nat-

ural i,anguage System", the 12th International

Jotnl Conference on Artificial Intelligence, Syd-

ney, Australia. Trost. tl. and E. Buchberger 11986). "Towards

the Automatic Acquisition of Lexieal Data",

the llth International Conference on Computa-

tional Linguistics, ltonn, Germany, pp. 387 389. Wilensky, R. (1990). "Extending the Lexicon by

Exploiling Subregnlarities", the DA RPA Speech

and Natural Language Workshop, llidden Val-

ley, Pennsylvania, pp. 365-370. Zernik, U. and M. Dyer 11985). '"lbwards a Self-

Extending Lexicon", the 22rd Annual Meeting

of the Associatio~t for Computational Linguis-

tics, University of Chicago, Chicago, Illinois,

pp 284 292. Zernik, U. (1987). "l:mguage Acquisition: Learning

a lIierarehy of Phrases", the lOth International

Joint Conference o)1 Artzficial lntelliqence, Mi-

lan, italy, pp. 125 131