Semantic Verb Classes Tile first step was the induction of purely syntactic subcategorisation fi'ames for verbs from the heterogeneous British National CoTpus (BNC). I used the robust statistical head-entity parser as described in (Carroll and Rooth, 1998) which utilises an English context-free grammar and a lexicalised probability model to produce parse forests, and extracted the maximum probability (Viterbi) parses, for a total of 5.5 million sentences. The trees were mapped to subcategorisation frame tokens consisting of a inain verb and its argmnents. Each syntactic category was accompanied by the lexical head, the pret)ositional phrase by the lexical prepositional head plus the head noun of the subordinated noun phrase. Proper names were accompanied by the identifier pn. The head information in the frames was lemmatised. For example, the sentence Samrout handled the plaudits during the awards ceremony would be represented by the frame token handle subj*pn*sammut obj*plaudit pp*during*ceremony.

To generalise over the verbs' usage of subcategorisation frames, I defined as 88 frame types the most frequent frames which appeared at least 2,000 times in total in the BNC sentence parses, disregarding the lexical head information. On the basis of the frame types I collected information about the joint frequencies of the verbs in the BNC and the subcategorisation frame types they appeared with. These frequency counts then represented the syntactic description of the verbs.

Tim next step was to refine the subcategorisation frame types by a preferential ordering on conceptual classes for the argument slots in the fl'ames. The basis I could use for the selectional preferences was provided by the lexical heads ill the fi'anm tokens. For example, the nouns appearing in the direct object slot of the transitive frame for the verb drink included coffee, milk, beer, indicating a conceptual class like beverage tbr this argument slot.

I followed (Resnik, 1993)/(Resnik, 1997) who defined selectional preference as the amount of information a verb provides about its semantic argument classes. He utilised the WordNet taxonomy (Beckwith et al., 1991) for a probabilistic model capturing the co-occurrence behaviour of verbs and conceptual classes, where the conceptual classes were identified by WordNet synsets, sets of synonymous nouns within a semantic hierarchy. Referring to the above example, the three nouns coffee, milk, beer are in three different synsets -since they are not synonyms-, but are all subordinated to the synset {beverage, drink, potable}. The goal in this example would therefore be to determine the relevant synset as the most selectionally preferred synset for the direct object slot of the verb drink.

Redefined fbr iny usage, the selectional preference of a verb v tbr a certain semantic class c within a subcategorisation franm slot s was deternfined by the association ass between verb and semantic lass: =des Pl, C, lV~pOg ~ (5) with the probabilities estimated by maxinmnl likelihood: f(v,, P(C*lVs) - f(vs) p(Cs) = f(c.,) _ f(cs) f(c's) /(8) and the following interpretation: (6) (7) 1. f(v,, c,): number of times a semantic class ap-

peared in a fi'ame slot of a verb's fi'ame type 2. f(v,): frequency of a verb regarding a specific

fi'ame type, i.e. the joint Dequency of verb and

frame type 3. f(Cs): numl)er of times a semantic class ap-

peared in a fi'ame slot of a frame type disre-

garding tim verb

argument slot within a certain frame type, since summing over all possible classes within a subcategorisation fl'ame slot equals the lmlnber of tinms the slot; appeared 5. f(s): uulnber of times the franle type appeared,

since the frequency of a. frame type equals the

frequency of that frame with a certain slot

marked The fi'equencies of a semantic class concerning an argument slot, of a frame type (dependent or independent of a verb) were calculated by all approach slightly difl'erent to Resnik's, originally proposed by (Ribas, 1994)/(Ribas, 1995). For each noun appearing in a certain argument position its fi'equency was divided by the nmnber of senses the noun was assigned by the WordNet hierarchy, t to take account of the uncertainty about the sense of the noun. The fi'action was allocated to each conceptual class in the hierarchy to which the noun belonged and accumulated upwards until a top node was reached. Tile result was a numerical distribution over the WordNet classes: s(c,/-- E /(noun) (8) I restricted tlm possible (:onceptual classes within 1;he fl'ames' argmnent slots to 23 Wor(tNet nodes, 2 1;o facilitate generalisation and comI)arison of the verbs' seleetional preference behaviour.

On the basis of the inforlnation al)out subcategorisation frame types and their arguments' concet)tual classes Iclustered 153 verbs from Levin's classitica(;ion. I chose (i) some l)olysemous verbs to investigate how this l)henomenon could be handled 1)y the clustering algorithms, and (ii) high and low frequent verbs to see the intluence of frequency on th(; algorithms: the 1~3 verbs had 226 verb senses which belonged to 30 different semantic lasses. D)ur of the verbs were low-Dequeney verbs with a total corpus frequency below 100.

To cluster the verbs I applied two different algorithms, and each algorithm clustered the verl)s bot, h (h) according to only the syntactic information about tlm subcategorisation frames, and (B) according to the intbrmation at)out the subcategorisation ti'ames including their selectional 1)referelmes. ,. lterative clustering based on a dcfinition

by (Ilugh, es, 109/,): In the l)eginning, each vert) represent;ed a singleton cluster. Iteratively, the distances between tim clusters were lneasure(l and the closest chlsters merged togel;her. For the rel)resentation of the. verbs, each verl) v was assigned a distribution over the ditfere.nt tyl)es of subcategorisatioll fl'anms i, according 1;o the. maximum likelihood estimate, of (k) the. verb apl)earing with the frame tyl)e: f(v,/,) f(,,,) (9) with f(v,t) the joint fi'equency of verb and frmne type, and f(v) the fl'e(tuency of the verb, and (B) the verb appearing with the frame tyt)e mid a selectionally t)refe.rred (:lass coml)ination C for the m'gmnent )osil;ions .s in t: i,(~,, ely ) =,,ef p(tl v) * J,(Clv , t) (10) with p(/,lv) defined as in equation (9), and p(C\]v, t) =&/ Ec:6,:l,,.~, \[Isct a.s.s'(v.~, c') (11) which intuitively estimates the probability of a certain class combination by comparing its association value with the sum over all possible class combinations, concerning the respective verb and frame. Starting out with each verb representing a singleton cluster, I iteratively determined the two closest chlsters by applying tim informationtheoretic measure relative cutropy :~ (Kulll)ack mid Leibler, 1951) to comi)are the distributions. The nearest clusters were merged into one cluster, and their distributions were merged 1)y calculating a weighted average. Based on test runs I defined lleuristics about how many elusl, eriug iteral;ions were pertbrmed. In addition, i liraire(1 the maximum mnuber of verbs within one (:luster to four elements because otherwise the. verbs showed the tendency to cluster together in a few large clusters only; so after the overall clustering process was finished, each cluster with more tlmn four members initialised a fllrther clustering pass on itself. Unsupervised latent, class aualysis as described in (l~ooth, 1998), based on the cxpcetation'maximisation al.qorithm: The algorithm identified categori(:al types among indirect, ly observed multinomial distributions 1) 3, apl)lying the EM-algorithm (\])elnpsteret al., 1977) to maximise the joint prol)ability of (h) t;he verb and frmne tyl)e: p(v, t), and (B) the verl) and frame type considering the selectional I)referenees: p(v, t, C). \]TUl)Ut to the algorithm were absolute, frequencies of the verl)s at)l)earing with the sul)categorisation frames. Test runs showed that 80 clusters modelled the semantic verl) classes best. To 1)e able to comI)a.re the analysis wit;h the iterative clustering al)proach , I also limited tim numb(~r of verbs wit;hin a (:lus|;er 1;o four considering that; generally all verbs ai)l)ear within each (:lusl;er when using this apl)roach , the verbs wil;h l:he highest l)rol)abilities where chosen. D)r version (h) the frequencies were provide.d by the joint frequencies of verbs and frame tyI)es, for version (B) I used the association va.lues of the verbs with tile frame tyl)eS considering seleetional preferences, as described 1)y equation (10). The unsupervised algorithm then classified joint events of verbs and subeategoris~tion frmncs with 200 iterations of the EM-algorithm into 80 clusters r, based on the iteratively estimated vahles v(v, 0 = v, l,) = T T (12)

aConcerning the two typical prol)lems one has with this measure, (i) zero frequencies were smoothed 1)y adding 0.5 to all frequencies, and (ii) since the measure is not symmetric, the resl)ective smaller vahm was used as distance. Information Clusters

Total Correct SFs 31 20 SFs + Pretls 30 14 Total

90

81 Verbs

Recall Precision

36% 61%

20% 38% Correct

55

31 Figure 1: Evaluation based on Iterative Clustering hfformation Clusters

~lbtal Correct SFs 80 36 SFs ~1-Prefs 80 22

Total 107(159) 153(226)

Verbs(Senses) Correct Recall Precision 58(9O) 38(4O)% ,54(57)% 47(56) 31(25)0/o 31(25)% Figure 2: Evaluation based on Latent Classes I,(,,, t, c) = v, t, c) = T T Cl ) (13) for versions (h) and (B), respectively. 3 Evaluation The evaluation of the resulting clusters was based on Levin's classification. Figures 1 and 2 present the success of the two clustering algorithms, considering tim two difl'erent informational versions (/~) and (B). They contain the total mnnber of clusters the algorithms had formed (clusters containing between two and four verbs in the iterative algorithm, and the fixed immber of 80 clusters in the l&l;ent (:lass rarelysis), the prol)ortion of correct clusters (non-singleton clusters which were subsets of a Levin (:lass, for example the cluster conl;aining the verl)s need, like, ,want, desire is a subset of the Levin (:lass Desire), and the numl)er of verbs wMlin those clusters. In figure, 2 the nulnl)er of verbs in brackets rethrs to the respective number of Lheir senses, since a verb could be clustered several times according to its senses. For examl)le, the verl) want could t)e meml)er of the (:lasses Desire and Declaration.

Recall was define(l by the I)ercentage of verbs (verb senses) within the correct clusters compared to the total munber of verbs (verb senses) to be clustered: I,,e,'bs ......... , ,.,,.,, ..... I ?*C'C = 153

(Iv ,.b .......... , ........... l)

. 226 and precision was defined by the percentage of verbs (verb senses) apl)earing in the correct clusters compared to the numl)er of verbs (verb senses) apl)earing in any cluster:

\[ ve.rbs..o,..~,.,t ~.t,~t,~,.~ \[ wee = Ive,.r,s,,, ~,,.,,~,., I (iv-+ .................. , ........... I) Concerning t)recision, the assignntent of verbs into semantic classes was most successfifl when using the il;erative distance clustering method; 61% of all verbs were clustered into correct classes. Clustering the verbs into latent classes was with 54% less successtiff. With both clustering methods the results became worse when adding information about the selectional preferences tbr the arguments in the subcategorisation fl'ames.

A baseline ext)eriment was performed in order to determine how hard the task of verb clustering was: each verb was randomly assigned another verb as "closest neighbour", which resulted in only 5% el the, verl)s being paired with a verb Don1 the same Levin (:lass. Performing the same experiment by assigning the closest neighbour on the basis of moasm'ing the relative entropy between two verbs' distributions over subcategorisation fl'ames resulted in 61% of the verbs pointing to a verb flom the same Levin class. 4 Discussion d The classitications of both clustering approaches illustrate the close relationship between alternation behaviour and semantic classes, lYor exalnple, the common preferences of verbs (see the tlve most probable frames) ill the iteratively crea.ted Desire (:lass were towards a sul)ject followed by an infinitival phrase (subj :to). Alternatively a l;ransitive subj :obj flame was used, partly followed by an additional infinitival phrase indicated by to: s Verl) Frame need subj:to

subj:ol)j

subj

subj:obj:to

subj:obj:pp.for

sul)j:to

subj:ol)j

subj

sul)j:obj:adv

sub.i:obj:obj

subj:to

subj :obj

subj

sul)j :ol)j :to

subj :to:adv desire subj :obj

subj

sul)j:to

sul~j:obj:to

sul)j:sent Adding ilfformation about the selectional preferenees of the verbs' argmnents hell)s to gel; a deeper idea about their lexical semantics. D:)r exalnple, mar~,'n, er of Motion verbs 1)referably appeared with a subject only, sometimes with a following adverl). The subject was an inanimate ol)ject, for move it might also be a part (such as a body part like fin_ ger) or a grout), roll and fly alternatively used the transitive frmne type subj :obj, preferal)ly with a living entity as subject, followed by an inanimate ob.iecl;: roll fly Fl'itllle sub.i (l'hysObject) subj (l'hysObject):adv subj(Agent):obj (lq~ysObject) subj (IJ fel,'orm) :ol)j ( lq C,'sObject) subj(Agent):obj (lhu't) subj(l'hysOI)ject) subj (l'hysOI)jcct):adv sub.i(Lifel,'orm) :obj (l'hysObjcct) subj (l,illaForm) :pp.to (1A fel"or n 0 subj(Lifeleorm):l)p.to (Agent) sul).i(l 'hysObject) subj (l)hysOl~ject):adv sul)j (1 're'i,) sul~j(Groul)):adv subj(Part) :adv Parallel examples created by the latent class analysis present the clusters with the most probable verbs and frmnes, according to cluster membershi I) (first column). The dot indicates whether the verb-fi'mne combination was seen in the data, the mmtber next to the verb frame gives the probability of the verbfrmne combination. Some verbs of Telling were clustered mainly according to their similar transitive use combined with an infiifitival phrase: ~°_ Clusi;er d

o

,, o (}.17 advise • 0.12 teltch • 0.12 instruct • g o < • • • g o • • l)rol)ability 0.38 (I.32 0.10 0.05 0.02 0.34 0.34 0.14 0.(14 0.03 0.53 0.15 0.11 (1.10 0.02 0.25 0.24 0.20 0.(17 (I.02 l'rob~dfility 0.24 0.10 0.07 0.07 0.05 0.3d 0.12 0.(17 0.05 0.0,1 0.20 0.11 0.09 0.0,1 (1.0,1 g o • • • o'° cb g o = o • • • The verl)s of Aspect alternate between a subject only, realised by an action, an inanimate subject followed by an infinitiwfl phrase, and a living subject followed by a gerund: g', ClHster o <

Both approaches established a relationship between alternation behaviour and semantic class by only considering information about the syntactic usage of the subcategorisation Dames. The refinement by the frames' selectional preferences allowed fllrther demarcations by the identifying (:onceptual restrictions on tile use of the frames. Since tim latent class analysis is a soft; clustering method, it additionally distinguishes between the dith;rent verbs' senses and the resl)ective uses of subcategorisation Dames. For example, the verb play was clustered with meet 1)ecause of tile common strong tendency towards a transitive ti"ame illustrating a gen(;ral meeting, and it, was clustered with figh, t t)eemlse of their colnmon preference for an intransitive fi'ame together with a prepositional phrase headed 1)y against, illustrating a more aggressix'(; me.eting like a fight: Cluster I~ g Cluster 5 meet l)lay ~L

An extensive investigation of tile linguistic reliability of the clustered verbs and frames showed that l;he character(sing usages could be under\](ned by corpus data, for example the above cited transitive use ~ d © ;~ • • g o b0 ;5 • • • g o bO • • of the verb fly concerning the subj : obj frame type with a living subject and ml inanimate object can be illustrated by the BNC-sentence In March the manufacturer's test pilot flew the aircraft for its annual inspection check flight. The clusters were therefore created on a reliable linguistic basis representing (a selective part of) the verbs' properties. Comparing the two informational versions, however, showed that refining the fralnes with selectional preferences points to a problem caused by data sparseness in the verb description. Investigating the automatically created distribution of the verbs over the enriched fl'ame types revealed that, for example, even the high fl'equent, alternating verb move contains 97% (smoothed) zeroes within its distribution. In accordance with this fiuding even subtle similarities, e.g. the sole fact that two verbs have non-zero wflues for certain fl'ame types, highly correlates the two verbs. For example, a semantic cluster contained the two verbs promise and love, because both have non-zero attribute values for the subj :to frame, demmlding an agent for the subject slot; in their alternation behaviour (including selectional preferences) the two verbs differ, however, so they should not be packed into one cluster. A possible suggestion to handle the problem of data sparseness could be to formulate the conceptual class types in a way which ensures an increased data potential for each type.

Concerning the polysemy of verbs, the (hard) iterative distance clustering failed to model verb senses; a polysemous verb was either not at all assigned to any cluster, or assigned to a cluster describing one of the verb's senses. The (soft) latent (:lass analysis was able to filter the multiple senses and assign them to distinct (:lusters, but tended to split senses. Low-frequency verbs presented another problem, because the verbs' distributions contained mostly zeroes. They were assigned to clusters nearly randomly.

An investigation of selected WordNet conceptual classes revealed that the selectional preferences within the subcategorisation frames were donfinated by a few WordNet classes, mainly LifeForm and Agent. The demarcation between these two concepts was not obvious when referring to actually appearing nouns within the frames, since both contain a large number of common subordinated nouns. In contrast, some WordNet classes were not chosen at all, e.g. Unit or Anticipation. Since the WordNet hierarchy in general had turned out to define intuitively correct seleetional preferences, an improved classification utilised for my conceptual classification should be substituted by finer synsets, i.e. one should consider using a different cut through the WordNet hierarchy.

I proposed two algorithms for automatically classif~,ing verbs semantically, based on their alternation behaviour. Taking Levin's classification as a standard for 153 manually chosen verbs with 226 verb senses and their assignment into 30 semantic classes, the iterative distance clustering succeeded for 61% of the verbs considering the syntactic usage of the fl'ames only, and for 38% when adding information about the frmne arguments' selectional preferences. The latent class analysis succeeded for 54% and 31%, respectively.

An investigation of the resulting clusters showed that the assignment of the verbs was actually based on their shared linguistic properties: the verbs in a cluster presented common alternation behaviour, refined by adding selectional preferences to the syntactic description of the subcategorisation frmnes.

It is impressive that as little lexical idiosyncratic verb information as the syntactic use of subcategorisation fl'ames like subj : to or subj : pp. against suffices as a basis for a semantic class distinction towards Levin's narrow classification system including fine concepts as Desire or Manner of Motion. The potential is partly characterised by specific frames, but in the majority of cases by successflflly combining the frames in order to define the syntactic alternation, hnproving the definition and demarcation of conceptual classes should provide further potential concerning the inclusion of selectional preferences into the syntactic description. References Richard Beckwith, Christiane Fellbaum, Derek

Gross, and George A. Miller. 1991. Wordnet: A

Lexical Database Organized on Psycholinguistic

Principles. In Uri Zernik, editor, Lcxical Acqui-

sition - Exploiting On-Line Resources to Bnild a

Lczicon, chapter 9, pages 211 232. Lawrence Erl-

baron Associates, Hillsdale - New Jersey. Glenn Carroll and Mats Rooth. 1998. Valence In-

duction with a Head-Lexicalized PCFG. In Pro-

ceedings of the 3rd Confcrcncc on Empirical Meth-

ods in Natu~nl Language Processing, Granada,

Spain. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977.

Maximum Likelihood from Incomplete Data via

the EM algorithm. Journal of the Royal Statistical Society, 39(B):1-38. Bonnie J. Dorr and Doug Jones. 1996. Role of Word

Sense Dismnbiguation in Lexical Acquisition: Pre-

dicting Semantics from Syntactic Cues. In Pro-

ceedings of the 16th International Conference on

Comp'utational Linguistics, Copenhagen. John Hughes. 1994. Automatically Acquiring Clas-

sification of Words. Ph.D. thesis, University of

Leeds, School of Computer Studies. Douglas A. Jones, Robert C. Berwick, Franklin Conference of thc European Chaptcr of the Associ-

Cho, Zeeshan Khan, Karen T. Kohl, Naoyuki No- ation for Computational Linguistics, pages 45-52.

mura, Anand Radhakrislman, Ulri('h Sauerlan(1,

and Brian Ulicny. 1994. Verb (,'lasses and Al-

ternations ill Bangla, German, English, and Ko-

rean. Technical l{el)ort MIT AI MEMO 1517,

Massachusetts Institute of Technology. Judith L. Kla.vans and Min-Yen Kan. 1998. The

Role of Verbs in DOeulnent Analysis. In Pwceed-

ings of thc 17th Intcrnational Co~@rcncc on Com-

putational Linguistics, Montreal, Canada. S. Kullback and R. A. Leibler. 1951. On Infl)rmation

and Sufficiency. Annals of Mathematical Statis-

tics, 22:79-86. Maria Lapata. 1999. Acquiring Lcxical Generaliza-

tions from Corpora: A Case Study for Diathe-

sis Alternations. In Proceedings of the 37th An-

nual Mccting of the Association for Computa-

tional Linguistics, pages 397 404:. Beth Levin. 1993. English Verb Classes and Al-

ternations. The University of Chi(:ago Press,

Chicago, 1st edition. Malka Rat)i)al)ort Hovav and Beth Levin. 1998.

Building Verb Meanings. In M. Butt and

W. Geuder, editors, Lcxical and Compositional

Factors, pages 97-134. CSLI Publications, Stan-

ford, CA. Philip Resnik. 1993. Selection and Information:

A Class-Based AppTvach to Lexical Relationsh, ips.

Ph.D. thesis, University of Pennsylvania. Philip Resnik. 1997. Selectional Preference and

Sense Disambiguation. In Proceedings of the ACL

SIGLEX Workshop on ~hflginfl ~::ct with, Lcxical

Semantics: Wh, y, Wh, at, and llow? l~5"ancesc Ribas. 1994. An Experiment on Learn-

ing Appropriate SelectionM Restrictions fi'om a

Parsed Corpus. In Procecdings of the 15th Inter-

national Conference on Computational Linguis-

tics, pages 769 774. Francesc Ribas. 1995. On Learning Mot'e Appropri-

ate Selcctional Restrictions. In Pwcccdings of the

7th Conference of the Eurot)ean Chaptcr of the As-

sociation for Computational Linguistics, Dublin,

Ireland. Mats Rooth. 1998. Two-Dimensional Clusters in

Grammatical Relations. In Inducing Lexicons

with th, c EM Algorithm, AIMS Report 4(3). Insti-

tut ffir Maschinelle Si)raehverarl)eitung, Univer-

sitgt Stuttgart. Sabine Schulte im Walde. 1998. Automatic Se-

nmntic Classification of Verbs According to Their

Alternation Behaviour. Master's thesis, Institut

ffir Maschinelle Sprachverarbeitung, UniversitSt

Stuttgart. Suzamm Stevenson and Paola Merlo. 1999. Auto-

Inatic Verb Classification Using Distributions of

Grammatical Features. hi P~vcccdings of the 9th