Much work in lexical semantics has been concerned with accounting for regular poly­semy, that is, the regular and predictable sense alternations to which certain classes of words are subject (Apresjan 1973). It has been argued that in some cases, the different interpretations of these words must arise from the interaction between the semantics of the words during syntactic composition, rather than by exhaustively listing all the possible senses of a word in distinct lexical entries (Pustejovsky 1991). The class of phenomena that Pustejovsky (1991, 1995) has called logical metonymy is one such example. In the case of logical metonymy additional meaning arises for particular verb-noun and adjective-noun combinations in a systematic way: the verb (or adjec­tive) semantically selects for an event-type argument, which is a different semantic type from that denoted by the noun. Nevertheless, the value of this event is pre­dictable from the semantics of the noun. An example of verbal logical metonymy is given in (1) and (2): (1a) usually means (1b) and (2a) usually means (2b).

(1) a. Mary finished the cigarette.

b. Mary finished smoking the cigarette.

(2) a. Mary finished her beer.

b. Mary finished drinking her beer.

* Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello Street, Sheffield

S1 4DP, UK. E-mail: mlap@dcs.shef.ac.uk. f School of Informatics, University of Edinburgh, 2 Buccleuch Place, Edinburgh EH8 9LW, UK. E-mail:

alex@inf.ed.ac.uk.

Consider the verb-noun combinations in (7) and (8). Our task is to come up with (7b) and (8b) as appropriate interpretations for (7a) and (8a). Although the interpreta­tions of (7a) and (8a) are relatively straightforward for English speakers, given their general knowledge about coffees and films and the activities or events associated with them, a probabilistic model requires detailed information about words and their interdependencies in order to generate the right interpretation. Examples of such in-terdependencies are verbs co-occurring with coffee (e.g., drink, make, prepare) or verbs that are related to begin (e.g., make, realize, understand).

(7) a. John began the coffee.

b. John began drinking the coffee.

(8) a. Mary enjoyed the film.

b. Mary enjoyed watching the film.

A relatively straightforward approach to the interpretation of (7a) and (8a) would be to extract from the corpus (via parsing) paraphrases in which the additional in­formation (e.g., drinking and watching), which is absent from (7a) and (8a), is fleshed out. In other words, we would like to find in the corpus sentences whose main verb is begin followed either by the progressive VP complement drinking or by the infinitive to drink, selecting for the NP coffee as its object. In the general case we would like to find the activities or events related both to the verb begin and the noun coffee (e.g., drinking, buying, making, preparing). Similarly, in order to paraphrase (8a) we need information about the VP complements that are associated with enjoy and can take film as their object (e.g., watching, making, shooting).

The above paraphrase-based model is attractive given its simplicity: All we need to do is count the co-occurrences of a verb, its complements, and their objects. The approach is unsupervised, no manual annotation is required, and no corpus-external resources are used. Such a model relies on the assumption that the interpretations of (7a) and (8a) can be approximated by their usage; that is, it assumes that the likeli­hood of uttering the metonymic construction is equal to that of uttering its interpre­tation. However, this assumption is not borne out. Only four sentences in the BNC are relevant for the interpretation of begin coffee (see (9)); likewise, four sentences are relevant for the interpretation of enjoy film (see (10)).

(9) a. Siegfried bustled in, muttered a greeting and began to pour his coffee.

b. She began to pour coffee.

c. Jenna began to serve the coffee.

d. Victor began dispensing coffee.

(10) a. I was given a good speaking part and enjoyed making the film.

b. He's enjoying making the film.

c. Courtenay enjoyed making the film.

d. I enjoy most music and enjoy watching good films.

e. Did you enjoy acting alongside Marlon Brando in the recent film The Freshman?

The attested sentences in (9) are misleading if they are taken as the only evidence for the interpretation of begin coffee, for on their own they suggest that the most likely interpretation for begin coffee is begin to pour coffee, whereas begin to serve coffee and begin dispensing coffee are less likely, as they are attested in the corpus only once. Note that the sentences in (9) fail to capture begin to drink coffee as a potential interpretation for begin coffee. On the basis of the sentences in (10), enjoy making the film is the most likely interpretation for (8a), whereas enjoy watching the film and enjoy acting in the film are equally likely.

This finding complies with Briscoe, Copestake, and Boguraev's (1990) results for the LOB corpus: begin V NP is very rare when the value of V corresponds to a highly plausible interpretation of begin NP. Indeed, one can predict that problems with finding evidence for begin V NP will occur on the basis of Gricean principles of language production, where the heuristic be brief (which is part of the maxim of manner) will compel speakers to utter begin coffee as opposed to begin V coffee if V is one of the plausible interpretations of begin coffee. Thus on the basis of this Gricean reasoning, one might expect metonymies like (7a) and (8a) to occur with greater frequencies than their respective paraphrases (see (7b) and (8b)). Tables 1-3 show BNC counts of verb-noun metonymies (commonly cited in the lexical semantics literature (Pustejovsky 1995; Verspoor 1997)) and their corresponding interpretations when these are attested in the corpus. The data in Tables 1-3 indicate that metonymic expressions are more often attested in the BNC with NP rather than with VP complements.

The discrepancy between an interpretation and its usage could be circumvented by using a corpus labeled explicitly with interpretation paraphrases. Lacking such a corpus, we will sketch below an approach to the interpretation of metonymies that retains the simplicity of the paraphrase-based account but no longer assumes a tight correspondence between a metonymic interpretation and its usage. We present an

BNC frequencies for enjoy.

enjoy coffee 8 1

BNC frequencies for want.

want cigarette 18 3

unsupervised method that generates interpretations for verbal metonymies without recourse to manually annotated data or taxonomic information; it requires only a part-of-speech-tagged corpus and a partial parser.

We model the interpretation of a verbal metonymy as the joint distribution P(e, o, v) of three variables: the metonymic verb v (e.g., enjoy), its object o (e.g., film), and the sought-after interpretation e (e.g., making, watching, directing). By choosing the ordering (e, v, o) for the variables e, v, and o, we can factor P(e, o, v) as follows:

The probabilities P(e), P(v | e), and P(o | e, v) can be estimated using maximum likeli­hood as follows:

Most frequent complements of enjoy and film.

Although P(e) and P(v | e) can be estimated straightforwardly from a corpus f (e) amounts to the number of the times a given verb e is attested, N is the number of verbs found in the corpus (excluding modals and auxiliaries), and P(v | e) can be ob­tained through parsing, by counting the number of times a verb v takes e as its comple­ment), the estimation of P(o | e, v) is problematic. It presupposes that co-occurrences of metonymic expressions and their interpretations are to be found in a given corpus, but as we've seen previously, there is a discrepancy between a metonymic interpretation and its usage. In fact, metonymies occur more frequently than their overt interpreta­tions (expressed by the term f (o, e, v) in (14)), and the interpretations in question are not explicitly marked in our corpus. We will therefore make the following approximation:

The rationale behind this approximation is that the likelihood of seeing a noun o as the object of an event e is largely independent of whether e is the complement of another verb. In other words, v is conditionally independent of e, since the likelihood of o is (largely) determined on the basis of e and not of v. Consider again example (8a): Mary enjoyed the film. Here, film, the object of enjoy, is more closely related to the underspeci-fied interpretation e rather than to enjoy. For example, watching movies is more likely than eating movies, irrespective of whether Mary enjoyed or liked watching them. We estimate P(o | e) as shown in (16). The simplification in (15) results in a compact model with a relatively small number of parameters that can be estimated straightfor­wardly from the corpus in an unsupervised manner. By substituting equations (12), (13), and (16) into (11) and simplifying the relevant terms, (11) can be rewritten as follows:

Assume we want to generate meaning paraphrases for the verb-noun pair enjoy lm (see (8a)). Table 4 lists the most frequent events related to the verb enjoy and the

most frequent verbs that take lm as their object (we describe how the frequencies ( v e) and ( o e) were obtained in the following section). We can observe that seeing, watching making and using are all events associated with enjoy and with lm and will be therefore generated as likely paraphrases for the metonymic expression enjoy film (see Table 4, in which the underlined verbs indicate common complements between the metonymic verb and its object).

Note that the model in (17) does not represent the fact that the metonymic verb v may have a subject. This in practice means that the model cannot distinguish between the different readings for (18a) and (18b): in (18a) the doctor enjoyed watching the film, whereas in (18b) the director enjoyed making or directing the film. The model in (17) will generate the set of events that are associated with enjoying films (e.g., watching, making, seeing, going to), ignoring the contribution of the sentential subject. We present in Section 2.5.1 an extension of the basic model that takes sentential subjects into account.

(18) a. The doctor enjoyed the film. b. The director enjoyed the film.

It is important to stress that the probabilistic model outlined above is a model of the interpretation rather than the grammaticality of metonymic expressions. In other words, we do not assume that it can distinguish between well-formed and odd metonymic expressions (see the examples in (4)). In fact, it will generally provide a set of interpre­tation paraphrases, even for odd formulations. The model in (11) has no component that corresponds to the occurrence of v and o together. Choosing the ordering (o, v, e) for the variables o, e, and v would result in the following derivation for P(e,o, v):

The term P(v | o) in (19) explicitly takes into account the likelihood of occurrence of the metonymic expression. This means that no interpretation will be provided for odd metonymies like enjoy the highway as long as they are not attested in the corpus. Such a model penalizes, however, well-formed metonymies that are not attested in the corpus. A striking example is enjoy the ice cream, which is a plausible metonymy not attested at all in the BNC and thus by (19) would be incorrectly assigned no interpretations. This is because the maximum-likelihood estimate of P(v | o) relies on the co-occurrence frequency f (v, o), which is zero for enjoy the ice cream. But the probabilistic model in (11) will generate meaning paraphrases for metonymic verb-object pairs that have not been attested in the corpus as long as the co-occurrence frequencies f (v, e) and f (o, e) are available.

Finally, note that our model is ignorant with respect to the discourse context within which a given sentence is embedded. This means that it will come up with the same ranked set of meanings for (20b), irrespective of whether it is preceded by sentence (20a) or (21a). The model thus does not focus on the meaning of individ­ual corpus tokens; instead it determines the most dominant meanings for a given verb-noun combination overall, across all of its instances in the corpus.

b. John finished three cigarettes.

c. John finished making three cigarettes.

c. John finished smoking three cigarettes.

We estimated the parameters of the model outlined in the previous section from a part-of-speech-tagged and lemmatized version of the BNC, a 100-million-word collection of samples of written and spoken language from a wide range of sources designed to represent current British English (Burnard 1995). The counts f (v, e) and f (o, e) (see (17)) were obtained automatically from a partially parsed version of the BNC created using Cass (Abney 1996), a robust chunk parser designed for the shallow analysis of noisy text. The parser's built-in function was employed to extract tuples of verb-subjects and verb-objects (see (22)). Although verb-subject relations are not relevant for the present model, they are important for capturing the influence of the sentential subject (see Section 2.5) and modeling the interpretations of polysemous adjectives (which we discuss in Section 3).

(22) a. change situation SUBJ b. come off heroin OBJ c. deal with situation OBJ

(23) a. isolated people SUBJ b. smile good SUBJ

The tuples obtained from the parser's output are an imperfect source of informa­tion about argument relations. Bracketing errors, as well as errors in identifying chunk categories accurately, results in tuples whose lexical items do not stand in a verb­argument relationship. For example, inspection of the original BNC sentences from which the tuples in (23) were derived reveals that the verb be is missing from (23a) and the noun smile is missing from (23b) (see the sentences in (24)).

(24) a. Wenger found that more than half the childless old people in her study of rural Wales saw a relative, a sibling, niece, nephew or cousin at least once a week, though in inner city London there were more isolated old people. b. I smiled my best smile down the line.

In order to compile a comprehensive count of verb-argument relations, we dis­carded tuples containing verbs or nouns attested in a verb-argument relationship only once. Instances of the verb be were also eliminated, since they contribute no semantic information with respect to the events or activities that are possibly associated with the noun with which the verb is combined. Particle verbs (see (22b)) were retained only if the particle was adjacent to the verb. Verbs followed by the preposition by and a head noun were considered instances of verb-subject relations. The verb-object tuples also included prepositional objects (see (22c)). It was assumed that PPs adja­cent to the verb headed by any of the prepositions in, to, for, with, on, at, from, of, into, through, and upon were prepositional objects. This resulted in 737,390 distinct types of verb-subject pairs and 1,078,053 distinct types of verb-object pairs (see Table 5, which presents information about the tuples extracted from the corpus before and after the filtering).

2 The POS tagging of the BNC (Leech, Garside, and Bryant 1994) distinguishes between verb particle constructions like down in climb down the mountain and up in put up the painting, on the one hand, and prepositions, on the other. So this allowed us to distinguish PP complements from NP ones.

Paraphrases for verb-noun combinations taken from the literature.

John began the story — telling

John began the song — singing

John began the sandwich — eating/making

Mary wants a job — to have

John began the book — reading/writing

Bill enjoyed Steven King's last book — reading

John began the cigarette — smoking

Harry wants another cigarette — to smoke

Model-derived paraphrases for verbal metonymies, ranked in order of likelihood.

is begin song, for which the model generates the interpretations rehearse, write, hum, and play, in addition to sing.

The model also exhibits slight variation in the interpretations for a given noun among the different metonymic verbs (compare begin book and enjoy book and begin cigarette and want cigarette in Table 7). This is in line with claims made in the lexical semantics literature (Copestake and Briscoe 1995; Pustejovsky 1995; Verspoor 1997), and it ultimately contributes to an improved performance against a "naive baseline" model (see Section 2.4).

In some cases, the model comes up with counterintuitive interpretations: bite into is generated as the most likely interpretation for begin sandwich (although the latter interpretation is not so implausible, since eating entails biting into). The model also fails to rank have as one of the five most likely interpretations for want job (see Ta­ble 7). The interpretations get and take are, however, relatively likely; note that they semantically entail the desired interpretation—namely, have—as a poststate. The inter­pretations make and create imply the act of hiring rather than finding a job. Our model cannot distinguish between the two types of interpretations. It also cannot discover related meanings: for example, that get and take mean have or that tell, retell, and recount (see Table 7) mean tell. (We return to this issue in Section 4.)

In the following section we test our model against verb-noun pairs randomly selected from the BNC and evaluate the meaning paraphrases it generates against human judgments. We explore the linear relationship between the subjects' rankings and the model-derived probabilities using correlation analysis.

Although there is no standard way to evaluate the paraphrases generated by the model (there is no gold standard for comparison), a reasonable way to judge the model's per­formance would seem to be its degree of agreement with human paraphrase ratings. This can be roughly measured by selecting some metonymic constructions, deriving their paraphrase interpretations using the model outlined in Section 2.1, eliciting hu­man judgments on these paraphrases, and then looking at how well the human ratings correlate with the model probabilities for the same paraphrases.

In the following section we describe our method for assembling the set of exper­imental materials and eliciting human-subject data for the metonymy paraphrasing task. We use correlation analysis to compare the model probabilities against human judgments and explore whether there is a linear relationship between the model-derived likelihood of a given meaning and its perceived plausibility.

In Section 2.4.1 we introduce a naive model of verbal metonymy that does not take the contribution of the metonymic verb into account; metonymic interpretations (i.e., verbs) are simply expressed in terms of their conditional dependence on their objects. We investigate the naive model's performance against the human judgments and the paraphrases generated by our initial model (see Section 2.4).

2.3.1 Method.

2.3.1.1 Materials and Design. From the lexical semantics literature (Pustejovsky 1995; Verspoor 1997; McElree et al. 2001) we compiled a list of 20 verbs that allow logical metonymy. From these we randomly selected 12 verbs (attempt, begin, enjoy, finish, expect, postpone, prefer, resist, start, survive, try, and want). The selected verbs ranged in BNC frequency from 10.9 per million to 905.3 per million. Next, we paired each one of them with five nouns randomly selected from the BNC. The nouns had to be attested in the corpus as the object of the verbs in question. Recall that verb-object pairs were identified using Abney's (1996) chunk parser Cass (see Section 2.2 for details). From the retrieved verb-object pairs, we removed all pairs with BNC frequency of one, as we did not want to include verb-noun combinations that were potentially unfamiliar to the subjects. We used the model outlined in Section 2.1 to derive meaning paraphrases for the 60 verb-noun combinations.

Our materials selection procedure abstracts over semantic distinctions that are made in linguistic analyses. For instance, current models of lexical semantics typically assign verbs such as enjoy a nonmetonymic sense when they are combined with NPs that are purely temporal or eventive in nature, as in enjoy the marriage or enjoy the lecture (Copestake and Briscoe 1995; Verspoor 1997). This is largely because a logical form can be constructed in such cases without the use of semantic type coercion; the event-denoting NP itself is the argument to the predicate enjoy. We did not rule out such nouns from our materials, however, as our evaluation was conducted on randomly selected verb-noun pairs.

More generally, we abstract over several criteria that Verspoor (1997) used in dis­tinguishing metonymic from nonmetonymic uses within the corpus, and we adopt a linguistically naive approach for two reasons. First, whereas Verspoor (1997) could deploy more refined criteria because she was hand-selecting the materials from the corpus and was focusing only on two metonymic verbs (begin and finish), our materials were randomly sampled and covered a wider range of metonymic constructions. And second, paraphrases for nonmetonymic cases (e.g., that enjoy the lecture can be para­phrased as enjoy attending the lecture or enjoy listening to the lecture) may be useful for some potential NLP applications (see the discussion in Section 4.2), since they provide

Number of generated interpretations as frequency cutoff for f (v,e) and f (o, e) is varied.

more detailed information about meaning than would be given by a logical form that simply features enjoy(e, x, e'), where e' is the (event) variable that denotes the lecture.

Recall from Section 2.2 that thresholding is an option for the counts f (v, e) and f( o, e) . We derived model paraphrases without employing any thresholds for these counts. Obtaining f (v, e) from the parsed data was relatively straightforward, as there was no structural ambiguity involved. The parser's output was postprocessed to re­move potentially erroneous information, so there was no reason to believe that the frequencies f (v, e) and f (o, e) were noisy. Furthermore, recent work has shown that omitting low-frequency tuples degrades performance for language-learning tasks such as PP attachment (Collins and Brooks 1995; Daelemans, van den Bosch, and Zavrel 1999), grapheme-to-phoneme conversion, POS tagging, and NP chunking (Daelemans, van den Bosch, and Zavrel 1999). For our task, employing thresholds for f (v, e) and f( o, e) dramatically decreases the number of derived interpretations. Table 8 shows the decrease in the number of interpretations as the cutoff for f (v, e) and f (o, e) is varied for five verb-object pairs that were included in our experimental study. Note that discard­ing counts occurring in the corpus only once reduces the number of interpretations by a factor of nearly three. Furthermore, applying frequency cutoffs reduces the range of the obtained probabilities: only likely (but not necessarily plausible) interpretations are obtained with f (o, e) > 4 and f (v, e) > 4. However, one of the aims of the experiment outlined below was to explore the quality of interpretations with varied probabilities. Table 9 displays the 10 most likely paraphrases (and their log-transformed probabili­ties) for finish room as the cutoff for the frequencies f (v, e) and f (o, e) is varied. Notice that applying a cutoff of three or four eliminates plausible interpretations such as dec­orate, wallpaper, furnish, and tidy. This may be particularly harmful for verb-noun (or adjective-noun) combinations that allow for a wide range of interpretations (like finish room).

We estimated the probability P(e, o, v) for each verb-noun pair by varying the term e. In order to generate stimuli covering a wide range of paraphrases corresponding to different degrees of likelihood, for each verb-noun combination we divided the set of generated meanings into three "probability bands" (high, medium, and low) of equal size and randomly chose one interpretation from each band. This division ensured that subjects saw a wide range of paraphrases with different degrees of likelihood.

Our experimental design consisted of two factors: verb-noun pair (Pair) and proba­bility band (Band). The factor Pair included 60 verb-noun combinations, and the factor Band had three levels, high, medium, and low. This yielded a total of Pair x Band = 60 x 3 = 180 stimuli. In order to limit the size of the experiment, the 180 stimuli were administered to two separate groups of subjects. The first group saw meaning paraphrases for the verbs attempt, begin, want, enjoy, try, and expect, whereas the sec-

Ten most likely interpretations for finish room (with log-transformed probabilities) as frequency threshold is varied.

Randomly selected example stimuli with log-transformed probabilities derived by the model.

ond group saw paraphrases for finish, prefer, resist, start, postpone, and survive. Example stimuli are shown in Table 10.

Each experimental item consisted of two sentences, a sentence containing a meto-nymic construction (e.g., Peter started his dinner) and a sentence paraphrasing it (e.g., Peter started eating his dinner). The metonymic sentences and their paraphrases were created by the authors as follows. The selected verb-noun pairs were converted into simple sentences by adding a sentential subject and articles or pronouns where appro­priate. The sentential subjects were familiar proper names (BNC corpus frequency > 30 per million) balanced for gender. All sentences were in the past tense. In the para­phrasing sentences, the metonymy was spelled out by converting the model's output to a verb taking either a progressive or infinitive VP complement (e.g., started to eat or started eating). For verbs allowing both a progressive and an infinitive VP complement, we chose the type of complement with which the verb occurred more frequently in the corpus. A native speaker of English other than the authors was asked to confirm that the metonymic sentences and their paraphrases were syntactically well-formed (items found syntactically odd were modified and retested). Examples of the experimental stimuli the subjects saw are provided in (27) and (28). A complete list of the experimental items is given in Appendix B.

(27) a. high: Michael attempted the peak

Michael attempted to climb the peak b. medium: Michael attempted the peak

Michael attempted to claim the peak c. low: Michael attempted the peak

Michael attempted to include the peak (28) a. high: Jean enjoyed the city

Jean enjoyed living in the city b. medium: Jean enjoyed the city

Jean enjoyed coming to the city c. low: Jean enjoyed the city

Jean enjoyed cutting the city 2.3.1.2 Procedure. The experimental paradigm was magnitude estimation (ME), a technique standardly used in psychophysics to measure judgments of sensory stim­uli (Stevens 1975). The ME procedure requires subjects to estimate the magnitude of physical stimuli by assigning numerical values proportional to the stimulus magni­tude they perceive. Highly reliable judgments can be achieved in this fashion for a wide range of sensory modalities, such as brightness, loudness, or tactile stimulation.

The ME paradigm has been extended successfully to the psychosocial domain (Lodge 1981), and recently Bard, Robertson, and Sorace (1996) and Cowart (1997) showed that linguistic judgments can be elicited in the same way as judgments of sensory or social stimuli. ME requires subjects to assign numbers to a series of linguistic stimuli in a proportional fashion. Subjects are first exposed to a modulus item, to which they assign an arbitrary number. All other stimuli are rated proportional to the modulus. In this way, each subject can establish his own rating scale, thus yielding maximally fine-grained data and avoiding the known problems with the conventional ordinal scales for linguistic data (Bard, Robertson, and Sorace 1996; Cowart 1997; Schutze 1996). In particular, ME does not restrict the range of the responses. No matter which modulus a subject chooses, he or she can subsequently assign a higher or lower judgment by using multiples or fractions of the modulus.

In the present experiment, each subject took part in an experimental session that lasted approximately 20 minutes. The experiment was self-paced, and response times were recorded to allow the data to be screened for anomalies. The experiment was con­ducted remotely over the Internet. Subjects accessed the experiment using their Web browser, which established an Internet connection to the experimental server running WebExp 2.1 (Keller, Corley, and Scheepers 2001), an interactive software package for administering Web-based psychological experiments. (For a discussion of WebExp and the validity of Web-based data, see Appendix A).

2.3.1.3 Instructions. Before participating in the actual experiment, subjects were pre­sented with a set of instructions. The instructions explained the concept of numeric magnitude estimation of line length. Subjects were instructed to make estimates of line length relative to the first line they would see, the reference line. Subjects were told to give the reference line an arbitrary number, and then assign a number to each fol­lowing line so that it represented how long the line was in proportion to the reference line. Several example lines and corresponding numerical estimates were provided to illustrate the concept of proportionality.

The subjects were instructed to judge how well a particular sentence paraphrased another sentence, using the same technique that they had applied to judging line length. Examples of plausible (see (29a)) and implausible (see (29b)) sentence para­phrases were provided, together with examples of numerical estimates.

(29) a. Peter started his dinner Peter started eating his dinner b. Peter started his dinner Peter started writing his dinner

Subjects were informed that they would initially have to assign a number to a reference paraphrase. For each subsequent paraphrase, subjects were asked to assign a number indicating how good or bad that paraphrase was in proportion to the reference.

Subjects were told that they could use any range of positive numbers for their judgments, including decimals. It was stressed that there was no upper or lower limit on the numbers that could be used (exceptions being zero or negative numbers). Subjects were urged to use a wide range of numbers and to distinguish as many degrees of paraphrase plausibility as possible. It was also emphasized that there were no "correct" answers and that subjects should base their judgments on first impressions and not spend too much time thinking about any one paraphrase.

2.3.1.4 Demographic Questionnaire. After the instructions, a short demographic ques­tionnaire was administered. The questionnaire asked subjects to provide their name, e-mail, address, age, sex, handedness, academic subject or occupation, and language region. Handedness was defined as "the hand you prefer to use for writing"; language region was defined as "the place (town, federal state, country) where you learned your first language." 2.3.1.5 Training Phase. The training phase was meant to familiarize subjects with the concept of numeric magnitude estimation using line lengths. Items were presented as horizontal lines, centered in the window of the subject's Web browser. After viewing an item, the subject had to provide a numerical judgment via the computer keyboard. After the subject pressed Return, the current item disappeared and the next item was displayed. There was no opportunity to revisit previous items or change responses once Return had been pressed. No time limit was set either for the item presentation or for the response, although response times were recorded for later inspection.

Subjects first judged the modulus item, and then all the items in the training set. The modulus was the same for all subjects, and it remained on the screen all the time to facilitate comparison. Items were presented in random order, with a new randomization being generated for each subject.

The training set contained six horizontal lines. The range of the shortest to longest item was one to ten (that is, the longest line was ten times the length of the shortest). The items were distributed evenly over this range, with the largest item covering the maximal window width of the Web browser. A modulus item in the middle of the range was provided.

2.3.1.6 Practice Phase. The practice phase enabled subjects to practice magnitude esti­mation of verb-noun paraphrases. Presentation and response procedure was the same as in the training phase, with linguistic stimuli being displayed instead of lines. Each subject judged the whole set of practice items, again presented to him or her in random order.

Correlation of elicited judgments and model-derived probabilities for metonymic verb-noun pairs.

task and allows us to interpret how well the model is doing in relation to humans. To calculate intersubject agreement, we used leave-one-out resampling. The technique is a special case of n-fold cross-validation (Weiss and Kulikowski 1991) and has been previously used for measuring how well humans agree on judging semantic similarity (Resnik and Diab 2000; Resnik 1999).

For each subject group we divided the set of the subjects' responses with size m into a set of size m — 1 (i.e., the response data of all but one subject) and a set of size one (i.e., the response data of a single subject). We then correlated the mean ratings of the former set with the ratings of the latter. This was repeated m times. Since each group had 30 subjects, we performed 30 correlation analyses and report their mean. For the first group of subjects, the average intersubject agreement was .74 (Min = .19, Max = .87, StdDev = .12), and for the second group it was .73 (Min = .49, Max = .87, StdDev = .09). Our model's agreement with the human data is not far from the average human performance of .74.

In the following section we introduce a naive model of verbal metonymy. We compare the naive model's performance against the human judgments and the para­phrases generated by our initial model. We discuss extensions of the basic model in Section 2.5.1.

2.4.1 Naive Baseline Model. In the case of verbal metonymy a naive baseline model can be constructed by simply taking verb-noun co-occurrence data into account, ig­noring thus the dependencies between the polysemous verb and its progressive or infinitival VP complements. Consider the sentence John began the book. In order to gen­erate appropriate paraphrases for begin book, we will consider solely the verbs that take book as their object (i.e., read, write, buy, etc.). This can be simply expressed as P(e | o), the conditional probability of a verb e given its object o (i.e., the noun figuring in the metonymic expression), which we estimate as follows:

Our previous experiments focused on evaluating the plausibility of meaning para­phrases generated by a model that does not take into account the contribution of the sentential subject. However, properties of the subject NP appear to influence the inter­pretation of the metonymic expression in otherwise neutral contexts, as is illustrated in (31), in which the interpretation of enjoy the book is influenced by the sentential subject: Authors usually write books, whereas critics usually review them.

(31) a. The critic enjoyed the book. b. The author enjoyed the book.

In this section we present an extension of the basic model outlined in Section 2.1 that takes sentential subjects into account. We evaluate the derived paraphrases and their likelihood again by comparison with human data. This time we compare our model against paraphrase data generated independently by subjects that participated in an experimental study (McElree et al. 2001) that was not designed specifically to test our model.

2.5.1 The Extended Model. We model the meaning of sentences like (31) again as the joint distribution of the following variables: the metonymic verb v, its subject s, its object o, and the implicit interpretation e. By choosing the ordering (e, v,s,o),wecan factor P(e,o,s,v) as follows:

The terms P(e) and P(v | e) are easy to estimate from the BNC. For P(e) all we need is a POS-tagged corpus and P(v | e) can be estimated from Cass's output (see equa­tions (12) and (13)). The estimation of the terms P(s | e, v) and P(o | e, v, s) is, however, problematic, as they rely on the frequencies f (s, e, v) and f (o, e, v, s), respectively. Re­call that there is a discrepancy between a metonymic interpretation and its usage. As we discussed earlier, metonymic interpretation is not overtly expressed in the corpus. Furthermore, the only type of data available to us for the estimation of P(s | e,v) and P( o | e, v, s) is the partially parsed BNC, which is not annotated with information re­garding the interpretation of metonymies. This means that P(s | e,v) and P(o | e,v,s) need to be approximated somehow. We first assume that the sentential subject s is con­ditionally independent of the metonymic verb v; second, we assume that the sentential object o is conditionally independent of v and s:

The rationale behind the approximation in (33) is that the likelihood of a noun s being a subject of a verb e is largely independent of whether e is the complement of a metonymic verb v. For example, authors usually write, irrespective of whether they enjoy, dislike, start, or finish doing it. The motivation for the approximation in (34) comes from the observation that an object is more closely related to the verb that selects for it than a subject or a metonymic verb. We are likely to come up with book or letter for o if we know that o is the object of read or write. Coming up with an object for o is not so straightforward if all we know is the metonymic verb (e.g., enjoy, finish) or its sentential subject. It is the verbs, rather than other sentential constituents, that impose semantic restrictions on their arguments. We estimate P(s | e) and P(o | e) using maximum likelihood:

The count f(s,e) amounts to the number of times a noun s is attested as the subject of a verb e; f(o,e) represents the number of times a noun is attested as an object of e. Verb-argument tuples can be easily extracted from the BNC using Cass (see Section 2.2 for details). Table 11 illustrates the model's performance for the metonymic constructions in (37). The table shows only the five most likely interpretations the model came up with for each construction. Interestingly, different interpretations are derived for different subjects. Even though pianists and composers are semantically related, pianists are more likely to begin playing a symphony, whereas composers are more likely to conduct or write a symphony. Similarly, builders tend to renovate houses and architects tend to design them.

(37) a. The composer/pianist began the symphony.

b. The author/student started the book.

c. The builder/architect started the house.

d. The secretary/boss finished the memo.

In the following section we compare the interpretations generated by the model against paraphrases provided by humans. More specifically, we explore whether there is a linear relationship between the frequency of an interpretation as determined in a norming study and the probability of the same interpretation as calculated by the model.

2.5.2 Method. For the experiment described in this section, we used the norming data reported in McElree et al. (2001). In McElree et al.'s study subjects were given sentence fragments such as (38) and were asked to complete them. Potential completions for fragment (38a) include writing or reading. The study consisted of 142 different sentences similar to those shown in (38) and included 15 metonymic verbs. Thirty sentences were constructed for each of the metonymic verbs start, begin, complete, and finish, and a total of 22 sentences for attempt, endure, expect, enjoy, fear, master, prefer, resist, savor, survive, and try.

(38) a. The writer finished_the novel.

b. The soldier attempted_the mountain.

c. The teenager finished_the novel.

The completions can be used to determine interpretation preferences for the metonymic constructions simply by counting the verbs that human subjects use to complete sen­tences like those in (38). For example, five completions were provided by the subjects for fragment (38b): climb, hike, scale, walk, and take. Of these climb was by far the most likely, with 78 (out of 88) subjects generating this interpretation. The most likely in­terpretations for (38a) and (38c) were, respectively, write (13 out of 28 subjects) and read (18 out of 22).

For each of the sentences included in McElree et al.'s (2001) study, we derived in­terpretation paraphrases using the model presented in Section 2.5.1. We next compared the interpretations common in the model and the human data.

2.5.3 Results. In Experiment 1 we evaluated the paraphrases generated by the model by eliciting plausibility judgments from subjects and showed that our model produces an intuitively plausible ranking of meanings. Here, we evaluate the quality of the

3 McElree et al.'s (2001) linguistic materials were manually constructed and not controlled for frequency.

For example, one would expect (38b) to be relatively rare, even in a large corpus. This is true for the BNC, in which the combination attempt mountain is not attested at all.

We have demonstrated that the meanings acquired by our probabilistic model corre­late reliably with human intuitions. These meanings go beyond the examples found in the theoretical linguistics literature. The verb-noun combinations we interpret were randomly sampled from a large, balanced corpus providing a rich inventory for their meanings. We have shown that the model has four defining features: (1) It is able to derive intuitive meanings for verb-noun combinations, (2) it generates clusters of meanings (following Vendler's (1968) insight), (3) it predicts variation in interpretation among the different nouns: The same verb may carry different meanings depending on its subject or object (compare begin book to begin house and the author began the house to the architect began the house), and (4) it represents variation in interpretation among the different metonymic verbs (e.g., begin book vs. enjoy book). This latter property demonstrates that although the model does not explicitly encode linguistic constraints for resolving metonymies, it generates interpretations that broadly capture linguistic differences (e.g., attempt imposes different constraints on interpretation from begin or enjoy). Furthermore, these interpretations for metonymic verb-noun pairs are discov­ered automatically, without presupposing the existence of a predefined taxonomy or a knowledge base.

Note that the evaluation procedure to which we subject our model is rather strict. The derived verb-noun combinations were evaluated by subjects naive to linguistic theory. Although verbal logical metonymy is a well-researched phenomenon in the theoretical linguistics literature, the experimental approach advocated here is, to our knowledge, new. Comparison between our model and human judgments yielded a re­liable correlation of .64 when the upper bound for the task (i.e., intersubject agreement) is on average .74. Furthermore, our model performed reliably better than a naive base­line model, which achieved a correlation of only .42. When compared against norming data, an extended version of our model that takes subject information into account reached a correlation of .42. Comparison against norming data is a strict test on unseen data that was not constructed explicitly to evaluate our model but is independently motivated and does not take our corpus (i.e., the BNC) or our particular task into account.

We next investigate whether such an approach generalizes to other instances of logical metonymy by looking at adjective-noun combinations. Adjectives pose a greater challenge for our modeling task, as they can potentially allow for a wider range of interpretations and can exhibit preferences for a verb-subject or verb-object paraphrase (see Section 1). Following the approach we adopted for verbal metonymy, we define the interpretation of polysemous adjective-noun combinations as a paraphrase generation task. We provide a probabilistic model that not only paraphrases adjective-noun pairs (e.g., fast plane) with a related verb (e.g., fly) but also predicts whether the noun modified by the adjective (e.g., plane) is likely to be the verbal object or subject. The model achieves this by combining distributional information about how likely it is for any verb to be modified by the adjective in the adjective-noun combination or its corresponding adverb with information about how likely it is for any verb to take the modified noun as its object or subject. We obtain quantitative information about verb-adjective modification and verb-argument relations from the BNC and evaluate our results by comparing the model's predictions against human judgments. Consistent with our results on verbal metonymy, we show that the model's ranking of meanings correlates reliably with human intuitions.

Consider again the adjective-noun combinations in (39). In order to come up with the interpretation of plane that flies quickly for fast plane, we would like to find in the corpus a sentence whose subject is the noun plane or planes and whose main verb is fly.We would also expect fly to be modified by the adverb fast or quickly. In the general case, we would like to gather from the corpus sentences indicating what planes do fast. Similarly, for the adjective-noun combination fast scientist, we would like to find in the corpus information indicating what the activities that scientists perform fast are, whereas for easy problem we need information about what one can do with problems easily (e.g., one can solve problems easily) or about what problems are (e.g., easy to solve or easy to set).

b. fast scientist c. fast programmer d. easy problem

In sum, in order to come up with a paraphrase of the meaning of an adjective-noun combination, we need to know which verbs take the head noun as their subject or object and are modified by an adverb corresponding to the modifying adjective. This can be expressed as the joint probability P(a, e, n, rel), where e is the verbal predicate modified by the adverb a (directly derived from the adjective present in the adjective-noun combination) bearing the argument relation rel (i.e., subject or object) to the head noun n. By choosing the ordering (e, n, a, rel) for the variables a, e, n, and rel,wecan rewrite P(a, e, n, rel), using the chain rule, as follows:

Although the terms P(e) and P(n | e) can be straightforwardly estimated from the BNC (see (12) for P(e); P(n | e) can be obtained by counting the number of times a noun n co-occurs with a verb e either as its subject or object), the estimation of P(a | e, n) and P(rel | e, n, a) faces problems similar to those for metonymic verbs. Let us consider more closely the term P(rel | e, n, a), which can be estimated as shown in (41).

By substituting equations (45)-(48) into (41) and simplifying the relevant terms, (41) can be rewritten as follows:

Assume we want to discover a meaning paraphrase for the adjective-noun combination fast plane. We need to find the verb e and the relation rel (i.e., subject or object) that maximize the term P(fast, e, plane, rel). Table 12 gives a list of the most frequent verbs modified by the adverb fast in the BNC (see the term f (a, e) in equation (49)), and Table 13 lists the verbs for which the noun plane is the most likely object or subject (see the term f(rel,e,n) in equation (49)). In the following section, we describe how the frequencies f(rel,e,n), f(a,e), and f(e) were estimated from a lemmatized version of the BNC.

Table 12 can be thought of as a list of the activities that can be fast (i.e., going, growing, flying), whereas Table 13 specifies the events associated with the noun plane. Despite our simplifying assumptions, the model given in (49) will come up with plau­sible meanings for adjective-noun combinations like fast plane. Note that the verbs fly, come, and go are most likely to take the noun plane as their subject (see Table 13). These week.

b. One of the programmers was a good athlete, but the other was struggling to finish the courses.

c. The fast programmer came first in the 100m.

verbs also denote activities that are fast (see Table 12, in which the underlined verbs are events that are associated both with the adverb fast and the noun plane). Further note that a subject interpretation is more likely than an object interpretation for fast plane, since none of the verbs likely to have plane as their object are modified by the adverb fast (compare Tables 12 and 13).

As in the case of metonymic verbs, the probabilistic model outlined above ac­quires meanings for polysemous adjective-noun combinations in an unsupervised manner without presupposing annotated corpora or taxonomic information. The ob­tained meanings are not discourse-sensitive; they can be thought of as default semantic information associated with a particular adjective-noun combination. This means that our model is unable to predict that programmer that runs fast is a likely interpretation for fast programmer when the latter is in a context like the one given in (5) (repeated here as (50)).

(50) a. All the office personnel took part in the company sports day last

As in the case of verbs, the parameters of the model were estimated using a part-of-speech-tagged and lemmatized version of the BNC. The counts f (e) and N (see (49)) reduce to the number of times a given verb is attested in the corpus. The frequency f (rel, e, n) was obtained using Abney's (1996) chunk parser Cass (see Section 2.2 for details).

Generally speaking, the frequency f (a, e) represents not only a verb modified by an adverb derived from the adjective in question (see example (51a)), but also con­structions like the ones shown in (51b) and (51c), in which the adjective takes an infinitival VP complement whose logical subject can be realized as a for PP (see ex­ample (51c)). In cases of verb-adverb modification we assume access to morphological information that specifies what counts as a valid adverb for a given adjective. In most cases adverbs are formed by adding the suffix -ly to the base of the adjective (e.g., slow-ly, easy-ly). Some adjectives have identical adverbs (e.g., fast, right). Others have idiosyncratic adverbs (e.g., the adverb of good is well). It is relatively straightforward to develop an automatic process that maps an adjective to its corresponding adverb, modulo exceptions and idiosyncracies; however, in the experiments described in the following sections, this mapping was manually specified.

(51) a. comfortable chair — a chair on which one sits comfortably b. comfortable chair — a chair that is comfortable to sit on c. comfortable chair — a chair that is comfortable for me to sit on

In cases in which the adverb does not immediately succeed the verb, the parser is not guaranteed to produce a correct analysis, since it does not resolve structural ambiguities. So we adopted a conservative strategy, in which to obtain the frequency f (a, e), we looked only at instances in which the verb and the adverbial phrase mod­ifying it were adjacent. More specifically, in cases in which the parser identified an AdvP following a VP, we extracted the verb and the head of the AdvP (see the exam­ples in (52)). In cases where the AdvP was not explicitly identified, we extracted the verb and the adverb immediately following or preceding it (see the examples in (53)), assuming that the verb and the adverb stand in a modification relation. The examples below illustrate the parser's output and the information that was extracted for the frequency f (a, e ).

come slowly

b. [np The issues] [vp will not be resolved] [AdvP easily.] resolve easily

c. [np Arsenal] [vp had been pushed] [AdvP too hard.] push hard write well

b. [np The accidents] [vp could have been easily avoided.] avoid easily

c. [np A system of molecules] [VP is easily shown] [vp to stay constant.] show easily

d. [np Their economy] [vp was so well run.] run well

Adjectives with infinitival complements (see (51b) and (51c)) were extracted from the parser's output. We concentrated solely on adjectives immediately followed by infinitival complements with an optionally intervening for PP (see (51c)). The adjective and the main verb of the infinitival complement were counted as instances of the quantity f (a, e). The examples in (54) illustrate the process.

(54) a. [NP These early experiments] [VP were easy] [VP to interpret.] easy interpret

b. [NP It] [VP is easy] [PP for an artist] [VP to show work independently.] easy show

c. [NP It] [VP is easy] [VP to show] [VP how the components interact.] easy show

Finally, the frequency f (a, e) collapsed the counts from cases in which the adjective was followed by an infinitival complement (see the examples in (54)) and cases in which the verb was modified by the adverb corresponding to the related adjective (see the examples in (52)-(53)). For example, assume that we are interested in the frequency f (easy, show). In this case, we will take into account not only sentences (54b) and (54c), but also sentence (53b). Assuming this was the only evidence in the corpus, the frequency f(easy,show) would be three.

Once we have obtained the frequencies f (a, e) and f (rel, e, n), we can determine what the most likely interpretations for a given adjective-noun combination are. If we know the interpretation preference of a given adjective (i.e., subject or object), we may vary only the term e in P(a, n, rel, e), keeping the terms n, a, and rel constant. Alternatively, we could acquire the interpretation preferences automatically by varying both the terms rel and e. In Experiment 4 (see Section 3.3) we acquire both meaning paraphrases and argument preferences for polysemous adjective-noun combinations.

In what follows we illustrate the properties of the model by applying it to a small number of adjective-noun combinations (displayed in Table 14). The adjective-noun

Paraphrases for adjective-noun combinations taken from the literature.

easy problem — a problem that is easy to solve easy text — text that reads easily difficult language — a language that is difficult to speak, learn, write, understand careful scientist — a scientist who observes, performs, runs experiments carefully comfortable chair — a chair on which one sits comfortably good umbrella — an umbrella that functions well

Object-related interpretations for adjective-noun combinations, ranked in order of likelihood.

understand

Subject-related interpretations for adjective-noun combinations, ranked in order of likelihood.

see read

combinations and their respective interpretations are taken from the lexical semantics literature (i.e., Pustejovsky (1995) and Vendler (1968)). The five most likely model-derived paraphrases for these combinations are shown in Tables 15 and 16.

The model comes up with plausible meanings, some of which overlap with those suggested in the lexical semantics literature (underlined interpretations indicate agree­ment between the model and the literature). Observe that the model predicts different meanings when the same adjective modifies different nouns and derives a cluster of meanings for a single adjective-noun combination. An easy problem is not only a prob­lem that is easy to solve (see Vendler's (1968) identical interpretation in Table 14) but also a problem that is easy to deal with, identify, tackle, and handle. The meaning of easy problem is different from the meaning of easy text, which in turn is easy to read, handle, interpret, and understand. The interpretations the model arrives at for difficult language are a superset of the interpretations suggested by Vendler (1968). The model comes up with the additional meanings language that is difficult to interpret and language that is difficult to use. Although the meanings acquired by the model for careful scientist do not overlap with the ones suggested by Vendler (1968), they seem intuitively plau­sible: a careful scientist is a scientist who calculates, proceeds, investigates, studies, and analyzes carefully. These are all possible actions associated with scientists.

The model derives subject- and object-related interpretations for good umbrella, which is an umbrella that covers well and is good to keep, good for waving, good to hold, good to run for, and good to leave. A subject interpretation can be also derived for easy text. Our parser treats inchoative and noninchoative uses of the same verb as dis­tinct surface structures (e.g., text that one reads easily vs. text that reads easily); as a result, read is generated as a subject and object paraphrase for easy text (compare Tables 15 and 16). The object-related interpretation is nevertheless given a higher probability than the subject-related one, which seems intuitively correct (there is an understood but unexpressed agent in structures like text that reads easily). In general, subject and object interpretations are derived on the basis of verb-subject and verb-object constructions that have been extracted from the corpus heuristically without taking into account in­formation about theta roles, syntactic transformations (with the exception of passiviza-tion), or diathesis alternations such as the middle or causative/inchoative alternation.

Although the model can be used to provide several interpretations for a given adjective-noun combination, not all of these interpretations are useful or plausible (see the subject interpretations for easy text). Also, the meanings acquired by our model are a simplified version of the ones provided in the lexical semantics literature. An adjective-noun combination may be paraphrased with another adjective-noun combi­nation (e.g., a good meal is a tasty meal) or with an NP instead of an adverb (e.g., a fast decision is a decision that takes a short amount of time). We are making the simpli­fying assumption that a polysemous adjective-noun combination can be paraphrased by a sentence consisting of a verb whose argument is the noun with which the adjec­tive is in construction (cf. earlier discussion concerning nonmetonymic uses of verbs like enjoy).

In the following section we evaluate against human judgments the meaning para­phrases generated by the model. As in the case of verbs, the model is tested on examples randomly sampled from the BNC, and the linear relationship between the subjects' rankings and the model-derived probabilities is explored using correlation analysis. In Section 3.5.1 we assess whether our model outperforms a naive baseline in deriving interpretations for metonymic adjectives.

The experimental method in Experiment 4 was the same as that in Experiment 1. Mean­ing paraphrases for adjective-noun combinations were obtained using the model intro­duced in Section 3.1. The model's rankings were compared against paraphrase judg­ments elicited experimentally from human subjects. The comparison between model probabilities and their perceived likelihood enabled us to explore (1) whether there is a linear relationship between the likelihood of a given meaning as derived by the model and its perceived plausibility and (2) whether the model can be used to derive the argument preferences for a given adjective, that is, whether the adjective is biased toward a subject or object interpretation or whether it is equibiased.

3.3.1 Method.

3.3.1.1 Materials and Design. We chose nine adjectives according to a set of minimal criteria and paired each adjective with 10 nouns randomly selected from the BNC. We chose the adjectives as follows: We first compiled a list of all the polysemous adjectives mentioned in the lexical semantics literature (Vendler 1968; Pustejovsky 1995). From these we randomly sampled nine adjectives (difficult, easy, fast, good, hard, right, safe, slow, and wrong). These adjectives had to be relatively unambiguous syntactically: In fact, these nine adjectives were unambiguously tagged as "adjectives" 98.6% of the time, measured as the number of different part-of-speech tags assigned to the word in the BNC. The nine selected adjectives ranged in BNC frequency from 57.6 per million to 1,245 per million.

Correlation of elicited judgments and model-derived probabilities for metonymic adjective-noun pairs.