The central resource is a collection of all gram­matical relations holding between words in the corpus. The corpus currently used in WASP-Bench is the British National Corpus (BNC): . Using finite-state techniques operating over part-of-speech tags, we process the whole cor­pus finding quintuples of the form: {Rel, Wl, W2, Prep, Position}, where Rel is a relation, Wl is the lemma of the word for which Rel holds, W2 is the lemma of the other open-class word involved, Prep is the preposition or parti­cle involved and Position is the position of Wl in the corpus. Relations may have null values for W2 and Prep. The database contains 70 million quintuples.

The current inventory of relations is shown in Table 1. All inverse relations, ie. subject-of etc, found by taking W2 as the head word in­stead of Wl are explicitly represented, to give a total of twenty-six distinct relations. These pro­vide a flexible resource to be used as the basis of the computations of the workbench. Keeping the position numbers of examples allows us to find associations between relations and to dis­play examples.

The user enters the word and using the gram­matical relations database, the system com­poses a word sketch for this word. This is a page of data such as Table 2, which shows, for the word in question (Wl), ordered lists of high-salience grammatical relations, relation-W2 pairs, and relation-W2-Prep triples for the word.

The number of patterns shown is set by the user, but will typically be over 200. These are listed for each relation in order of salience, with the count of corpus instances. The instances can be instantly retrieved and shown in a con­cordance window. Producing a word sketch for a medium-to-high frequency word takes in the order of ten seconds.

Salience is calculated as the product of Mu­tual Information I (Church and Hanks, 1989) and log frequency. I for a word W1 in a gram­matical relation Rel with a second word W2 is calculated as:

The notation here is adopted from (Lin, 1998) (who also spells out the derivation from the definition of J). \\Wl, Rel,W2\\ denotes the frequency count of the triple {Wl, Rel, W2}in the grammatical relations database. Where Wl, Rel or W2 is the wild card (*), the fre­quency is of all the dependency triples that match the remainder of the pattern.

The word sketches are presented to the user as a list of relations, with items in each list or­dered according to salience. Our experience of working lexicographers' use of Mutual Informa­tion or log-likelihood lists shows that, for lex­icographic purposes, these over-emphasise low frequency items, and that multiplying by log frequency is an appropriate adjustment.

The next task is to enter a preliminary list of senses for the word, possibly in the form of some arbitrary mnemonics: for example, MONEY, CLOUD and RIVER for three senses of bank. This inventory may be drawn from the user's knowledge, from a perusal of the word sketch, or from a pre-existing dictionary entry.

As Table 2 shows, and in keeping with "one sense per collocation" (Yarowsky, 1993) in most cases, high-salience patterns or clues indicate just one of the word's senses. The user then has the task of associating, by selecting from a pop-up menu, the required sense for unam­biguous clues. The number of relations marked will depend on the time available, as well as the complexity of the sense division to be made. The act of assigning senses to patterns may very well lead the user to discover fresh, unconsid­ered senses usages of the word.

The pattern-sense associations are then sub­mitted to the next stage: automatic disam­biguation.

The workbench currently uses Yarowsky's de­cision list approach to WSD (Yarowsky, 1995). This is a bootstrapping algorithm that, given some initial seeding, iteratively divides the corpus examples into the different senses. Yarowsky notes that the most effective ini­tial seeding option he considered was labelling salient corpus collocates with different senses. The user's first interaction with the workbench is just this.

At the user-input stage, only clues involving grammatical relations are used. At the WSD al­gorithm stage, some "bag-of-words" and n-gram clues are also considered. Any content word (lemmatised) occurring within a A:-word window of the nodeword is a bag-of-words clue. N-gram clues capture local context which may not be covered by any grammatical relation. The n-gram clues are all bigrams and trigrams in­cluding the nodeword. iV-grams and context-word clues frequently duplicate the grammati­cal relations already found, but the merit of the decision list approach is that probabilities are not combined, so such dependencies are not a problem.

The output of the algorithm is both a sense dis­ambiguated corpus, and a decision list. The de­cision list can be viewed as a lexical entry or as a WSD program. It will contain {Rel, W2} pairs (as in the original word sketch), bag-of-words words, and n-grams. The components of the decision list which assign to a particular sense can be displayed as "sense profiles", in a manner comparable to the original word sketch. They will contain new clues, not originally seen in the word sketch and may point to new senses or usages needing addition to the lexical entry Users can then re-run the WSD algorithm, it­erating until they are satisfied with the sense inventory, and with the accuracy of the disam­biguation performed.

For the last two years, a set of 6000 word sketches has been used in a large dictio­nary project (Rundell, 2002), with a team of thirty professional lexicographers covering ev­ery medium-to-high frequency noun, verb and adjective of English. The feedback received is that they are hugely useful, and transform the way the lexicographer uses the corpus. They radically reduce the amount of time the lex­icographers need to spend reading individual instances, and give the dictionary improved claims to completeness, as common patterns are far less likely to be missed.

Performance as a WSD system was evaluated on the SENSEVAL-2 English lexical sample exercise.

The words to be tested were divided between the first author and one paid volunteer, who had no previous experience of WASP-Bench. We carried out the procedure as above, with the difference that instead of having to establish a sense inventory, the inventory was already given as that of WordNet. After assigning sufficient clues to cover the various senses, these assign­ments were submitted as seeds to the disam­biguation algorithm. Using the example sen­tences from the BNC this gave us a decision list of clues, which could then be used to disam­biguate the test sentences.