Determination of semantic similarity between words is an important component of linguis­tic tasks ranging from text retrieval and filter­ing, word sense disambiguation or text match­ing. In the past five years, this work has evolved in conjunction with the availability of powerful computers and large linguistic resources such as WordNet (Miller,90), the EDR concept dictionary (EDR,93), and large text corpora.

Similarity methods can be broadly divided into "relation based" methods which use relations in an ontology to determine similarity and "distribu­tion based" methods which use statistical analysis as the basis of similarity judgements. This article describes a new method of similarity matching, in­herited feature based similarity matching (IFSM) which integrates these two approaches.

Relation based methods include both depth based and path based measures of similarity. The Most Specific Common Abstraction (MSGA) method compares two concepts based on the tax-onomic depth of their common parent; for exam­ple, "dolphin" and "human" are more similar than "oak" and "human" because the common concept "mammal" is deeper in the taxonomy than "living thing".

Zhonghui Xu, Kenneth Haase

MIT Media Laboratory 20 Ames Street Cambridge, MA 02139 USA {xu,haase}Omedia.mit.edu

Path-length similarity methods are based on counting the links between nodes in a semantic network. (Rada,89) is a widely adopted approach to such matching and (Sussna,93) combines it with WordNet to do semantic disambiguation.

The chief problems with relation-based similar­ity methods lie in their sensitivity to artifacts in the coding of the ontology. For instance, MSG A algorithms are sensitive to the relative depth and detail of different parts of the concept taxon­omy. If one conceptual domain (say plants) is sketchily represented while another conceptual do­main (say, animals) is richly represented, similar­ity comparisons within the two domains will be in­commensurable. A similar problem plagues path-length based algorithms, causing nodes in richly struct ured parts of the ontology to be consistently judged less similar to one another than nodes in shallower or less complete parts of the ontology.

Distribution-based methods are based on the idea that the similarity of words can be derived from the similarity of the contexts in which they occur. These methods differ most significantly in the way they characterize contexts and the similarity of contexts. Word Space (Schutze,93) uses letter 4-grams to characterize both words and the contexts in which they appear. Similarity is based on 4-grams in common between the con­texts. Church and Ranks ('89) uses a word win­dow of set size to characterize the context of a word based on the immediately adjacent words. Other methods include the use of expensive-to-derive features such a.s subject-verb-object (SVO) relations (Hindle,90) or other grammatical rela­tions (Grefenstette,94). These choices are not sim­ply implementational but imply different similar­ity judgements. The chief problem with distribu­tion based methods is that they only permit the formation of first-order concepts definable directly in terms of the original text. Distribution based methods can acquire concepts based on recurring patterns of words but not on recurring patterns of concepts. For instance, a distributional sys­tem could easily identify that an article involves lawyers based on recurring instances of words like "sue" or "court". But it could not use the oc­currence of these concepts as conceptual cues for developing concepts like "litigation" or "pleading" in connection with the "lawyer" concept.

One notable integration of relation based and distributional methods is Rcsnik's annotation of a relational ontology with distributional informa­tion (Resnik,95a,95b). Resnik introduces a "class probability" associated with nodes (synsels) in WordNet and uses these to détermine similarity. Given these probabilities, lie computes the simi­larity of concepts based on the "information" that would be necessary to distinguish them, measured using information-theoretic calculations.

The Inherited Feature Similarity Measure (IFSiYl) is another integrated approach to measuring simi­larity. It uses a semantic knowledge base where concepts are annotated with distingmsh'ing fea­tures and bases similarity on comparing these sets of features. In our experiments, we derived the feature sets by a distributional analysis of a large corpus.

Most existing relation-based similarity methods directly use the relation topology of the seman­tic network to derive similarity, either by strate­gies like link counting (Racla,89) or the determina­tion of the dept h of common abstractions (Kolod-ner,89). IFSM, in contrast, uses the topology to derive descriptions whose comparison yields a sim­ilarity measure. In particular, il assumes an on­tology where:

!.. Fach concept has a set of features

2. Fach concept inherits feature's from its gen­eralizations (hypernyms)

3. Fach concept lias one or more "distinctive features" which are not inherited from ils hy­pernyms.

Note that we neither claim nor require that the features completely characterize their concepts or that inheritance of features is sound. We only re­quire that there be some set of features we use for similarity judgements. For instance, a similarity judgement between a penguin and a robin will be partially based on the feature "can-fly" assigned to the concept bird, even though it does not apply individually to penguins.

Fig I shows a simple example of a fragment of a conceptual taxonomy with associated features. Inherited features arc in italic while distinctive

iiiiliiT(< nmle x/inve-cliild >) iih)IIici(<; tannic >< haw-child

Fig. I Fragment of conceptual taxonomy features arc in bold. In our model, features have a weight based on the importance of the feat ure to (he concept.

We have chosen to automatically generate fea­tures distributionally by analyzing a large corpus. We describe this generation process below, but we will lirst turn to the evaluation of similarity based on featural analysis.

IFSM consists of a hierarchical conceptual the­saurus, a set of distinctive features assigned to each object and weightings of the features. We can use, for example, WordNet or the EDR con­cept dictionary as a hierarchical conceptual the­saurus. Currently, there are no explicit methods to determine sets of distinctive features and their weightings of each object (word or concept).

Here we adopt an automatic extraction of fea­tures and their weightings from a large text cor­pus. This is the same approach as that of the dis­tributed semantic models. However, in contrast to those models, here we hope to make the level of the representation of features high enough to capture semantic behaviors of objects.

For example, if one relation and one object can be said to describe the features of object, we can define one feature of "human" as "agent of walk­ing". If more context is allowed, we can define a feature of "human" as "agent of utilizing fire". A wider context gives a precision to the contents of the features. However, a wider context expo­nentially increases the possible number of features which will exceed current limitations of computa­tional resources. In consideration of these factors, we adopts triple relations such as "dog chase cat", "cut paper with scissors" obtained from the cor-

.....X\..................^.........................y..................

Surface Triples - -

Merged Surface Triples

■ ......1...............1...............1/4...........

Filtering Heuristics

Fig.4 Abstracted triple extraction from corpus

pus as a resource of features, and apply class based abstraction (Resnik 95a) to triples to reduce the size of the possible feature space.

As mentioned above, features extracted from the corpus will be represented using synsets/concepts in IFSM. Since no large scale corpus data with semantic tags is available, the current implemen­tation of IFSM has a word sense disambiguation problem in obtaining class probabilities. Our cur­rent basic strategy to this problem is similar to (Resnik,95a) in the sense that synsets associated with one word are assigned uniform frequency or "credit" when that word appears in the corpus. We call this strategy the "brute-force" approach, like Resnik. On top of this strategy, we introduce filtering heuristics which sort out unreliable data using heuristics based on the statistical properties of the data.

This section describes the feature extraction pro­cedure. If a sentence "a dog chased a cat" ap­pears in the corpus, features representing "chase cat" and "dog chase" may be attached to "dog" and "cat" respectively. Fig 4 shows the overall process used to obtain a set of abstracted triples which are sources of feature and weighting sets for synsets.

{greet recognize}/"express greetings upon meeting .." {ducproccss due_process_of_law}

{allege aver say}/" lie alleged that we was the victim ." [female femalcperson} (4,1,3)

(refuse reject pass^up turn„down decline}/" refuse to accept;" {request petition solicitation] (0,0.25,G)

5 {animal nnilnntejx:ing beast brute creature fauna}/ (win gain}/"win something through one's efforts" {contest competition} (5,0.49,0)

"()" shows (# of Surface-Support ,Frequency,Occurrences) Table 3: Example of abstracted triples

abstract triple holds frequency, occurrence num­ber, and word list which supports each of throe abstracted synsets.

A filtering heuristic thai eliminates abstract triples whose surface supporl is three (i.e., sup­ported by only one surface pattern) is applied to each set of abstracted triples, and results in the following sizes of abstracted triple sets in the 320K (level 500), 150K (level 1500) and 5OK (level 3000) respectively. Kach triple is assigned a evaluation score which is a sum of normalized surface support score (= surface support, score/maximum surface support score) and normalized frequency (■— fre­quency / maximum frequency). Table 3 shows the top five abstracted triples with respect to their evaluation scores. Items in the table shows subject synset, verb synset, object synset, surface sup­port, frequency and occurrence numbers.

All the subjects in the top five abstract triples of level 500 are "organization". This seems to be reasonable because the contents of the 101 0 corpus are news articles and these triples seem to show some highly abstract briefing of the contents of the corpus.

The effectiveness of the filtering and /or scoring heuristics can be measured using t wo closely re­lated criteria. One measures the plausibility of abstracted triples i.e., the recall and precision ra­tio of the plausible abstracted triples, 'fhe other criteria shows the correctness of the mappings of the surface triple patterns to abstracted triples.

varsity UnitecLNations team subsidiary State state staff so­viet school Politburo police patrol party panel Organization order operation newspaper mission Ministry member maga­zine line law..firm law land Justice_l)epartment jury industry house headquarters government gang 1TIA division court coun­try council Conference company committee college club Cabi­net business board association Association airline Table 4. Surface supports of "orgjuiization'

Tins is measured by counting the correct surface supports of each abstracted triple, for example, considering a set of surface words supporting "or­ganization" of the f of level 500 shown in table 4, the word "panel" might be used as "panel board". This abilily is also measured by developing the word sense disambiguate)!' which inputs the sur­face triple and selects the most, plausible deep triple based on abstracted triple scores matched with the dee}) triple. The surface supports in Ta­ble 4 show the intuitive tendency that a sufficient number of triple data will generate solid results.

This paper described a similarity calculation model between objects based on common and dis­tinctive features and proposes an implementation procedure! for obtaining features based on abstract triples extracted from a large text corpus (1010 corpus) utilizing taxonoinical knowledge (Word-Net). The experiment,, which used around 80K surface triples, shows that the abstraction level 3000 provides a good basis for feature; descrip­tion. A feature! extraction experiment based on large; triple data is our next ge)ab