SRD follows four steps in acquiring features:

• Select semantic generalizations. For each noun-sense in a pair, SRD extracts all hyper-nyms at depth 8 or higher in the WordNet noun-sense hierarchy.

• Extract syntactic phrases. SRD looks for phrases in corpora that occur before or after each noun in a pair and which obey one of several syntactic templates. SRD also looks for joining phrases between each pair of nouns that contain 5 words or less.

• Clean-up these phrases. SRD lemmatizes the words in each phrase and removes function words such as articles, possessive pronouns, adjective and adverbs.

• Record observed patterns. For each noun pair, SRD records the following types of syn­tactic patterns together with their corpus fre­quencies: joining terms that comprise at least one verb; phrases that are composed of one verb and one preposition; and phrases that are composed of a simple verb or a phrasal verb.

Due to the large number of features extracted in these steps, SRD employs five different models that use different combination of features and which pool their votes to determine a single predic­ation for each learning task. We describe below the feature sets used for each component. The features have binary values: 1 if the feature is present for a noun pair, and 0 otherwise.

Each model employs WordNet hypernyms (from the top 8 layers of the noun hierarchy) of both noun-senses as semantic features, while models 1 and 2 employ the following additional features for each noun pair (N1, N2):

1. The most frequent syntactic patterns that appear between N1 and N2 in corpora

2. The most frequent syntactic patterns that appear between N2 and N1 in corpora

Model 1 and Model 2 differ only in the syntactic templates used to validate inter-noun patterns. Model 1 fixates on patterns that contain a verb, while Model 2 accepts patterns that contain either a preposition or a verb, or both. This yields, on aver­age, 5,000 binary features for Model 1 for each of the seven relation types, and an average of 10,000 binary features for Model 2.

In addition to WN-derived hypernymic-features, models 3 and 4 employ the following:

1. The most frequent syntactic patterns that immediately precede N1 in a corpus

2. The most frequent syntactic patterns that immediately follow N1 in a corpus

3. The most frequent syntactic patterns that immediately precede N2 in a corpus

4. The most frequent syntactic patterns that immediately follow N2 in a corpus

In Model 3 each syntactic pattern comprises a hyphenated verb, while the syntactic patterns in Model 4 each contain a preposition or a verb. SRD generates, on average, 1,500 binary features in Model 3 and 2,500 features in Model 4 for each re­lation-type.

In addition to WN-derived hypernymic-features, model 5 employs the following:

1. A set of linguistic features for N1, indicat­ing whether this noun is a nominalized verb, or whether it frequently appears in a specific semantic case role (e.g., agent).

2. The same set of linguistic features as de­termined for N2.

SRD generates, on average, approximately 700 binary features for each relation-type in Model 5.

The SVM learning paradigm seems particularly suitable to our task for a number of reasons. Firstly, it behaves robustly for all seven learning tasks, ignoring the noise in the training set. This is important, since e.g., some training pairs for the In­strument-Agency relation were labeled as both true and false. Secondly, SVM has an automated mech­anism for parameter tuning, which reduces the overall computational effort.

SRD employs polynomial SVMs because they appear to perform better for this task compared with simple linear SVMs or radial-basis functions. We used the WEKA implementation of John Plat-t's Sequential Minimal Optimization method (Platt, 1998) to train the feature weights on all the avail­able training data. Using SMO to train the polyno­mial SVM takes approx. 2.8 CPU sec. per model.

The motivation for a multiple model scheme ap­proach comes from empirical results. SRD yields higher results relative to the five single models schemes that compose our system when evaluated using 10-fold cross validation on the training data.

Three types of baseline values were proposed for this task. Baseline 1 ("majority baseline") is ob­tained by always guessing either "true" or "false", according to whichever is the majority category in the testing data-set for the given relation. Baseline 2 ("alltrue baseline") is achieved by always guess­ing "true". Baseline 3 ("probmatch baseline") is obtained by randomly guessing "true" or "false" with a probability matching the distribution of "true" or "false" in the testing dataset.

classl class2 class3 class4 class5 class6 class7

Figure1. Comparison of SRD's F-scores for each se­mantic relation and the corresponding baselines.

Figure 1 plots the F-scores obtained for each se­mantic relation. We observe that SRD has exhibits poor performance on two particular relations, Ori­gin-Entity and Theme-Tool, denoted "class4" and "class5" in the plot of Figure 1. SRD achieves the same F-measure score as the random prediction baseline for Theme-Tool class, suggesting that the features used are simply not capable of building a discriminator for this semantic relation. SRD's F-score for Origin-Entity class is 10% higher than the random baseline, but still performs below the other two baselines. SRD's best performance is achieved for Product-Producer and Part-Whole, with an F-score 11% higher than the highest baseline.