For each training and test sentence, we reduce the annotated target entities el and e2 to single nouns nounl and noun2, by keeping their last nouns only, which we assume to be heads.

We create a global feature set for all relations. The features extracted are of three types, i.e., lex­ical, morpho-syntactic and semantic. The feature set consists of the following 6 types of features.

Feature set 1: Lemma of target entities el and el. The lemma of the entities annotated in the given sentence.

Feature set 2: Stem and POS of words be­tween el and el. The stem and POS tag of the words between two nominals. First all the words between two nominals were extracted and then the Porter's stemming was performed to reduce words to their base forms (Porter, 1980). Meanwhile, OpenNLP postag tool was used to return part-of-speech tagging for each word.

Feature set 3: syntactic pattern derived from syntactic parser between el and el. Typically, the verb phrase or preposition phrase which con­tain the nominals are important for relation clas­sification. Therefore, OpenNLP Parser was per­formed to do full syntactic parsing for each sen­tence. Then for each nominal, we look for its par­ent node in the syntactic tree until the parent node is a verb phrase or preposition phrase. Then the label of this phrase and the verb or preposition of this phrase were extracted as the syntactic features.

Besides, we also extracted other 3 feature types with the aid of WordNet.

Feature set 4: WordNet semantic class of el and el. The WordNet semantic class of each an­notated entity in the relation. If the nominal has two and more words, then we examine the seman­tic class of "wl-w2" in WordNet. If no result re­turned from WordNet, we examine the semantic class of head in the nominal. Since the cost of manually WSD is expensive, the system simply used the first (most frequent) noun senses for those words.

Feature set 5: meronym-holonym relation between el and el. The meronym-holonym relation between nominals. These information are quite important for component-whole and Member-Collection relations. WordNet3.0 provides meronym and holonym information for some nouns. The features are extracted in the fol­lowing steps. First, for nominal el, we extract its holonym from WN and for nominal e2, we extract its Synonyms/Hypernyms. Then, the system will check if there is same word between el's holonym and e2's synonym & hypernym. The yes or no result will be a binary feature. If yes, we also ex­amine the type of this match is "part-of" or "mem­berjof" in holonym result. Then this type is also a binary feature. After that, we exchange the posi­tion of el and e2 and perform the same process­ing. By creating these features, the system can also take the direction of relations into account.

Feature set 6: hyponym-hypernym rela­tion between nominal and the word of "con­tainer". This feature is designed for Content-Container relation. For each nominal, WordNet returns its hypernym set. Then the system examine if the hypernym set contains the word "container". The result leads to a binary feature.

Our system is to build up a global classifier based on global feature set for all 9 non-Other relations. Generally, for this multi-class task, there are two strategies for building classifier, which both con­struct classifier on a global feature set. The first scheme is to treat this multi-class task as an multi-way classification. Since each pair of nominals corresponds to one relation, i.e., single label clas­sification, we build up a 10-way SVM classifier for all 10 relations. Here, we call it multi-way clas­sification. That is, the system will construct one single global classifier which can classify 10 rela­tions simultaneously in a run. The second scheme is to split this multi-class task into multiple binary classification tasks. Thus, we build 9 binary SVM classifiers, one for each non-Other relation. Noted that in both strategies the classifiers are built on global feature set for all relations. For the sec­ond multiple binary classification, we also experimented on different prob, thresholds, i.e., 0.25 and 0.5. Furthermore, in order to reduce errors and boost performance, we also adopt the major­ity voting strategy to combine different classifiers.

The classifiers for all relations were optimized independently in a number of 10-fold cross-validation (CV) experiments on the provided train­ing sets. The feature sets and learning algorithms which were found to obtain the highest accuracies for each relation were then used when applying the classifiers to the unseen test data.

Among the above 7 system, SR5 system shows the best macro-averaged Fl measure. Table 2 de­scribes the statistics and performance obtained per relation on the SR5 system.

Table 3 shows the performance of these 7 sys­tems on the test data as a function of training set size.

The first three systems are based on three feature sets, i.e.,Fl-F3, with different classification strat­egy. The next three systems are based on all six feature sets with different classification strategy. The last system adopts majority voting scheme on the results of four systems, i.e., SRI, SR2, SR4 and SR5. Based on the above series of exper­iments and results shown in the above 3 tables, some interesting observations can be found as fol­lows.

Obviously, although we did not perform WSD on each nominal and only took the first noun sense as semantic class, WordNet significantly improved the performance. This result is consistent with many previous work on Semeval 2007 Task 4 and once again it shows that WordNet is important for semantic relation classification. Specifically, whether for multi-way classification or multiple binary classification, the systems involved features extracted from WordNet performed better than the others not involved WN, for example, SR4 better than SRI (74.82% vs 60.08%), SR5 better than SR2 (75.43% vs 72.59%), SR6 better than SR3 (72.19% vs 68.50%).

Generally, the performance of multiple binary classifier is better than multi-way classifier. That means, given a global feature set for 9 relations, the performance of 9 binary classifiers is better than a 10-way classifier. Specifically, when F1-F3 are involved, SR2 (72.59%) and SR3 (68.50%) are both better than SRI (60.08%). However, when F1-F6 feature sets are involved, the performance of SR4 is between that of SR5 and SR6 in terms of macro-averaged F\ measure. With respect to ac­curacy measure (Acc), SR4 system performs the best.

Moreover, for multiple binary classification, the threshold of probability has impact on the perfor­mance. Generally, the system with prob, threshold 0. 25 is better than that with 0.5, for example, SR2 better than SR3 (72.59% vs 68.50%), SR5 better than SR6 (75.43% vs 72.19%).

As an ensemble system, SR7 combines the re­sults of SRI, SR2, SR4 and SR5. However, this majority voting strategy has not shown significant improvements. The possible reason may be that these classifiers come from a family of SVM clas­sifiers and thus the random errors are not signifi­cantly different.

Besides, one interesting observation is that SR4 system achieved the top 2 performance on TD1 data amongst all participating systems. This shows that, even with less training data, SR4 sys­tem achieves good performance.