The SemEval-2010 Task 8 "Multi-Way Classifi­cation of Semantic Relations Between Pairs of Nominals" consists in identifying which seman­tic relation holds between two nominals in a sen­tence (Hendrickx et al., 2010). The set of rela­tions is composed of nine mutually exclusive se­mantic relations and the Other relation. Specifi­cally, the task requires to return the most informa­tive relation between the specified pair of nomi­nals ei and e2 taking into account their order. An­notation guidelines show that semantic knowledge about ei and e2 plays a very important role in dis­tinguishing among different relations. For exam­ple, relations Cause-Effect and Product-Producer are closely related. One of the restrictions which might help to distinguish between them is that products must be concrete physical entities, while effects must not.

Recently, there has emerged a large number of freely available large-scale knowledge bases. The ground idea of our research is to use them as source of semantic information. Among such resources there are DBpedia, YAGO, and Open-Cyc. On the one hand, DBpedia and YAGO have been automatically extracted from Wikipedia. They have a good coverage of named entities, but their coverage of common nouns is poorer. They seem to be more suitable for relation extraction be­tween named entities. On the other hand, Cyc is a manually designed knowledge base, which de­scribes actions and entities both in common life and in specific domains (Fenat, 1995). Cyc has a good coverage of common nouns, making it in­teresting for our task. The full version of Cyc is freely available to the research community as Re­searchCyc.

We approached the task using the system intro­duced by Giuliano et al. (2007) as a basis. They exploited two information sources: the whole sen­tence where the relation appears, and WordNet synonymy and hyperonymy information. In this paper, we (i) investigate usage of Cyc as a source of semantic knowledge and (ii) linguistic infor­mation, which give useful clues to semantic re­lation extraction. From Cyc, we obtain informa­tion about super-classes (in the Cyc terminology generalizations) of the classes which correspond to nominals in a sentence. The sentence itself provides linguistic information, such as local con­texts of entities, bag of verbs and distance between nominals in the context.

The different sources of information are rep­resented by kernel functions. The final system is based on four kernels (i.e., local context ker­nel, distance kernel, verbs kernel and generaliza­tion kernel). The experiments were carried out us­ing support vector machines (Vapnik, 1998) as a classifier. The system achieves an overall F\ of

!http://dbpedia.org/

http ://www.mpi-inf.mpg.de/yago-naga/ yago/

77.62%.

In order to implement the approach based on shal­low syntactic and semantic information, we em­ployed a linear combination of kernels, using the support vector machines as a classifier. We de­veloped two types of basic kernels: syntactic and semantic kernels. They were combined by exploit­ing the closure properties of kernels. We define the composite kernel Kc(x\,X2) as follows.

Each basic kernel K% is normalized.

All the basic kernels are explicitly calculated as follows where <p(-) is the embedding vector. The resulting feature space has high dimensionality. However, Equation 2 can be efficiently computed explicitly because the representations of input are extremely sparse.

Sentences have been tokenized, lemmatized and PoS tagged with TextPro. Information for gener­alization kernel has been obtained from Research-Cyc. All the experiments were performed using jSRE customized to embed our kernels. jSRE uses the SVM package FIBSVM (Chang and Fin, 2001). The task is casted as multi-class classifica­tion problem with 19 classes (2 classes for each relation to encode the directionality and 1 class to encode Other). The multiple classification task is handled with One-Against-One technique. The SVM parameters have been set as follows. The cost-factor Wi for a given class i is set to be the ratio between the number of negative and positive examples. We used two values of regularization parameter C: (i) Cdef = ^ where x are all examples from the training set, (ii) optimized G grid value obtained by brute-force grid search method. The default value is used for the other parameters.

Table 1 shows the performance of different ker­nel combinations, trained on 8000 training exam­ples, on the test set. The system achieves the best overall macro-average F\ of 77.62% using Klc + Kv + KD + Kgenis. Figure 1 shows the learning curves on the test set. Our experimen­tal study has shown that the size of the training

http ://textpro.fbk.eu/

Figure 1 : Fearning curves on the test set per rela­tion

Table 1: Performance on the test set. Combina­tions marked with * were run with Cgrid, others with Cdef.

set influences the performance of the system. We observe that when the system is trained on 8000 examples the overall F\ increases for 14.01% as compared to the case of 1000 examples.

The experiments have shown that Klc is the core kernel of our approach. It has good performance on its own. For instance, it achieves precision of 66.16%, recall 72.67% and Fx of 69.13% evalu­ated using 10-fold cross-validation on the training set.

The paper describes a system for semantic rela­tions extraction, based on the usage of semantic information provided by ResearchCyc and shal­low syntactic features. The experiments have shown that the external knowledge, encoded as super-class information from ResearchCyc with­out any word sense disambiguation, significantly contributes to improve overall performance of the system. The problem of the lack of coverage may be overcome by the usage of other large-scale knowledge bases, such as DBpedia. For future work, we will try to use the Cyc inference en­gine to obtain implicit information about nominals in addition to the information about their super­classes and perform word sense disambiguation.

The research leading to these results has received funding from the ITCH project (http://itch.fbk.eu), spon­sored by the Italian Ministry of University and Research and by the Autonomous Province of Trento and the Copilosk project (http : / /copilosk . fbk. eu), a Joint Research Project under Future Internet - Internet of Content program of the Information Technology Center, Fondazione Bruno Kessler.