Automatic semantic interpretations of natural lan­guage text rely on (1) semantic theories that cap­ture the subtleties employed by human communi­cations; (2) lexico-semantic resources that encode various forms of semantic knowledge; and (3) com­putational methods that model the selection of the optimal interpretation derived from the textual data. Two of the SemEval 2007 tasks, namely Task 4 (Classification of Semantic Relations between Nom-inals)and Task 8 (Metonymy Resolution) employed distinct theories for the interpretation of their cor­responding semantic phenomena, but, nevertheless, they also shared several lexico-semantic resources, and, furthermore, both these tasks could have been cast as classification problems, in vein with most of the recent work in computational semantic process­ing. Based on this observation, we have designed and implemented a semantic architecture that was used in both tasks. In Section 2 of this paper we give a brief description of the semantic theories cor­responding to each of the two tasks, while in Section 3 we detail the semantic architecture. Section 4 de­scribes the experimental results and evaluation.

We have used three lexico-semantic resources: (i) the WordNet lexico-semantic database; (ii) VerbNet; and (iii) the Lexical Conceptual Structure (LCS) database. Used only by Task 4, WordNet is a lexico-semantic database created at Princeton University(Fellbaum, 1998), which encodes a vast majority of the English nouns, verbs, adjectives and adverbs, and groups synonym words into synsets. VerbNetis a broad-coverage, comprehensive verb lexicon created at University of Pennsylvania, compatible with WordNet, but with explicitly stated syntactic and semantic information, using Levin verb classes (Levin, 1993) to systematically construct lexical en­tities. Classes are hierarchically organized and each class in the hierarchy has its corresponding syntac­tic frames, semantic predicates and a list of typical verb arguments. The Lexical Conceptual Structure (Traum and Habash, 2000) is a compositional ab­straction with language-independent properties. An LCS is a directed graph with a root. Each node is as­sociated with certain information, including a type, a primitive and a field. An LCS captures the semantics of a lexical item through a combination of semantic structure and semantic content.

The two semantic tasks addressed in this paper are: Classification of Semantic Relations between Nominals (Task 4), defined in (Girju et al., 2007) and Metonymy Resolution (Task 8), defined in (Markert and Nissim, 2007). Please refer to these task description papers for more details. Both are cast as classification tasks: given an unlabeled in­stance, a system must label it according to one class of a set specific to each task.

The training and testing datasets for the metonymy resolution task are annotated in an XML format. There are 1090 training and 842 testing instances for companies, and 941 training and 908 testing instances for locations. Each training instance corresponds to a context in which a single name is annotated with its read­ing (metonymic/literal/mixed) and, in case of metonymy, its type (metotype). The testing dataset for this task is annotated in a similar manner, only the reading of the name is left unknown and must be decided by the system.

For the classification of semantic relations be­tween nominals, there exist seven training sets of 140 instances each for the seven semantic relations, and seven corresponding testing sets of around 70 instances each. A training instance is annotated with information about the boundaries of the two nom-inals whose relation must be determined, the truth value of their relation, the WordNet sense of each nominal, and the query that was employed by the an-notators to retrieve this example from the Web. The testing instances are similar, with the only difference being that the truth value of the relations is unknown and must be determined.

The semantic architecture that we have designed is illustrated in Figure 1, which contains the basic modules and resources used in the various phases of processing the input data towards the final submis­sion format. The grayed-out modules are all used only for the semantic relations classification task, while the part of the figure represented by dotted lines appears only in the metonymy resolution al­gorithm. The input to the system, for both tasks, comprises the annotated instances, either from the training or the testing dataset. Before any feature is extracted, the data passes through a pipeline of pre­processing modules. The text is first split into tokens in a heuristic manner. The resulting tokenized text is given as input to Brill's part of speech tagger, which associates each word with its part of speech (e.g., NN, PRP). The data further goes through Collins' syntactic parser, which builds the syntactic trees for all the sentences in the text.

Additionally, for semantic relations classification, the system creates the dependency structures for all the sentences, using the dependency parser built at Stanford and described in (de Marneffe et al., 2006). The dependency parser extracts some of 48 grammatical relations for each pair of words in a sentence. A second module that is specific only to this task is (Surdeanu and Turmo, 2005)'s seman­tic role labeler, which extracts the shallow seman­tic structure for each sentence, that is, the predicates and their arguments.

In order to extract the features for the machine learning algorithm, the modules described above are used, and, in addition, information from Word­Net, VerbNet and the LCS Database is incorporated,

Syntactic Parser POS Tagger Tokenizer Dependency Parser PropBank Parser Extraction Feature Selection

Handcrafted LCS Database

Models Generation

o o o EE E

ANNOTATED INSTANCES Submission Generation INSTANCES

along with other features, based on the manual an­notations for both the training and testing datasets by the task organizers. These other features use the grammatical annotations for the possibly metonymic name, in the case of metonymy resolution, and the query that was used to retrieve that particular in­stance and the disambiguated WordNet sense for the two nominals, in the case of semantic relations clas­sification.

The features implemented for the two tasks are described in Tables 2 and 3. Their types are: syn­tactic, semantic, lexical and other. The syntactic features express the relationships between the tar­get words and words from the rest of the sentence (e.g., the part of speech of the previous word in the sentence, or the dependency relations between two words). The semantic features make use of the in­formation given by the resources used by the system (e.g., the VerbNet class of the verb whose argument the word is, or the lexicographic category of a word in WordNet). The lexical feature is the lemma of the word. The other feature is the query provided by

Task 4.

Using these sets of features, a number of models were generated by different machine learning tech­niques included with the Weka data mining software (Witten and Frank, 2005). The machine learning classifiers comprise decision trees, decision rules, logistic regression, and "lazy" classifiers like k-nearest-neighbor. Because of too many features gen­erated for a relatively small training dataset, feature selection is performed by Weka before creating the models. Metonymy resolution uses in addition the entire set of features, since the dataset has seven times more instances than the other task. For the classification of semantic relations, the initial total and the number of features that remain after the se­lection are printed in Table 4.

For metonymy resolution, there are six sub-tasks to be resolved, which result from all combinations between organization/location and coarse/medium/fine granularity of the label. For the classification of nominal relations, there are 28 sub-tasks, resulting from the processing of the seven se-

Table 4: The number of features before and af­ter Weka selection, for each semantic relation dataset: R1 Cause-Effect, R2 Instrument-Agency, R3 Product-Producer, R4 Origin-Entity, R5 Theme-Tool, R6 Part-Whole, and R7 Content-Container.

mantic relations, in which four experiments are con­ducted, each with an increasing number of train­ing instances. We treated each subtask as a sepa­rate classification problem. Its training set and fea­tures are fed into Weka to create several models. Each classification algorithm mentioned before is employed to obtain one model. For each subtask, the voting module selects the best performing model on 10-fold crossvalidation, which is used to classify the test instances. These annotated instances make up the submission dataset for that particular subtask. To note is that the coarse metonymic level and the semantic relations classification are binary classifi­cations, while the rest of the metonymic subtasks are multi-class classifications, performed in a single stage.

Both the metonymy resolution system and the sys­tem for classification of semantic relations per­formed well in the SemEval 2007 competition. The experiments presented in this paper were done on the training and testing datasets for each subtask. To note is that no other training data was collected or used than the one provided by the organizers.

This paper has presented a semantic architecture that participated in the SemEval 2007 competition to evaluate two tasks, one for metonymy resolution, and the other for the classification of semantic re­lations between nominals. Although the tasks were very different, the architecture produced competitive results. The experimental results are reported in this paper in a detailed manner, and some interesting ob­servations can be drawn from them.