Corry is a system for coreference resolution in En­glish. It supports both local (Soon et al. (2001)-style) and global (ILP, Denis and Baldridge (2007>style) models of coreference. The backbone of the system is a family of SVM classifiers for pairs of mentions: each mention type receives its own classifier. A sep­arate anaphoricity classifier is learned for the ILP setting. Corry relies on a rich linguistically moti­vated feature set, which has, however, been manu­ally reduced to 64 features for efficiency reasons.

Corry has only participated in the "open" setting, as it has already a number of preprocessing mod­ules integrated into the system: the Stanford NLP toolkit for parsing (Klein and Manning, 2003) and NE-tagging (Finkel et al., 2005), Wordnet for se­mantic classes and the U.S. census data for assigning gender values to person names.

Three runs have been submitted for the Se-mEval task 1 on Coreference Resolution, optimizing Corry's performance for BLANC, MUC and CEAF. The runs differ with respect to the model (local for BLANC, global for MUC and CEAF) and the defi­nition of mention types.

In our previous study (Uryupina, 2008) we have shown that up to 35% recall and 20% precision er­rors in coreference resolution for MUC corpora are due to inaccurate mention detection. We have there­fore invested substantial efforts into our mention de­tection module.

Most state-of-the-art coreference resolution sys­tems operate either on gold markables or on the output of an ACE-style mention detection module. We are not aware of extensive studies on mention extraction algorithms for such datasets as SemEval (OntoNotes) where mentions are complex NPs not constrained with respect to their semantic types.

We rely on the Stanford NLP toolkit for extract­ing named entities (Finkel et al., 2005) and parse trees for each sentence (Klein and Manning, 2003). We then merge the output of the NE-tagger and the parser to create a list of mentions in the following way:

1. Named entities are considered mentions if they correspond to a sequence of parsing con­straints.

2. Pronouns are considered mentions if they are not a part of an NE-mention.

3. NPs are considered "candidate mentions" if they are not a part of an NE-mention. The set of candidate mentions is then filtered to eliminate pairs of NPs with the same head noun (coor­dinate NPs receive unique artificial heads). For possessive NPs we adjust the boundaries and the head to exclude the "'s" token. The remain­ing candidates are aligned with NE-mentions -if an NE and an NP have the same last word, they are considered the same mention of a spe­cial type. Finally, the list of candidates is op­tionally filtered using a small stop-list (for ex­ample, all the "there" NPs in "There is .." are discarded).

We rely on the Stanford NLP toolkit, WordNet and the U.S. census data to assign numerous proper­ties to our mentions: semantic type, number, gender and others.

Corry relies on two SVM classifiers for coreference and anaphoricity. The former determines whether two given mentions Mi and Mj are coreferent or not. The latter determines whether a given mention Mi is anaphoric or discourse new. In Section 4 we show how these classifiers help us build coreference chains. We use the SVM-Light package (Joachims, 1999) for learning our classifiers.

The strength of our system lies in its rich fea­ture set for the coreference classifier. In our previous studies (Uryupina, 2006; 2007) we have tested up to 351 nominal/continuous (1096 boolean/continuous) features showing significant improvements over ba­sic feature sets advocated in the literature. For the SemEval task 1, we have reduced our rich feature set to 64 nominal/continuous features for efficiency rea­sons: on the one hand, our new set is large enough to cover complex linguistic patterns of coreference, on the other hand, it allows us to test different settings and investigate possibilities for global modeling.

Our anaphoricity classifier is used by the ILP model. It relies on 26 boolean/continuous features. More details on the classifier itself can be found in (Uryupina, 2003).

Corry supports both global and local views of coref­erence. Our evaluation experiments (cf. Section 5) show that the choice of a particular model should be motivated by the desired scoring metric.

Our local model of coreference is a reimplementa-tion of the algorithm, proposed by Soon et al. (2001) with an extended feature set. The core of Soon et al.'s (2001) approach is a link-based classifier: it determines whether a given pair of markables are coreferent or not. During testing, a greedy cluster­ing algorithm (link-first) is next used to build coref­erence chains on the output of the classifier.

We have slightly extended this model to allow separate classifiers for different mention types: each candidate anaphor receives a type (e.g. "pronoun") and is processed with a corresponding classifier. We, thus, rely on a family of classifiers, with the same feature set and the same machine learner. The ex­act definition of mention types is a parameter to be determined empirically on the development set.

Our global model is largely motivated by Denis and Baldridge (2007; 2008) and Finkel and Manning (2008). Following these studies, we use Integer Lin­ear Programming to find the most globally optimal solution, given the decisions made by our corefer­ence and anaphoricity classifiers.

In general, an ILP problem is determined by an objective function to be maximized (or minimized) and a set of task-specific constraints. The function is defined by costs link<ij>, and dnewj reflecting potential gains and losses for committing to specific variable assignments. We assume that costs can be positive (for pairs of markables that are likely to be coreferent) or negative (for pairs of markables that are unlikely to be coreferent). The costs are com­puted by an external module (such as a family of lo­cal classifiers described above). The objective func­tion then takes the form:

Binary variables L<ij> indicate that two mark­ables Mi and Mj are coreferent in the output assign­ment. Binary variables Dj indicate that the mark-able Mj is considered anaphoric in the output as­signment. The ILP solver thus assigns values to L<i,j>,Vi,j : i < j and Dj,Vj whilst maximizing the objective in (1). We take the transitive closure of all the proposed L<ij> to build the output partition.

Note that the objective in (1) is not constrained in any way and will thus allow illegal variable as­signments. For example it does not constrain the assignment of L and D variables to be consistent with one another and does not enforce transitivity. The following constraints suggested in the literature (Denis and Baldridge, 2007; Denis and Baldridge, 2008; Finkel and Manning, 2008) ensure that these and other coreference properties are respected:

We refer the reader to the above-mentioned pa­pers for detailed discussions of these constraints and their impact on coreference resolution. As we show in Section 5 below, the usability of a particular con­straint should be determined experimentally based on the desired system behaviour.

Corry has participated in the gold and regular open settings for English. We have collected a number of runs on the development data to optimize the per­formance level for a particular score: BLANC (Re-casens and Hovy, in prep), MUC (Vilain et al., 1995) or CEAF (Luo, 2005). The runs differ with respect to the model (local vs. global with varying sets of con­straints) and the definition of mention types. We de­liberately left the B-CUBE score (Bagga and Bald­win, 1998) completely out of our preliminary ex­periments. The official SemEval scorer was used for these experiments.

Our experiments on the development set show that no configuration is able to produce equally re­liable scores according to all the metrics (note, for example, that on the test set the BLANC difference between Corry-M and Corry-B in the gold setting is almost 10%). We believe that it is a challenging point for future research.

We have selected the best configurations for each score and submitted them as separate runs. The Corry-C system, optimized for CEAF-^4, is a global model with the L, D and A constraints. For the gold setting, mention types are defined as pronouns and non-pronouns. For the regular setting, the system distinguishes between "speech" pronouns, 3rd per­son pronouns, names and nominals.

Corry-M, optimized for MUC, is a global model with the D constraint and separate classifiers for pronouns, names and nominals. Note that, compared to Corry-C, this setting allows for more coreference links - it is well known from the literature (cf., for example, Bagga and Baldwin (1998)) that the MUC metric is biased towards recall.

Finally, Corry-B, optimized for BLANC, is a local model that distinguishes between pronouns, nominals and names. The fact that such a simple model is able to outperform much more complex versions of Corry strengthens the importance of fea­ture engineering.

We have presented Corry - a system for coreference resolution in English. Our plans include extending it to cover multiple languages. However, as the main strength of Corry lies in its rich linguistically moti­vated feature set, this remains an issue.