This paper presents a coreference resolution system for Ital­ian based on BART (Versley et al., 2008). BART is a mod­ular toolkit for coreference resolution that supports state-of-the-art statistical approaches to the task and enables ef­ficient feature engineering. BART has originally been cre­ated and tested for English, but its flexible modular archi­tecture ensures its portability to other languages and do­mains.

Even though the basic linguistic notions used in coreference resolution - noun phrases, pronouns, definiteness markers - can be found in a relatively wide range of Germanic and Romance languages, relatively few coreference sys­tems aim at covering multiple languages using one coref-erence component. While there are linguistic differences to English within these languages - Romance languages such as Italian and Spanish have zero subjects and empty pro­nouns - the greater obstacle seems to be that the tagsets or syntactic structures commonly used for processing these languages differ considerably, but also that processing tools which are easily available for English are not available or difficult to obtain for other languages. In comparison to lan­guages such as Japanese, however, where definiteness is not marked and zero pronouns can also occur in object position, which makes detailed syntactic-semantic information nec-essary, the variance within Germanic and Romance lan­guages is small enough that it is conceivable that one sys­tem could perform coreference resolution on all of these languages.

Several researchers did attempt to port their approach to dif­ferent languages: (Mitkov et al., 1998), who present adap­tations of the MARS approach to pronoun resolution to Polish and Arabic, (Harabagiu and Maiorano, 2000), who perform resolution on English-Romanian parallel texts, and (Luo and Zitouni, 2005), who perform coreference resolu­tion on English, Chinese and Arabic data and compare the usefulness of syntactic features for coreference resolution in these languages.

To our knowledge, we present a first full-scale coreference resolution system for Italian. Note also that the system op­erates on a raw text and not on a set of predefined "gold" mentions - i.e. it can be used as a module for a real-world application.

We have evaluated our system on the ICab dataset (Magnini et al., 2007). The data comprise articles for four days of the "Adige" newspaper. We have used one day of "Adige" for testing and three days - for training. On this split, our system achieves a performance level of 56.1% (MUC F-score). On a similar task for English, state-of-the-art tools achieve a slightly better performance level (low to mid six­ties for system mentions on MUC or ACE corpora). How­ever, an English system retrained on Italian dataset yields much lower performance figures. We believe that our im­provement over this baseline is due, in part, to the special­ized mention tagger (Biggio et al., 2009) and, in part, to a number of language-specific adjustments to BART dis­cussed below.

The BART coreference resolution toolkit has four main components: preprocessing pipeline, feature extraction module, decoder and encoder. In addition, an independent LanguagePlugin module handles all the language specific information and is accessible from any component. The architecture is shown on Figure 1. Each module can be accessed independently and thus adjusted to leverage the system's performance on a particular language or domain. The preprocessing pipeline converts an input document into a sequence of mentions with assigned properties (number, gender etc). The feature extraction module describes pairs of mentions (Mj, Mj}, i < j as a set of features. Table 1 shows the features we used. All the feature values are computed automatically, without any manual intervention.

Our work on adapting BART to Italian has followed two directions: we have developed an Italian language plugin and a new preprocessing pipeline.

Aliasing. Our work on the language plugin has mostly included investigating Italian-specific aliasing techniques. A list of company/person designators (e.g., "S.p.a" or "D.ssa") has been manually crafted. We have extracted from the training data several patterns of name variants for the locations (e.g. "Provincia di Verona" and "Verona" may refer to the same place). Finally, we have relaxed ab­breviation constraints, allowing for lower-case characters in the abbreviations - a pattern that is much more common for Italian than for English. Table 2 shows the per­formance level for a coreference resolution system based on the aliasing feature alone. The first row represents a language-agnostic approach to aliasing, the second row -the aliasing approach we have created specifically for Ital­ian. It suggests that, although a universal aliasing algorithm is able to resolve some coreference links between named entities, creating a language-specific module boosts the sys­tem's performance substantially. It should be noted that most coreference resolution systems rely on a very generic approach to aliasing, ignoring any language-specific struc­tures of proper names. We believe that a coreference reso­lution system could benefit a lot from a more sophisticated aliasing algorithm (cf., for example, (Patman and Thomp­son, 2003) for a related study from the text mining commu­nity).

Preprocessing. We have run several evaluation exper­iments with the different designs of the preprocessing pipeline to optimize the system's performance on the Icab dataset. For the testing data, the preprocessing is straight­forward: we input all the chunks detected by a mention tagger (Biggio et al., 2009) and assign relevant properties from the output of the corresponding component of a shal­low NLP toolkit for Italian, TextPro (Pianta et al., 2007). The properties include part-of-speech, morphological fea­tures such as number and gender, as well as semantic type. For the training data, however, this strategy leads to only a moderate performance level for two main reasons. First, manually annotated ("gold") mentions tend to be much longer than those extracted by the tagger ("system mentions"). This means that our matching and aliasing models, learned directly from the gold training data, may

Table 3: System performance (MUC and CEAF-</>4 re­call, precision and F scores) with different preprocessing pipelines

Table 4: Performance (MUC and CEAF-</>4 recall, preci­sion and F scores) on gold mentions: language agnostic vs. Italian-specific system not be applicable to automatically extracted testing men­tions. To rectify this problem, we have adjusted gold men­tion boundaries to cover only the heads, not the extents. Second, the training data contain a number of embedding mentions - chunks that span over another mention (e.g. "la popolazione del sobborgo" is a mention of the second level of embedding, as it spans over another, first-level mention, "sobborgo"). Our mention tagger can only extract mentions of the first and second level of embedding. We have, there­fore, discarded all the gold mentions with the higher level of embedding to avoid unnecessary noise. We have also investigated an alternative parsing pipeline: within this strategy, the chunks, suggested by the men­tion tagger, are mapped into NP-like nodes in automati­cally constructed parse trees. The parser consisted on a dependency parser (Nivre et al., 2007) trained on a con­verted version of the Torino University Treebank (Bosco and Lombardo, 2006), a freely available treebank for Ital­ian, and a dependency-to-constituency converter. Because of the vastly larger size of iCab compared to the Torino tree-bank (ICab contains about 350.000 tokens of text whereas the treebank only contains about 60.000 tokens), the sim­pler and faster chunking-based pipeline works much better than full parsing (cf. Table 3). This is in stark contrast to English, where state-of-the-art parsing gives better results than even the best available chunkers. Note that morphological preprocessing for Italian, on the contrary, is much easier and more accurate, than for En­glish: thus, we can reliably obtain mentions properties (e.g., gender) from a shallow morphological analyzer (TextPro).

Table 5: Performance (MUC and CEAF-</>4 recall, preci­sion and F scores) on automatically extracted mentions

Our evaluation experiments follow two objectives. First, we want to find out, to what extent a generic language-agnostic system can be used for a new language. Second, we try to estimate the impact of our language-specific ad­justments.

For the language-agnostic setting, we have taken the En­glish version of BART, substituted all the external modules (tokenizer, POS-tagger, parser) with the Italian ones and re-trained the system on the Evalita dataset. Unfortunately the results are very moderate mainly due to the annota­tion guidelines: following the ACE standards, only a subset of mentions has been annotated for coreference, making a data-specific mention tagger a vital part of the system. We have therefore started by comparing our language-agnostic and Italian systems on the gold mentions (Table 4). In our last experiment we rely on an Italian mention tag­ger (Biggio et al., 2009) to detect mention boundaries. As this is a crucial part of a coreference-resolution system, we cannot replicate this experiment for the language-agnostic system. Table 5 shows the system performance with the mention tagger (i.e. when operating on a raw text, with no manual intervention).

As our evaluation experiments show, a language-agnostic system (designed primarily for English) can achieve a per­formance level in high forties (MUC F-score) when re­trained and tested on a new language, at least on gold mention boundaries. Though this number might appear low, note that it is a baseline requiring no extra engineer­ing. Compared to this level, we can improve our F-score by around 10% introducing a small number of language-specific changes. This shows that, with a modular corefer-ence resolution platform, such as BART, one can straight­forwardly develop a family of robust and reliable systems for various languages. We hope that our experiments will encourage researchers working on coreference in other lan­guages to create their own full-scale coreference resolution systems - as we have mentioned above, at the moment such modules exist only for very few languages other than En­glish.

To summarize, we have extended BART (Versley et al., 2008) to create a full-scale coreference resolution system for Italian. Its modular design has allowed us to port a large part of the functionality from English to Italian with no changes - we have only had to run a series of evalu­ation runs on the development set to pick the best decod­ing/encoding scheme and the most suitable machine learning algorithm from a range of solutions provided in the BART distribution. We have therefore focused ourattention on improving the system's performance by taking care of language-specific properties. Our experiments have shown that a coreference resolution system based on shallow pre­processing works better for a morphologically rich lan­guage, such as Italian, compared to parsing-oriented strate­gies more common for English.