<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="331"/><title>In Search of Missing Arguments: A Linguistic Approach</title><pubinfo>Proceedings of Recent Advances in Natural Language Processing,pages 331-338, Hissar, Bulgaria, 12-14 September 2011.</pubinfo><author surname="Ruppenhofer" givenname="Josef"><org  name="Saarland University" country="Germany" city="Saarbrucken"/></author><author surname="Gorinski" givenname="Philip"><org  name="Tilde" country="Latvia" city="Riga"/></author><author surname="Sporleder" givenname="Caroline"><org  name="Saarland University" country="Germany" city="Saarbrucken"/></author></firstpageheader><frontmatter><p><b>In Search of Missing Arguments: A Linguistic Approach</b></p><p><b>Josef Ruppenhofer Philip Gorinski and Caroline Sporleder</b></p><p>Dept. of Information Science Computational Linguistics</p><p>and Language Technology Saarland University</p><p>Hildesheim University {philipg,csporled}@coli.uni-saarland.de ruppenho@uni-hildesheim.de</p></frontmatter><abstract>Semantic argument structures are often in­complete in that core arguments are not lo­cally instantiated. However, many of these implicit arguments can be linked to ref­erents in the wider context. In this pa­per we explore a number of linguistically motivated strategies for identifying and re­solving such null instantiations (NIs). We show that a more sophisticated model for identifying definite NIs can lead to notice­able performance gains over the state-of-the-art for NI resolution. </abstract></header><body><section number="1" title="Introduction"><p>Semantic Role Labeling (SRL) is traditionally concerned with identifying the overtly realized ar­guments of a predicate. However, in a natural dis­course only a relatively small proportion of the theoretically possible semantic arguments tend to be locally instantiated in the same clause or sen­tence that contains the target predicate. The other arguments are so-called <i>null instantiations </i>(NIs). Even core arguments of a predicate, i.e., those that express participants which are necessarily present in the situation which the predicate evokes (see Section 2 for a more detailed explanation of core vs. peripheral arguments), are frequently not in­stantiated in the local context. While null instan­tiated arguments are not locally realized, they can often be inferred from the context.</p><p>Consider examples (1) and (2) below (taken from Arthur Conan Doyle's "The Adventure of Wisteria Lodge" and part of the SemEval-10 Task-10 corpus (Ruppenhofer et al., 2010)). We use A and B in the examples to indicate speakers.<footnote anchor="1"/> In a frame-semantic analysis of (1) <i>interesting </i>evokes the    Mental _stimulus_stimulus-focus</p><footnote label="1">We provide this information for clarity, it is not explicitly marked in the corpus.</footnote><p>(Mssf ) frame. This frame has two core semantic arguments, Experiencer and Stimulus, as well as eight peripheral arguments, such as Time, Manner, Degree. Of the two core arguments, neither is actually realized in the same sentence. Only the peripheral argument Degree (Deg) is instantiated and realized by <i>most. </i>To fully un­derstand the sentence, it is necessary to infer the fillers of the Experiencer and Stimulus roles, i.e., the reader needs to make an assumption about what is interesting and to whom. For humans this inference is easy to make as the Experiencer (Exp) and Stimulus (Stim) roles are actually filled by <i>he </i>and <i>a white cock </i>in the previous sentence. (Note that the two utterances in (1) are spoken by the same person.) Similarly, in (2) <i>right </i>evokes the Correctness (Corr) frame, which has four core arguments, only one of which is filled locally, namely Source (Src), which is realized by <i>You </i>(and co-referent with <i>Mr. Holmes). </i>However, another argument, Information (Inf), is filled by the preceding sentence (spoken by a different speaker, namely Holmes), which provides details of the fact about which Holmes was right.</p><doubt alpha="52.8" length="53" tooSmall="False" monospace="0.0">(1) A.    ["A white cock,"]Stimsaid [he]a/). "[Mostly</doubt><p><b>interesting,^!"</b></p><doubt alpha="63.0" length="46" tooSmall="False" monospace="0.0">(2) A.    ["Your powers  seem superior to your</doubt><doubt alpha="66.1" length="59" tooSmall="False" monospace="0.0">opportunities."] &amp;/B.    "[You]Src'rerightCorr,Mr. Holmes."</doubt><p>While humans have no problem inferring unin-stantiated roles that can be filled from the linguis­tic context, this is beyond the capacity of state-of-the-art semantic role labeling systems, which tac­itly ignore all roles that are not instantiated locally. SRL systems thus disregard much argument-level information that is potentially necessary for solv­ing text understanding tasks such as question an­swering or information extraction. That the prob­lem of locally unrealized roles is not restricted to the genre of narrative texts as in the examples above is evidenced by a study by Gerber and Chai (2010) who annotated implicit roles for a set of high frequency nouns in NomBank, which pro­vides predicate argument structure annotation for nominals in the Wall Street Journal portion of the penn Treebank.<page local="2" global="332"/> They found that implicit argu­ments add another 65% to the coverage of overtly instantiated roles in NomBank. Hence, the prob­lem also arises in the news domain, at least with nominal arguments, which tend to realize fewer roles overtly due to a more restrictive syntax.</p><p>Intuitively, it is not surprising that even core ar­guments often remain locally unexpressed since a coherent discourse is not a collection of sentences expressing random states-of-affairs but typically is concerned with a limited set of situations which tend to be interconnected. Hence, it is unlikely that an evocation of a situation in a given sentence im­mediately provides exhaustive information about all possible participants. It is much more likely that this information is spread out over several sentences. Traditional, sentence- or clause-based SRL is therefore clearly a simplification, albeit one that is useful as a first approximation.</p><p>In this paper, we propose a number of strate­gies for identifying implicit arguments and infer­ring their antecedents from the context. Our aim is not so much to provide a perfect system that gives the best possible performance; rather our work is of an exploratory nature. We investigate differ­ent linguistically motivated strategies for dealing with null instantiated arguments and thereby hope to shed more light on the nature of such arguments as well as evaluating potential avenues for future research on automatically inferring referents for such arguments.</p><p>This paper is structured as follows. In the next section we provide an overview of how FrameNet models semantic argument structures and null in­stantiations. Section 3 discusses previous ap­proaches to null instantiation resolution. In Sec­tion 4 we describe the data we used in our exper­iments. The following two sections (5 and 6) de­scribe our model and the experiments. Finally, we conclude in 7.</p></section><section number="2" title="Arguments and Null Instantiations in FrameNet"><p>A predicate argument structure in FrameNet con­sists of a <i>frame </i>evoked by a target predicate.</p><p>Each frame defines a number of potentially pos­sibly arguments or <i>frame elements </i>(FEs). For some FEs, FrameNet explicitly specifies a <i>seman­tic type. </i>For instance, the Experiencer of the Mental_stimulus_stimulus_focus frame (see ex. 1) is defined to be of type 'sentient'. We make use of this information in the experiments. The set of FEs is split into core arguments, pe­ripheral arguments, and extra-thematic arguments. <i>Core arguments </i>are seen as essential components of a frame; they distinguish the frame from other frames and represent participants which are nec­essarily present the situation evoked by the frame, though they may not be overtly realized in a given context. <i>Peripheral arguments </i>are optional and generalize across frames, in that they can be found in all semantically appropriate frames. Typical examples are Time or Manner. Finally, <i>extra-thematic arguments </i>are those that situate the event described by the target predicate against another state-of-affairs. For example, <i>twice </i>can express the extra-thematic argument Iteration. Since only core arguments are essential to a frame, only they are analyzed as null instantiated if missing. peripheral and extra-thematic arguments are, by definition, optional anyway.</p><p>Matters are complicated by the fact that not all core arguments of all frames can be realized simultaneously. Some frames have core argu­ments that are mutually exclusive. For exam­ple, in the Similarity (Sim) frame the enti­ties being compared for similarity can either be expressed by different FEs as in (3) or collectively as in (4). The frame therefore provides three FEs Entity-1 (Ent1), Entity-2 (Ent2), and En­tities (Ents), where the first two FEs are mutu­ally exclusive with the third. These two sets are said to form an <i>exclusion set. </i>At the same time, Entity-1 and Entity_2 are said to be in a <i>Re­quires </i>relation, which means that occurrence of one of these two core FEs requires that the other core FE occur as well.</p><p>(3) [How]Dimension   is   [it]Enti   <b>similar</b>sim   [to my solutionis^?</p><p>(4) [They]Ents are [very]Degree <b>similar</b>sim .</p><p><i>CoreSets </i>define another type of relation that is important in the context of null instantiations. The idea behind CoreSets is that FEs can be inter­dependent, i.e., express similar semantic content, which makes it unlikely that all of them will be overtly realized in a given context. An example are the Source (Src), Path (Pth), and Goal (GOAL) FEs of the Motion (Mtn) frame.<page local="3" global="333"/> They can be expressed together as in (5) (Ruppenhofer et al., 2006) but it is more likely that only one or two of them will be expressed (6). FEs that are interdependent in such way are grouped together in CoreSets. As long as one FE from a CoreSet is expressed, none of the others is annotated as omit­ted. If none is expressed, the contextually most relevant one is annotated as null-instantiated.</p><p>(5) [Fred]rheme <b>went</b>Mtn [from Berkeley]src [across North America and the Atlantic Ocean] Pth [to Paris]Goai.</p><p>(6) [Fred]rheme <b>went</b>Mtn [to Paris]Goai.</p><p>The annotation of null instantiations in SemEval-10 Task-10 follows the practice adopted by FrameNet, which is rooted in the work of Fillmore (1986). Omissions of core arguments of predicates are categorized along two dimensions, the licensor and the interpretation they receive. An NI can either be licensed by a particular lexical item or a particular grammatical construction. For example, in (7) the omission of the AUTHORITIES making the arrest is licensed by the passive construction. Such an omission can apply to any predicate with an appropriate semantics that allows it to combine with the passive construction. On the other hand, the omission in (8) is lexically specific: the verb <i>arrive </i>allows the Goal to be unspecified but the verb <i>reach, </i>also a member of the Arriving frame, does not (9).</p><p>(7) [A drunk burglar]Sspct was <b>arrestedArrest </b>after acci­dentally handing his ID to his victim.</p><p>(8) [We]Thm <b>arrived</b>Arrive [at 8pm]rm.</p><p>(9) *[We]rhm <b>reached</b>Arrive [at 8pm]rm</p><p>The above two examples also illustrate the sec­ond major dimension of variation. Whereas, in (7) the protagonist making the arrest is only existen-tially bound within the discourse (an instance of indefinite null instantiation, INI), the GOAL lo­cation in (8) is an entity that must be accessi­ble to speaker and hearer from the discourse or its context (definite null instantiation, DNI). Fi­nally, note that the licensing construction or lex­ical item fully and reliably determines the inter­pretation. Whereas missing by-phrases have al­ways an indefinite interpretation, whenever <i>arrive </i>omits the GOAL lexically, the GOAL has to be in­terpreted as definite.</p><p>As INIs do not need to be accessible within a context, the task of resolving NIs is restricted to DNIs. The complete task can then be modeled as a pipeline consisting of three sub-tasks: (i) iden­tifying potential NIs by taking into account in­formation about core arguments and relations be­tween them, (ii) automatically distinguishing be­tween DNIs and INIs by identifying NI licensing constructions or lexical items, and (iii) resolving NIs classified as DNIs to a suitable referent.</p></section><section number="3" title="Related Work"><p>The most closely related piece of work is the system building performed in the context of the SemEval-10 Task-10 (Ruppenhofer et al., 2010). The two participating systems which addressed the NI resolution task took very different ap­proaches. Tonelli and Delmonte (2010) developed a knowledge-based system called VENSES++ that builds on an existing text understanding system (Delmonte, 2008). VENSES++ employs deep syntactic parsing and uses hand-crafted lexicons to generate logical forms. It then makes use of a rule-based anaphora resolution procedure before em­ploying two different strategies for identifying and resolving NIs. For verbal predicates, argument pattern templates generated from FrameNet data are used to identify missing predicates and clas­sify lexically licensed NIs as DNI or INI. The only type of constructionally licensed NIs that can be detected by the system are those of agents in pas­sive constructions. NIs are resolved by reasoning about the semantic similarity between an NI and a potential filler using WordNet. For nominal pred­icates, the system employs a common sense rea­soning module that builds upon ConceptNet (Liu and Singh, 2004). The system is conservative and has a relatively high precision, e.g., 64.2% for the DNI v. INI distinction, but a low recall, identifying less than 20% of the NIs correctly.</p><p>The second system (Chen et al., 2010) is sta­tistical and extends an existing semantic role la­beler (Das et al., 2010). The system first classi­fies NIs as DNI or INI and then tries to find fillers for the former. Resolving DNIs is modeled in the same way as labeling overt arguments, however the search space is extended to pronouns, Nps, and nouns outside the sentence.<footnote anchor="2"/> When evaluat­ing a potential filler, the syntactic features which are used in argument labeling of overt arguments are replaced by two semantic features:<page local="4" global="334"/> The system checks first whether a potential filler in the context fills the null-instantiated role overtly in one of the FrameNet sentences, i.e. whether there is a prece­dent for a given filler-role combination among the overt arguments of the frame in FrameNet. If not, the system calculates the distributional similarity between filler and role. The surface distance be­tween a potential filler and an NI is also taken into account. While Chen et al.'s system has a higher recall than VENSES++, its performance is still rel­atively low, e.g., the accuracy for the DNI v. INI classification is 55%. The authors argue that data sparseness is the biggest problem.</p><footnote label="2">This disregards other role fillers such as whole sentences as in example (2) above.</footnote><p>Also very closely related is Gerber and Chai (2010), which presents a study of implicit argu­ments for a group of frequent nominal predicates. Gerber and Chai (2010) model the task as a clas­sical supervised task and implement a number of syntactic, semantic, and discourse features such as the the sentence distance between an NI and its potential filler, their mutual information, and the discourse relation holding between the spans con­taining the target predicate and the potential filler.</p><p>While both Gerber and Chai (2010) and the</p><p>SemEval-10 Task-10 deal with finding fillers for uninstantiated arguments, there are important dif­ferences between the two data sets, which make the results not directly comparable. Gerber and Chai's corpus consists of newswire texts (Wall Street Journal), which is annotated with Nom-Bank/PropBank roles. The data cover 10 nominal predicates from the commerce domain, with—on average—120 annotated instances per predicate. The Task-10 corpus consists of narrative texts an­notated under the FrameNet paradigm. Crucially, this corpus provides annotations for running texts not for individual occurrences of selected target predicates. It thus treats many different general-language predicates of all parts of speech. While the overall size of the corpus in terms of sentences is comparable to Gerber and Chai's corpus, the SemEval corpus contains many more target pred­icates and fewer instances for each.<footnote anchor="3"/> These prop­erties make it much harder to obtain good results on the SemEval corpus, which is supported by the fact that the NI resolution results obtained by the Task-10 participants are significantly below those reported by Gerber and Chai (2010).</p><footnote label="3">E.g., Ruppenhofer et al. (2010) report that there are 1,703 frame instances covering 425 distinct frame types, which gives an average of 3.8 instances per frame.</footnote><p>While the SemEval-10 Task-10 is harder than the problem tackled by Gerber and Chai (2010), we also believe it is more realistic. Given the com­plexity of annotating semantic argument structures in general and null instantiations in particular, it seems infeasible to annotate large amounts of text with the required information. Hence, automated systems will always have to make do with scarce resources. We investigate different strategies of incorporating linguistic background knowledge to overcome this data sparseness problem, e.g., by explicitly modeling the DNI v. INI distinction, which is ignored by Gerber and Chai (2010). We also think that the task is best modeled as a semi-supervised task which combines the training data with FrameNet data not annotated for NIs.</p><p>Another line of research that is related to the goals of our effort is the work on zero pronoun res­olution in pro-drop languages such as Japanese or Spanish. Iida et al. (2007) discuss the relevance of the semantic role labeling and zero-anaphora res­olution tasks to each other and study how methods used in one task can help in the other. Still, their work is different from our task in two respects. First, it has a different coverage. Of the kinds of omissions that we consider to be null instan­tiations, Iida et al. (2007) target only the subset of constructionally licensed omissions. In addition, they seem to treat cases of co-instantiation or argu­ment sharing—for instance subjects shared across conjoined VPs—as involving argument omission, which is not how similar cases would be treated in our FrameNet-style annotations. Second, in their system implementation Iida et al. (2007) use only syntactic patterns but no semantic informa­tion about the semantic class frame) of the predicate missing an argument or about the inter­relations between the predicate missing an argu­ment and the predicate(s) where coreferent men­tions of the missing argument appear. Palomar et al. (2001) similarly use syntactic rather than semantic information in their work on Spanish, which only allows constructionally licensed sub­ject omissions.</p><doubt alpha="66.7" length="6" tooSmall="False" monospace="0.0">4 Data</doubt><p>In our experiments we used the corpus distributed for the SemEval-10 Task-10 on "Linking Events and Their Participants in Discourse" (Ruppen­hofer et al., 2010). The data set consists of two texts from Arthur Conan Doyle, "The Adventure of Wisteria Lodge"(1908) and "The Hound of the Baskervilles" (1901/02).<page local="5" global="335"/> From the first text, the second part entitled "The Tiger of San Pedro" (henceforth "Tiger") was annotated and served as training data in the task; from the second text (henceforth "Hound") chapters 13 and 14 were an­notated and served as test data. The annotation consists of frame-semantic argument structure, co-reference chains, and information about null in­stantiation, i.e., the NI type (DNI vs. INI) and the filler, if available in the text. Table 1 provides ba­sic statistics about the data set.</p><p>In a qualitative analysis, we also considered a randomly chosen subset of 50 frame instances from the training data with at least one uninstanti-ated FE-set (see Section 6).</p></section><section number="5" title="Modeling"><p>We approach our three sub-decisions separately. The first sub-task, determining which, if any, frame elements are missing relies on information from the FrameNet release. Of particular impor­tance is information about the three types of rela­tionships between the core Frame elements: Core-Set, Excludes, and Requires. Given that we start with gold standard annotation of the overtly in­stantiated elements, we reason about the FE re­lations in the frame at issue to determine which FEs are to be considered as missing. For instance, consider the instance of the Similarity frame evoked by <i>different </i>in (10).</p><p>(10)       Falkner can be related to the "New South" litera­ture but [his approach]Ent1 was <b>differentsim.</b></p><p>As discussed in Section 2, there are two FE-relation instances defined for the Similarity frame: a Requires relation between Entity_1 and Entity_2 and an Excludes relation between En­tities and Entity_1 and Entity_2. Given that Entity_1 is instantiated, we conclude due to the Excludes relation that Entities does not have to be treated as NI; given the Requires relation, we conclude that Entity_2 does.</p><p>Our second sub-decision is to decide whether a frame element that we have found to be null-instantiated has an anaphoric (DNI) or an ex­istential (INI) interpretation. Our approach for making this decision is the following. First, we check whether the omission we are looking at is licensed by a specific grammatical construction which specifies the interpretation type of the argu­ment it suppresses. For instance, we would treat the missing by-phrase agent of a passive as omit­ted with existential interpretation. Besides pas­sive, we only consider imperatives at this point, although there are additional but less frequently occurring valence-suppressing constructions.</p><p>In our specific case of (10), there is no rele­vant construction that we can blame the omission on and we thus consider the omission to be lexi­cally licensed. Since that is so, we next look at the FrameNet annotations for the specific frame evok­ing element. Either we only look at the annota­tions of the particular lexical unit that occurs in our text, or we consider statistics aggregated across all lexical units in a frame. In either case, for the frame element under consideration we choose that type of interpretation type that is more com­mon in the annotated data. For <i>different </i>we find that uninstantiated cases of Entity_2 are always labeled DNI and so in processing (10) we would choose DNI as well. Heuristics are needed when there either are no relevant annotations or when the frequencies of DNI and INI are tied.<footnote anchor="4"/> The sim­plest heuristic is to simply choose one interpreta­tion type as a default, which is what we do.</p><p>The final decision we have to make concerns uninstantiated FEs for which we have settled on the anaphoric interpretation type. For these, we have to locate, if possible, a coreferring antecedent mention. Any coreferring mention will do since we evaluate against coreference chains.<footnote anchor="5"/> In theory, we could use customized strategies for antecedent finding depending, for instance, on whether the null instantiation is licensed by a construction or by a lexical item, or depending on the identity of the null-instantiated frame element. However, at the moment we treat the problem of antecedent finding in the same way for all null-instantiated frame elements.</p><p>One approach we pursue for identifying a suitable mention/chain relies on the semantic types that FrameNet specifies for frame elements. Specifically, we look up in FrameNet the semantic type(s) of the FE that is unexpressed. With that indata set   sentences   tokens   frame frame overt frame DNIs INIs instances types elements (resolved) formation in hand, we consider all the coreference chains that are active in some window of context, where being active means that one of the member mentions of the chain occurs in one of the con­text sentences.<page local="6" global="336"/> We try to find chains that share at least one semantic type with the FE in question. This is possible because for each chain, we have percolated the semantic types associated with any of their member mentions to the chain.<footnote anchor="6"/> If mul­tiple chains remain that are compatible with the FE in question, we select between them by some criterion. In particular, we prefer to link the FE to that chain that has the mention closest to the FE in question in terms of intervening leaf nodes.<footnote anchor="7"/>If we find no chain at all within the window that has semantic types compatible with our FE, we guess that the FE has no antecedent.<footnote anchor="8"/> Note also that in our current set-up we have defined the se­mantic type match to be a strict one. For instance, if our FE has the semantic type <i>Entity </i>and an ac­tive chain is of the type <i>Sentient, </i>we will not get a match even though the type <i>Sentient </i>is a descen­dant of <i>Entity </i>in the hierarchy in which semantic types are arranged.</p><footnote label="4">One might additionally choose to employ heuristics when the number of annotated instances is very small, or when the frequencies of DNI and INI are very close, though not tied. We have not used such heuristics here.</footnote><footnote label="5">Note that we have chains of length 1, since we for in­stance need to be able to reify whole sentences as referents that can be the antecedents for unexpressed Message, Con­tent or similar FEs of predicates such as know or confess.</footnote><doubt alpha="10.3" length="58" tooSmall="False" monospace="0.0">train     438       7,941   1,370 3T7 2,526 303 (245) 277"</doubt><doubt alpha="7.4" length="54" tooSmall="False" monospace="0.0">test        525 9,131    1,703 452 3,141 349 (259) 361</doubt><table caption="Table 1: Statistics for the SemEval-10 Task-10 corpus"></table></section><section number="6" title="Experiments"><p>To gain a better understanding of our results for the full NI resolution task, we performed a quali­tative analysis on a subset of 50 frames from the training set, in which one or more Frame elements were uninstantiated. We focus here on the first two sub-decisions that have to be made in the auto­matic analysis of null instantiations: which spe­cific FEs should be treated as null-instantiated and which interpretation type the relevant FEs have.</p><footnote label="6">In the official FrameNet database, not every frame ele­ment is assigned a semantic type. We modified our copy of FrameNet so that every FE does have a semantic type by sim­ply looking up in WordNet the path from the name of a frame element to the synsets that FrameNet uses to define semantic types.</footnote><footnote label="7">Other criteria are easily conceivable. We might, for in­stance, use a tree-based distance measure, or link the FE to the chain that has the most mentions within the window of context.</footnote><footnote label="8">Alternatively, we could have widened the window of context in the hope of hitting upon a suitable chain.</footnote><p>The distribution of frames in this set was as fol­lows: 33 frames occurring only once, 4 instances of Arriving, 3 instances of Self-motion and 2 of Departing. In 3 of the 6 instances of Calendric_unit and in all 3 instances of Self-motion , our NI analysis system made er­rors. These are two challenging frames to handle which happen to be frequent in our data.</p><p>We also see that in our data, we have many nouns as frame evoking elements (FEEs). 28 of50 FEEs are nouns, 15 verbs, and 7 adjectives. This distribution also contributes to an overall lower performance of our system because the error rate is highest for nouns, middling for adjectives, and lowest for verbs.<footnote anchor="9"/> In our first system setting, where we use frame-level NI statistics and where we use INI as the default interpretation type when FrameNet either has no relevant data or shows equal probability for DNI and INI, the error rate on nouns is 53.6%, on adjectives 28.6%, and on verbs 13.3%.</p><p>In the first setting with INI as default, the sys­tem made no error on 31 of the 50 frames (62%). The 50 frame instances analyzed contain 62 FE-Sets that are not instantiated. (Recall that a single predicate may omit more than one argument at the same time.) Of these 62 sets, 38 are classified cor­rectly as INI or DNI (61.3%) and the remaining 24 incorrectly. The predominant error type is the system positing INI where the gold value is DNI (16 of 24). The remaining errors are the other way around.</p><p>Given that for our data set, the baseline of guessing the DNI majority class is 52.2%, our sys­tem configuration has noticeably better precision at 62%. Importantly, we also have 100% recall for uninstantiated FE-sets unlike the systems in the SemEval task.</p><p>In our second NI analysis setting, we again use<page local="7" global="337"/></p><footnote label="9">The same differences among the parts-of-speech can also be seen, for instance, in the performance on labeling of ex­plicit FEs where the treatment of verbal predicators is more successful.</footnote><p>Accuracy</p><p>INI as the default value but we use lexical unit-specific NI-statistics rather than aggregate statis­tics over all lexical units in the frame. Doing so improves the result a bit: we classify 41 of 62 FE-sets (66%) correctly, for a 4.7% improvement over the previous setting. Table 2 provides a summary of the results.</p><p>Finally, we look at the sources of error for our first setting. As noted above, there were 19 frame instances where at least one FE-set was classi­fied incorrectly. The main reasons for these errors were:</p><p>• With 5 frames instances, the error results be­cause the aggregate frame-level statistics are distorted. This is due to two reasons: there are few annotated instances, or a "deviant" lexical unit is overrepresented.</p><p>• In another 5 frame instances, the use of INI as a default is inappropriate. These are cases where either no lexical unit in the frame is an­notated at all, or where the frame was created and annotated before the practice of annotat­ing missing arguments was adopted.</p><p>• In 4 frame instances, a misclassification oc­curs because the instance of the frame in our test data occurs in a special linguistic con­text that overrides the majority interpretation type that can be observed in the FrameNet data. For instance, the context in our data may be generic, while the majority of cases in FrameNet annotations are episodic.</p><p>• 4 frame instances belong to the linguistically difficult frames where the gold standard anal­ysis itself may not be fully worked out. A good example of this is Calendric unit.</p><p>While our manually inspected data set is small, it seems we must conclude from this qualitative analysis that even a reasonable, linguistically mo­tivated use of the available FrameNet data won't yield the correct result for NI-classification in all cases. One difficulty arises from FrameNet's an­notation practice, which does not select instances randomly. Hence, the statistics about indefinite v. definite interpretations for a given FE that can be gleaned from FrameNet are not necessarily accu­rate. At this point, we do not know the exact num­ber of frames where, for instance, a skew in the annotated Lus or the annotated instances of a par­ticular LU would lead to incorrect classifications. But even if FrameNet had annotated a large num­ber of randomly chosen instances for all Lus, our current system would not achieve perfect perfor­mance because it lacks a way of detecting con­structions and contexts (such as generic or habitual sentences) that can override the majority interpre­tation type. Complementing our system with an additional analysis step which attempts to identify different event types thus seems beneficial. The work by Reiter and Frank (2010) and Mathew and Katz (2009) on generic NPs and sentences could be a starting point.</p><p>Since there are only very few resolved NIs in the 50 frame data set we used to evaluate the first two sub-tasks, we evaluated the NI resolution task (i.e., the third sub-task) on the whole SemEval-2010 Task-10 test set. We employed the best per­forming SemEval system, SEMAFOR (Chen et al., 2010), as a baseline. Even though our NI res­olution strategy is still fairly basic, taking only the semantic type of potential fillers into account, our system reduces the resolution errors for the com­plete pipeline by 14% compared to SEMAFOR. This may be due to the fact that our DNI v. INI classification is better. As the DNI v. INI distinc­tion was not evaluated for the shared task, we can­not directly compare our results on this sub-task provide a confusion matrix for argument classi­fication (Table 3 in their paper), which suggests that only 3% of DNIs are correctly identified. The majority of unidentified DNIs are misclassified as</p><doubt alpha="65.9" length="44" tooSmall="False" monospace="0.0">against SEMAFOR. However, Chen et al. (2010)</doubt><p>INIs (52%).</p><p>SEMAFOR is, however, a bit better at identi­fying the correct boundaries for correctly found antecedents (100% NI linking overlap v. 89% for our system). The reason for this may be that we consider more varied antecedents. In partic­ular, we also consider full sentence antecedents. Example (11) illustrates the problem of identify­ing the correct boundaries for full sentence an­tecedents. The gold annotation identifies both (a) and (b) as the antecedent of the Content FE of the Experiencerfocus frame evoked by <i>pleasure </i>in (c), while our system resolved the NI only to (b).<page local="8" global="338"/></p><table caption="Table 2: Distinguishing DNIs and INIs" class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Maj. Baseline</p></td><td class="cell"><p>52.2%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PerFrame</p></td><td class="cell"><p>61.3%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>PerLU</p></td><td class="cell"><p>66.0%</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><p>(11)       a.     "I must congratulate you, Inspector, on han­dling so distinctive and instructive a case.</p><p>b. Your powers, if I may say so without offence, seem superior to your opportunities."</p><p>c. Inspector Baynes's small eyes twinkled with <b>pleasure</b>Expfoc.</p></section><section number="7" title="Conclusion"><p>We presented a novel approach to recognizing and resolving null instantiations. We split the task in three sub-task: identification of NIs, distinguish­ing definite and indefinite NIs, and resolving NIs to a suitable referent in the text. We paid par­ticular attention to the first two sub-tasks. The first task was addressed by making use of back­ground knowledge about interdependencies be­tween frame elements. For the second task, we employed a hybrid system which combined rules for identifying syntactic constructions with statis­tics about DNI v. INI distributions for different lexical units or frames. For the resolution task we made use of FrameNet's semantic type informa­tion for frame elements which we enriched with semantic information from WordNet.</p><p>We showed that our system has a noticeably bet­ter performance on the whole pipeline than the best system participating in the SemEval-10 NI resolution task. This is probably due to the fact that we employ a more sophisticated system for identifying DNIs.</p><p>However, an error analysis revealed that there are also areas where our system could be im­proved. Obtaining reliable statistics for lexically licensed NIs from FrameNet proves difficult be­cause FrameNet data were not randomly selected. It may be possible to overcome this shortcom­ing by trying to glean information about NIs from unannotated data, e.g., by using semantic similar­ity to cluster syntactic arguments. A preprocessing component which identifies different event types (generics, habituals etc.) might also help to iden­tify DNIs in a more reliable fashion. Furthermore, our strategy for finding antecedents is still fairly basic. Adding additional features, e.g., along the lines of Gerber and Chai (2010) will probably lead to better performance.</p></section><section title="Acknowledgments"><p>This research has been funded by the German Research</p><p>Foundation DFG (MMCI Cluster of Excellence and grant PI</p><doubt alpha="52.6" length="19" tooSmall="False" monospace="0.0">154/9-3).References</doubt><p>D. Chen, N. Schneider, D. Das, N. A. Smith. 2010. SEMAFOR: Frame Argument Resolution with Log-Linear Models. In <i>Proc. ofSemEval-2010,</i>264-267.</p><p>D. Das, N. Schneider, D. Chen, N. A. Smith. 2010.</p><p>Probabilistic Frame-semantic Parsing. In <i>Proc. of NAACL-HLT-10, </i>948-956.</p><p>R. Delmonte. 2008. <i>Computational Linguistic Text Processing Lexicon, Grammar, Parsing and Anaphora Resolution. </i>Nova Science, New York.</p><p>C. Fillmore. 1986. Pragmatically Controlled Zero Anaphora. In <i>Proceedings of the Twelfth Annual Meeting of the Berkeley Liguistics Society, </i>95-107.</p><p>M. Gerber, J. Y. Chai. 2010. Beyond NomBank: A Study of Implicit Arguments for Nominal Pred­icates. In <i>Proc. ofACL-2010, </i>1583-1592.</p><p>R. Iida, K. Inui, Y. Matsumoto. 2007. Zero-anaphora Resolution by Learning Rich Syntactic Pattern Fea­tures. <i>ACM Transactions on Asian Language Infor­mation Processing (TALIP), </i>6:1:1-1:22.</p><p>H. Liu, P. Singh. 2004. ConceptNet: A Practical Com-monsense Reasoning Toolkit. <i>BT Technology Jour­nal, </i>22(4):211-226.</p><p>T. Mathew, G. Katz. 2009. Supervised categorization of habitual and episodic sentences. In <i>Sixth Midwest Computational Linguistics Colloquium, </i>Blooming-ton, Indiana.</p><p>M. Palomar, L. Moreno, J. Peral, R. Mufioz, A. Ferrandez, P. Martmez-Barco, M. Saiz-Noeda. 2001. An Algorithm for Anaphora Resolution in Spanish Texts. <i>Computational Linguistics, </i>27:545­567.</p><p>N. Reiter, A. Frank. 2010. Identifying Generic Noun</p><p>Phrases. In <i>Proc. ofACL-10, </i>40-49.</p><p>J. Ruppenhofer, M. Ellsworth, M. R. L. Petruck, C. R. Johnson, J. Scheffczyk.  2006. FrameNet II: Ex­tended Theory and Practice. available at http:// framenet.icsi.berkeley.edu/index. php?option=com_wrapper&amp;Itemid=12 6.</p><p>J. Ruppenhofer, C. Sporleder, R. Morante, C. Baker, M. Palmer. 2010. SemEval-2010 Task 10: Linking Events and Their Participants in Discourse. In <i>Proc.</i></p><p><i>ofSemEval-2010, </i>45-50.</p><p>S. Tonelli, R. Delmonte. 2010. VENSES++: Adapting a Deep Semantic Processing System to the Identifi­cation of Null Instantiations . In <i>Proc. ofSemEval-</i> <i>2010, </i>296-299.</p></section></body></article>