When aiming at correction of errors in general, the basic condition which is to be met is that the seem, it is in fact not met fully satisfactorily in any tagset we know of; for more, cf. Kveton and Oliva (in prep.).

local assignment of PoS-tags is if not correct then at least (morphologically) plausible. In particular, the first errors to be corrected are those where the assignment of PoS-tags violates morphological (and possibly other local, e.g., phonological) laws of the language. Important point is, however, that only the error detection is strictly local - for the correction, a vaster context might be (and as a rule is) needed. From this it follows that the first phase should be the search for "impossible unigrams", i.e. for tags which are assigned in conflict with morpho­logical or lexical information. A simple (but un­realistic) example from English would be the cases that the word table were assigned the tag PLURAL-NOUN or the tag PREPOSITION. As real­istic examples from NEGRA®, it is possible to put forward tagging the (German !) word die as a masculine singular form of an article, tagging ein as a definite article, assigning a word starting with a capital letter and standing on a non-first position of a sentence a verbal tag (typically, such word is a verbal noun) or tagging bis as a preposition requiring dative case. A particular case of locally recognizable errors is constituted by numerals with round thousands, written in digits, with blank between the last three zeroes (e.g., 12 000) which in NEGRA® are systematically tokenized/segmented as two card­inal numbers following each other, e.g., 12 000 is segmented as <position> 12 tag=CARD <end of position> <position> 000 tag=CARD <end of position> while it is obviously to be segmented as a single numeral, i.e.

The errors described in the previous section were cases of incorrect morphological analysis (e.g., die tagged as masculine singular), errors in lexical analysis (the case of preposition bis tag­ged as requiring a dative case) and diverse errors in lemmatization, conversion and segmentation, and if discussed alone, they had better be class­ified as such. In fact, calling these kind of errors "impossible unigrams" (as above) makes little sense apart from serving as a motivation for er­ror detection based on search for "impossible n-grams", i.e. n-tuples (n e N) of tags which, if occuring as tags of adjacent words in a text of a particular language, constitute a violation of (syntactic) rules of this language. The starting point for application of this idea is the search for "impossible bigrams". These as a rule occur in a realistic large-scale PoS-tagged corpus, for the following reasons:

• in a hand tagged corpus, an "impossible bi­gram" results from (and unmistakeably sig­nals) either an ill-formed text in the corpus bo­dy (including wrong conversion) or a human error in tagging

• in a corpus tagged by a statistical tagger, an "impossible bigram" may result also from an ill-formed source text, as above, and further either from incorrect tagging of the training data (i.e. the error was seen as a "correct con­figuration (bigram)" in the training data, and was hence learned by the tagger) or from the process of so-called "smoothing", i.e. of as­signment of non-zero probabilities also to con­figurations (bigrams, in the case discussed) which were not seen in the learning phase.

For learning the process of detecting errors in PoS-tagging, let us make a provisional and in practice unrealistic assumption (which we shall correct immediately) that we have an error-free and representative (wrt. bigrams) corpus of sentences of a certain language at our disposal. By saying error-free and representative, we have in mind that, for the case of bigrams:

• any sentence in the set of sentences constitut­ing the corpus is a grammatical sentence of the language in question (error-freeness wrt. source)

• any bigram can occur in a grammatical sent­ence of the language if and only if it occurs at least once in the corpus (i.e. if any bigram is a possible bigram in the language, it occurs in the corpus (representativity), if any bigram is an "impossible bigram", it does not occur in the corpus (error-freeness wrt. tagging)).

Given such a (hypothetical) corpus, all the bi-grams in the corpus are to be collected to a set CB (correct bigrams), and then the complement of CB to the set of all possible bigrams is to be computed; let this set be called IB (incorrect bigrams). The idea is now that if any element of IB occurs in a PoS-tagged corpus whose correct­ness is to be checked, then the two adjacent cor­pus positions where this happened must contain an error (which then can be corrected). When implementing this approach to error de­tection, it is first of all necessary to realize that learning the "impossible bigrams" is extremely sensible both to error-freeness and to represent-ativity of the learning corpus:

• the presence of an erroneous bigram in the set of CB causes that the respective error cannot be detected in the corpus whose correctness is to be checked (even a single occurrence of a bigram in the learning corpus means correct­ness of the bigram),

• the absence of a correct bigram from the CB set causes this bigram to occur in IB, and hen­ce any of its occurrences in the checked corpus to be marked as a possible error (absence of a bigram in the learning corpus means incorrect­ness of the bigram).

However, the available corpora are neither error-free nor representative. Therefore, in practice these deficiencies have to be compensated for by appropriate means. When applying the approach to NEGRA®, we employed• bootstrapping for achieving correctness

• manual pruning of the CB and IB sets for achieving representativity.

We started by very careful hand-cleaning errors in a very small sub-corpus of about 80 sentences (about 1.200 words). From this small corpus, we generated the CB set, and pruned it manually, using linguistic knowledge (as well as linguistic imagination) about German syntax. Based on the CB set achieved, we generated the correspond­ing IB set and pruned it manually again. The resulting IB set was then used for automatic detection of "suspect spots" in the sample of next 500 sentences from the corpus, and for hand-elimination of errors in this sample where appropriate (obviously, not all IB violations were genuine errors !). Thus we arrived at a cleaned sample of 580 sentences, which we used just in the same way for generating CB set, pruning it, generating IB set and pruning this set, ar­riving at an IB set which we used for detection of errors in the whole body of the corpus (about 20.500 sentences, 350.000 positions). The procedure was then re-applied to the whole corpus. For this purpose, we divided the corpus into four parts of approximately 5.000 sentences each. Then, proceeding in four rounds, first the IB set was generated (without manual checking) out of 15.000 sentences and then the IB set was applied to the rest of the corpus (on the respect­ive 5.000-sentence partition). The corrections based on the results improved the corpus to such an extent that we made the final round, this time dividing the corpus into 20 partitions with ap­proximately 1.000 sentences each and then re-applying the whole process 20 times.

The "impossible bigrams" are a powerful tool for checking the correctness of a corpus, how­ever, a tool which works on a very local scale only, since it is able to detect solely errors which are detectable as deviations from the set of poss­ible pairs of adjacently standing tags. Thus, ob­viously, quite a number of errors remain unde­tected by such a strategy. As an example of such an as yet "undetectable" error in German we might take the configuration where two words tagged as finite verbs are separated from each other by a string consisting of nouns, adjectives, articles and prepositions only. In particular, such a configuration is erroneous since the rules of German orthography require that some kind of clause separator (comma, dash, coordinating conjunction) occur inbetween two finite verbs.

In order to be able to detect also such kind of er­rors, the above "impossible bigrams" have to be extended substantially. Searching for the gene­ralization needed, it is first of all necessary to get a linguistic view on the "impossible bi-grams", in other words, to get a deeper insight into the impossibility for a certain pair of PoS­tags to occur immediately following each other in any linguistically correct and correctly tagged sentence. The point is that this indeed does not happen by chance, that any "impossible bigram" comes into being as a violation of a certain - pre­dominantly syntactic - rule(s) of the language. Viewed in more detail, these violations might be of the following nature:

• Violation of constituency. The occurrence of an "impossible bigram" in the text signals that - if the tagging were correct - there is a basic constituency relation violated (resulting in the occurrence of the "imposible bigram"); as an example of such configuration, we might con­sider the bigram PREPOSITION - FINITE VERB (possible German example string: ...für-PREP reiche-VFIN...). From this it follows that either there is indeed an error in the source text (in our example, probably a missing word, e.g., Der Sprecher der UNO-Hilfsorganisation teilte mit, für Arme reiche diese Hilfe nicht. ) or there was a tagging error detected (in the example, e.g., an error as in the sentence ... für reiche Leute ist solche Hilfe nicht nötig... ). The source of the error is in both cases violation of the linguistic rule postulating that, in German, a preposition must always be followed by a corresponding noun (NP) or at least by an adjectival remnant of this NP.

• Violation of feature cooccurrence rules (such as agreement, subcategorization etc.). The point here is that there exist configurations such that if two wordforms (words with cert­ain morphological features) occur next to each

The above approach does not guarantee, how­ever, that all "impossible n-grams" are consider­ed. In particular, any "impossible trigram" [First,Second,Third] cannot be detected as such (i.e. as impossible) if the [First,Second], [Second,Third] and [First,Third] are all possible bigrams (i.e. they all belong to the set CB). Such an "impossible trigram" in German is, e.g., [nominative-noun, mainverb, nominative-noun] - this trigram is impossible since no German verb apart from sein/werden (which, as said above, are not tagged as main verbs in NEGRA®) can occur in a context where a nominative noun stands both to its right and to its left, however, all the respective bigrams occur quite commonly (e.g., Johann schläft, Jetzt schläft Johann, König Johann schläft). Here, an obvious generalization of the approach from "impossible bigrams" to "imposs­ible trigrams" (and "impossible tetragrams", etc.) is possible, however, we did not perform this in full due to the amount of possible trigrams as well as to the data sparseness problem which, taken together, would make the manual work on checking the results unfeasible in practice. We rather applied only about 20 "impossible tri-grams" and 6 "impossible tetragrams" stemming from "linguistic invention" (such as the trigram discussed above).