We propose a new type of Chinese case structure in this paper, which is represented as the combi­nation of a case pattern and a case element (see Figure 2). Case element remembers the bi-lexical dependency relation between all types of head-modifier pairs, which is also recognized in previ­ous work (Wu, 2003; Han et al., 2004; Kawahara and Kurohashi, 2006(a); Abekawa and Okumura, 2006). Case pattern keeps the pos-tag sequence of all the modifiers attached to a head to remem­ber the parsing history of a head node.

(b) a case structure compiled from (a)

We use 9,684 sentences from Penn Chinese Treebank 5.1 as the tagged corpus, and 7,338,028 sentences written in simplified Chinese from Chinese Gigaword (Graff et al., 2005) as the raw corpus for Chinese case structure construction.

Before constructing case structures from the raw corpus, we need to get the syntactic analysis of it. First, we do word segmentation and pos-tagging for the sentences in Chinese Gigaword by a Chinese morphological analyzer (Nakagawa and Uchimoto, 2007). Then a Chinese deterministic syntactic analyzer (Yu et al., 2007) is used to parse the whole corpus.

To guarantee the accuracy of constructed case structures, we only use the sentences with less than k words from Chinese Gigaword. It is based on the assumption that parsing short sentences is more accurate than parsing long sentences. The performance of the deterministic parser used for analyzing Chinese Gigaword (see Figure 3) shows smaller k ensures better parsing quality but suffers from lower sentence coverage. Referring to Figure 2, we set k as 30.

A case pattern consists of a sequence of pos-tags indicating the order of all the modifiers attached to a head (see Figure 1), which can be repre­sented as following.

Here, [posm,posm-1,...,pos1]l means the pos-tag sequence of the modifiers attached to a head from the left side, and [pos1,..., posn-1, posn ]r means the pos-tag sequence of the modifiers attached to a head from the right side.

We use the 33 pos-tags defined in Penn Chi­nese Treebank (Xue et al., 2002) to describe a case pattern, and make following modifications:

• group common noun, proper noun and pronoun together and mark them as 'noun';

• group predicative adjective and all the other verbs together and mark them as 'verb';

• only regard comma, pause, colon and semi-colon as punctuations and mark them as 'punc\ and neglect other punctuations.

As introduced in Section 2.1, a case element keeps the lexical preference between a head and its modifier. Therefore, the task of case element construction is to extract head-modifier pairs from both the tagged corpus and the raw corpus.

Although only the sentences with less than k (k=30) words from Chinese Gigaword are used as raw corpus to guarantee the accuracy, there still exist some dependency relations with low accuracy in these short sentences because of the non-perfect parsing quality. Therefore, we apply a head-modifier (HM) classifier to the parsed sentences from Chinese Gigaword to further ex­tract head-modifier pairs with high quality. This HM classifier is based on SVM classification. Table 1 lists the features used in this classifier.

The HM classifier is trained on 3500 sentences from Penn Chinese Treebank with gold-standard word segmentation and pos-tag. All the sentences are parsed by the same Chinese deterministic parser used for Chinese Gigaword analysis. The correct dependency relations created by the parser are looked as positive examples and the left dependency relations are used as negative examples. TinySVM is selected as the SVM toolkit. A polynomial kernel is used and degree is set as 2. Tested on 346 sentences, which are from Penn Chinese Treebank and parsed by the same deterministic parser with gold standard word segmentation and pos-tag, this HM classi­fier achieved 96.77% on precision and 46.35% on recall.

We develop a lexicalized Chinese dependency parser to use constructed case structures. This parser gives a probability P(T\S) to each possible dependency tree T of an input sentence S=w1,w2,...,w„ (w, is a node representing a word with its pos-tag), and outputs the dependency tree T that maximizes P(T\S) (see equation 1). CKY algorithm is used to decode the dependency tree from bottom to up.

To use case structures, P(T\S) is divided into two parts (see equation 2): the probability of a sentence S generating a root node wROoT, and the product of the probabilities of a node w, generat­ing a case structure CS,.

As introduced in Section 2, a case structure CSi is composed of a case pattern cpi and a case element cmi. Thus

P(CSi I wi) = P(cpi, cmi I wi)

A case element cmi consists of a set of de­pendencies {Dj}, in which each Dj is a tuple <wj, diSj, commaj>. Here Wj means a modifier node, diSj means the distance between Wj and its head, and commaj means the number of commas be­tween Wj and its head. Assuming any Dj and Dk are independent of each other when they belong to the same case element, P(cmiIWi,cpi) can be written as

Finally, P(wj,disj,commaj \ wi,cpi) is divided as P (wj, disj, commaj \ wt, cpt ) : P(wj \ wt, cpt) X P(disj, commaj \ wi, wj, cpi)

Maximum likelihood estimation is used to es­timate P(wrootIS) on training data set with the smoothing method used in (Collins, 1996). The estimation of P(cpiIWi), P(WjIWi,cpi), and P(disj, commaj IWi,Wj,cpi) will be introduced in the fol­lowing subsections.

Three steps are used to estimate P(cpiIWi) by maximum likelihood estimation using the con­structed case patterns:

• Estimate P(cpi|wi) only by the case pat­terns from the raw corpus and represent it as

Ptagged (cpi W,) ;

• Estimate P(cpi|wi) by equation 6, in which ^pattern is calculated by equation 7 to set proper ratio for the probabilities estimated by the case patterns from different corpora.

In equation 7, Stagged and Sraw mean the occur­rence of a lexicalized node wi=<lexi, posi> gener­ating cpi in the tagged or raw corpus, rjtagged and nraw mean the occurrence of a back-off node wi=<posi> generating cpi in the tagged or raw corpus. To overcome the data sparseness prob­lem, we not only apply the smoothing method used in (Collins, 1996) for a lexicalized head to back off it to its part-of-speech, but also assign a very small value to P(cpi|wi) when there is no cpimodifying wi in the constructed case patterns.

3.3 Estimating P(wj\wi,cpi) and P(disj, com­maj \wi,Wj,cpi) by Case Elements

To estimate P(wj\wi,cpi) and P(diSj, commaj |wi,Wj,cpi) by maximum likelihood estimation, we also use three steps:

• Estimate the two probabilities only by the case elements from the tagged corpus and repre­sent them as Ptagged (Wj | Wt, cpt) and

P,agg„d (diSj, commaj | wt, Wj, cpt) ;

• Estimate the two probabilities only by the case elements from the raw corpus, and represent

them as (Wj | W,, cp,) and

Praw (diSj, commaj | Wi, Wj, cpi) ;

• Estimate P(wjjwi,cpi) and P(diSj, commaj |wi,Wj,cpi) by equation 8 and equation 9.

= klement * P,agged (Wj | Wi , cpi ) +

= Keemen, * lagged (diS j , commaj | Wi, Wj , cpi) +

The smoothing method used in (Collins, 1996) is applied during estimation.

In order to set proper ratio for the probabilities estimated by the case elements from different corpora, we use a parameter Äelement in equation 8 and 9. The appropriate setting (Äelement =0.4) is learned by a development data set (see Figure 4).

Three parsers were evaluated in this experiment:

• 'baseline': a parser not using case struc­tures, where P(T|S) is calculated by equation 11 and P(Wj|wi) and P(diSj, commaj |wi,wj) are esti­mated by training data set only.

= P(wroot | S) * n P(Wj, disj, comma j | wi)

• 'w/ case elem': a parser only using case element, which also calculates P(T|S) by equa­tion 11 but estimates P(wj|wi) and P(disj, commaj \wi,wj) by constructed case elements.

• ' proposed': the parser introduced in Sec­tion 3, which uses both case elements and case patterns.

The evaluation results on testing data set (see Table 2) shows the proposed parser achieved 87.26% on UAS, which was 2.13% higher than that of the baseline parser. This improvement is regarded as statistically significant (McNemar's test: p<0.0005). Besides, Table 2 shows only us­ing case elements increased parsing accuracy by 1.30%. It means both case elements and case pat­terns gave help to parsing accuracy, and case elements contributed more in the proposed ap­proach.

Figure 5 and Figure 6 show the dependency trees of two example sentences created by both the baseline parser and the proposed parser. In Figure 5, the baseline parser incorrectly assigned ^iT/NN (signing) as the head of £f£/NN (co­operation). However, after using the case ele­ment Igg/NN (project) -¥ 1=H+VNN, the cor­rect head of -pH+VNN was found by the pro­posed parser. Figure 6 shows the baseline parser recognized JFJJ/VV (opening) as the head of |g /P (as) incorrectly. But in the proposed parser, the probability of JFJJ/VV generating the case pattern '[prep, punc, prep]l' was much lower than that of JFJJ/VV generating the case pattern '[prep]/. Therefore, the proposed parser rejected the incorrect dependency that |§/P modified H£/VV and got the correct head of |§/P as /VV (show) successfully.

Gold standard word segmentation and pos-tag are applied in previous experiments. However, parsing accuracy will be affected by the incorrect word segmentation and pos-tag in the real appli­cations. Currently, the best performance of Chi­nese word segmentation has achieved 99.20% on F-score, but the best accuracy of Chinese pos-tagging was 96.89% (Jin and Chen, 2008). Therefore, we think pos-tagging is more crucial for applying parser in real task compared with word segmentation. Considering about this, we evaluated the parsing models introduced in Sec­tion 4 with real pos-tag in this experiment.

Parsing model baseline__proposed

An HMM-based pos-tagger is used to get pos­tag for testing sentences with gold word segmen­tation. The pos-tagger was trained on the same training data set described in Section 4.1 and achieved 93.70% F-score on testing data set. Re­sults (see Table 7) show that even if with real pos-tags, the proposed parser still outperformed the baseline parser significantly. However, the results in Table 7 indicate that incorrect pos-tag affected the parsing accuracy of the proposed parser greatly. Some researchers integrated postagging into parsing and kept n-best pos-tags to reduce the effect of pos-tagging errors on parsing accuracy (Cao et al., 2007). We will also con­sider about this in our future work.