Dependency analysis is regarded as one of the standard methods of Japanese syntactic anal­ysis. The Japanese dependency structure is usually represented by the relationship between phrasal units called 'bunsetsu'. A bunsetsu usu­ally contains one or more content words, like a noun, verb or adjective, and zero or more func­tion words, like a postposition (case marker) or verb/noun suffix. The relation between two bunsetsu has a direction from a dependent to its head. Figure 1 shows examples of bunsetsu and dependencies. Each bunsetsu is separated by "I". The first segment "KARE-HA" consists of two words, KARE (He) and HA (subject case marker). The numbers in the "head" line show the head ID of the corresponding bunsetsus. Note that the last segment does not have a head, and it is the head bunsetsu of the sentence. The task of the Japanese dependency analysis is to find the head ID for each bunsetsu.

The analysis proposed in this paper has two conceptual steps. In the first step, dependency likelihoods are calculated for all possible pairs of bunsetsus. In the second step, an optimal de­pendency set for the entire sentence is retrieved. In this paper, we will mainly discuss the second step, a method for finding an optimal depen­dency set. In practice, the method proposed in this paper should be able to be combined with any systems which calculate dependency likeli­hoods.

It is said that Japanese dependencies have the following characteristics :

(1) Dependencies are directed from left to right (2) Dependencies don't cross

(3) Each segment except the rightmost one has only one head

(4) In many cases, the left context is not nec­essary to determine a dependency

The analysis method proposed in this paper as­sumed these characteristics and is designed to utilize them. Based on these assumptions, we can analyze a sentence backwards (from right to left) in an efficient manner. There are two merits to this approach. Assume that we are analyzing the M-th segment of a sentence of length N and analysis has already been done for the (M + 1)-th to N-th segments (M< N).

The first merit is that the head of the depen­dency of the M-th segment is one of the seg-

KARE-HA | FUTATABI (He-subj) (again) | PAI-WO | TSUKURI, | KANOJO-NI | OKUTTA.

Translation: He made a pie again and presented it to her.

ments between M + 1 and N (because of as­sumption 1), which are already analyzed. Be­cause of this, we don't have to keep a huge num­ber of possible analyses, i.e. we can avoid some­thing like active edges in a chart parser, or mak­ing parallel stacks in GLR parsing, as we can make a decision at this time. Also, we can use the beam search mechanism, by keeping only a certain number of analysis candidates at each segment. The width of the beam search can be easily tuned and the memory size of the pro­cess is proportional to the product of the input sentence length and the beam search width.

The other merit is that the possible heads of the dependency can be narrowed down be­cause of the assumption of non-crossing depen­dencies (assumption 2). For example, if the K-th segment depends on the L-th segment (M< K < L), then the M-th segment can't depend on any segments between K and L. According to our experiment, this reduced the number of heads to consider to less than 50%.

The technique of backward analysis of Japanese sentences has been used in rule-based methods, for example (Fujita, 1988). How­ever, there are several difficulties with rule-based methods. First the rules are created by humans, so it is difficult to have wide cover­age and keep consistency of the rules. Also, it is difficult to incorporate a scoring scheme in rule-based methods. Many such methods used heuristics to make deterministic decisions (and backtracking if it fails in a searching) rather than using a scoring scheme. However, the com­bination of the backward analysis and the sta­tistical method has very strong advantages, one of which is the beam search.

We combined the backward beam search strat­egy with a statistical dependency analysis. The detail of our statistic framework is described in (Uchimoto et al., 1999). There have been a lot of proposals for statistical analysis, in many languages, in particular in English and Japanese (Magerman, 1995) (Sekine and Grish-man, 1995) (Collins, 1997) (Ratnaparkhi, 1997) (K.Shirai et.al, 1998) (Fujio and Matsumoto, 1998) (Haruno et.al, 1997) (Ehara, 1998). One of the most advanced systems in English is pro­posed by Ratnaparkhi. It uses the Maximum Entropy (ME) model and both of the accuracy and the speed of the system are among the best reported to date. Our system uses the ME model, too. In the ME model, we define a set of features which are thought to be useful in dependency analysis, and it learns the weights of the features from training data. Our features include part-of-speech, inflections, lexical items, the existence of a comma or bracket between the segments, and the distance between the seg­ments. Also, combinations of those features are used as additional features. The system cal­culates the probabilities of dependencies based on the model, which is trained using a training corpus. The probability of an entire sentence is derived from the product of the probabilities of all the dependencies in the sentence. We choose the analysis with the highest probability to be the analysis of the sentence. Although the ac­curacy of the analyzer is not the main issue of the paper, as any types of models which use de­pendency probabilities can be implemented by our method, the performance reported in (Uchi-moto et al., 1999) is one of the best results re­ported by statistically based systems.

In this section, the analysis algorithm will be de­scribed. First the algorithm will be illustrated using an example, then the algorithm will be formally described. The main characteristics of the algorithm are the backward analysis and the beam search.

The sentence "KARE-HA FUTATABI PAI-WO TSUKURI, KANOJO-NI OKUTTA.(Hemadeapie again and presented it to her)" is used as an in­put. We assume the POS tagging and segmen­tation analysis have been done correctly before starting the process. The border of each seg­ment is shown by " | ". In the figures, the head of the dependency for each segment is represented by the segment number shown at the top of each segment.

Algorithm

The last segment has no dependency, so we don't have to analyze it. The second seg­ment from the end always depends on the last segment. So the result up to the sec­ond segment from the end looks like the following.

For each of the two candidates created at the previous stage, the dependencies of the fourth segment from the end ("PAI-WO") will be analyzed. For Cand1,thesegment can't have a dependency to the fifth seg­ment ("KANOJO-NI"), because of the non-crossing assumption. So the probabili­ties of the dependencies only to the fourth (Cand1-1) and the sixth (Cand1-2)seg-ments are calculated. In the example, these probabilities are assumed to be 0.6 and 0.4. A similar analysis is conducted for Cand2 (here probabilities are assumed to be 0.5, 0.1 and 0.4) and three candidates are cre­ated (Cand2-1, Cand2-2 and Cand2-3).

This segment ("TSUKURI,") has two depen­dency candidates. One is the 5th segment ("KANOJO-NI") and the other is the 6th seg­ment ("OKUTTA").Now,weusetheproba-bilities calculated using the ME model in order to assign probabilities to the two can­didates (Cand1 and Cand2 in the following figure). Let's assume the probabilities 0.1 and 0.9 respectively as an example. At the tail of each analysis, the total probability (the product of the probabilities of all de­pendencies) is shown. The candidates are sorted by the total probability.

As the analysis proceeds, a large number (almost L!) of candidates will be created. However, by limiting the number of candi­dates at each stage, the total number of candidates can be reduced. This is the beam search, one of the characteristics of the algorithm. By observing the analyses in the example, we can easily imagine that this beam search may not cause a serious problem in performance, because the candi­dates with low probabilities may be incor­rect anyway. For instance, when we set the beam search width = 3, then Cand2-2 and Cand2-3 in the figure will be discarded at this stage, and hence won't be used in the following analyses. The relationship of the beam search width and the accuracy ob­served in our experiments will be reported in the next section.

The analyses are conducted in the same way up to the first segment. For example, the result of the analysis for the entire sen­tence will be shown below. (Appropriate probabilities are used.)

Now, the formal algorithm is described induc­tively in Figure 3. The order of the analysis is quadratic in the length of the sentence.

In this section, experiments and evaluations will be reported. We use the Kyoto University Cor­pus (version 2) (Kurohashi et.al, 1997), a hand created Japanese corpus with POS-tags, bun-setsu segments and dependency information. The sentences in the articles from January 1, 1994 to January 8, 1994 (7,960 sentences) are used for the training of the ME model, and the sentences in the articles of January 9, 1994 (1,246 sentences) are used for the evaluation. The sentences in the articles of January 10, 1994 are kept for future evaluations.

In this paper, we proposed a statistical Japanese dependency analysis method which processes a sentence backwards. As dependencies normally go from left to right in Japanese, it is effective to analyze sentences backwards (from right to left). In this paper, we proposed a Japanese de­pendency analysis which combines a backward analysis and a statistical method. It can nat­urally incorporate a beam search strategy, an effective way of limiting the search space in the backward analysis. We observed that the best performances were achieved when the width is very small. Actually, 95% of the analyses ob­tained with beam width=1 were the same as the best analyses with beam width=20. The analysis time was proportional to the square of the sentence length (number of segments), as was predicted from the algorithm. The average analysis time was 0.03 second (average sentence length was 10.0 bunsetsus) and it took 0.29 sec­ond to analyze the longest sentence, which has 41 segments. This method can be applied to various languages which have the same or simi­lar characteristics of dependencies, for example Koran, Turkish etc.