Ill order to overcome these difficulties, following methods .~r~ proposed these years: We propose a new method to resolve am1. F, xample-Ba.sed Translation : tile method biguity in translation and meaning inbased oil trans\[atiou examples (pairs of source terpretation using linguistic statistics extext, aml its translation) \[Nagao 84, Sato 90, tracted from dual corpora of sourcu aud Smuita 90\] target languages in addition to tim logical 2. Statistics-Based Translation : the nmthod usrestrictions described on dictiomtry and ing statistical mr probabilistic information exgrammar rules for ambiguity resolution. tracted from a bilingual corpus \[Brown 90, It provides reasonable criteria for deterNomiyama 9\]\] mining a suitable equivalent translation or meaning by making tile dependency reStill, each (ff them has inherent problems and is lation in the source language be reflected insufficient for ambiguity resolution. For example, in the translated text. The method can either all e×amplc~b~mcd translation method or a be tractable because tile required staffsstatistics-based translation method needs a largetics can be computed semi-automatically scale database of translation exalnpl~, and it is in advance from a source language corpus difficult to collect all adequate amount of a bilinand a target language corpus, while an gual corpus. ordinal corpus-based translation method In this paper, we propose a new method to select needs a large volume of bilingual corpus the suitable equivalent translation using the staof strict pairs of a sentence and its translatistical data extracted independently from source tion. Moreover, it also provides the means and target language texts \[Muraki 91\]. The stato compute the linguistic statistics on the tistical data used here is linguistic statistics repre: pairs of meaning expressions. senting the dependency degree on the pairs of expressions in each text, especially statistics for cooccurrence, i.e., how frequently the expressions cooccur in the Sallle seutence~ the sanle paragraph or tile same chapter of each text. The dependency 1 Introduction relation in the source language is reflected in the It~ecently many kinds of natural lauguage protranslated text through bilingual dictionary by sc~ cessing systems like machine translation systems lecting the equivalent ranslation which ma.ximizes have been developed and put into practical use, but both statistics tot co-occurrence in tile source and ambiguity resolution ill translation and meaning intarg(~t language text. Moreover, the method also terpretation is still the primary issue in such sysprovid~ the means to compute tile linguistic statistems. These systems have conventionally adopted tics on the pairs of meaning expressions. We call a rule-ba.~ed isambiguation method, using linguistlds method for equivalent ranslation and meaning tic restrictions described logically in dictionary and selection DMAX Criteria (Double Maximize Critegrammar to select the suitable equivalent ranslaria based on Dual Corpora). tion and meaning. Generally speaking, it is imposFirst, we make comments on the characteristics sible to provide all the restrictions systematically and the linfits of the conventional methods of amin advance. Furthermore, such machine translabiguity resolution in translation and meaning intertion systems have suffered from inability to select pretation in the second section. Next, we describe the most suitable equivalent ranslation if the inthe details of DMAX Criteria for equivalent ransput expression meets two or more restrictions, and lation selection in the third section. And last, we have difficulty in accepting any input expression explain the means to compute the linguistic statistics on the pairs of meaning expressions. that meets no restrictions. ACRES DE COLING-92, NANTES, 23-28 ̂of.rr 1992 5 2 5 PROC, oJ, COLING-92, NA~CrEs. AUG. 23-28, 1992
In conventional methods, linguistic restrictions described in the dictionary and grammar are used to select the suitable equivMcnt translation or meaning. In general, these restrictions are de~ scribed logically on characteristics of another expression which modifies or is modified by the expression to be processed. For example, to translate predicates (verbs and predicative adjectives), semantic restrictions are deacribed on essential case arguments in forms of semantic markers to indicate features of words or terms in the thesaurus to show a hierarchy composed of word concepts.
Though these conventional methods have been very useful to realize natural language processing systems, ttmy have the following problems: 1. It is impossible to decide the most suitable
equivalent ranslation if the input expression
meets two or more restrictions. 2. Analysis fails when the input expression can
meet no restrictions. 3. Actually the practical systems depends on
such heuristics as pre-declded application or-
der of restrictions or some default equivalent
translations or meanings. 4. The description of the restrictions is based on
direct structural dependencies, therefore it is
quite difficult to describe the restrictions based
on sister-dependency orbetween expressions
belong to different sentences or paragraphs. 5. Restrictions on any dependencies cannot be
thoroughly described in advance.
For example, a Japanese word "booru" has two meanings, one is 'a bail(a round object used in a game or sport)' and the other is 'a bowl(a deep round container open at the tap especially used in cooking)'. When this word occurs in the following sentence, it must mean ~a bowP. JAP: Booru-nl mizu-o
bowl dative water obj,
or marker marker
ball ENG; To pour water in%o a bowl In this case, to select the meaning by the logical restrictions on dependencies, it is necessary to have described even the appearance or usage of the indirect object of the verb "irern". To describe such detail restrictions on ,all expressions may be possible, but it is quite difficult because the trouble of description and the cost of calculation. AcrEs DE COLING-92, NANTES, 23-28 AOOT 1992 5 2 6 ireru pour, put in or fill
Besides the conventional translation method above, a machine translation system based on translation examples (pairs of source texts and their translations) is also proposed \[Nagao 84, Sato 90, Sumita 90\]. This type of system, called Example-Based Machine Translation, has stored a large amount of bilingual translation examples ms a database, and translates input expressions by retrieving an example most similar to tim input from the database. There is no failure of output in this method because it selects the most similar example not the identical one.
However this example-based translation system needs a large-scale database of translation examples, and iL is difficult to collect an adcxluate amount of bilingual corpora. Even if it is possible, there is no means to divide the sentences of such corpora into fragments and link them automatically, and it costs us too much time and money to divide and link manually. Besides, this method can neither achieve precise meaning interpretation because it selects equivalent ranslation directly from the input expression and leaves meaning interpretation out of consideration.
To overcome this problem, we have also proposed a new mechanism based on sentential examples in dictionary, which utilize the merits of both the translation by logical restrictions and the examplebased methud, by selecting the equivalent ranslation which ha.s tlle most similar example to the input expression IDol 92\]. This mechanism can guarantee no failure in selecting an equivalent ranslation, but the description of relations are still based only on direct structural dependencies.
Several new methods especially of machine translation have been proposed lately, which select a suitable equivalent ranslation using statistical or probabilistic information extracted from language text \[Brown 90, Nomiyama 91\]. Because many machine readable texts have been already collected nowadays, it is not difficult to extract statistical information of each expression in the texts semiautomatically. Moreover, the statistical information reflects the context in which eactt word occurs and implies the logical restrictions based on indirect structural dependencies.
Although we call the systems in a same word "statistics-based translation", statistical information used in the methods is diverse, such as translation probability, connectivity of words, statistics for (co-)occurrence, etc. We make comments on the characteristics and the limits of these systems.
The first method uses fertility probabilities, translation probabilities and distortion probabilities \[Brown 90\]. Fertility means the number of the words in target language that the word of ttle PROC. OF COLING-92. NANTES. AUG. 23-28, 1992 source language produces, and distortion means tile distance between the position of the word of the source language and the one of the target language. Tile method has been applied to au experiinental translation system from French to English. How ever, since these probabilities are extracted from a large amount of text pairs that are translations of each other, this method must be suffered from tile santo difficulties ,as examplc~b~sed translation in collecting and analyzing an adequate amount of bilingual corpora, and it's very difficult to apply this method to the languages whose linguistic structures aren't similar each other, such as English and Japanese.
The second method uses tile statistics for occurfence in target language text \[Nomiyama 91\]. It is calculated ill advance how frequently the each expression occurs in the t~rget language text, which needs only to belong the same tiehl as the source language text beblngs, but not to be a translated text of tile source language text. If there are more than one possible equivalent ranslations, the most frequent ranslation is selected through this calculated data. Moreover, this nrethod can be applied to make good use of the conventional methods of selecting equivalent ranslations, tbr it employs the frequency data exclusively when logical restrictions cannot select one out of candldates.
However this method hms one big problem. The high frequency of the expression in the target language text may not originate from the frequency of the expression in the source language text to be translated, because one target language expression does not correspond to only one source language expression ill general.
Suppose the following sentence is a first examph.': JAP: Sorlo saibankan-wa
that judge subj.
marker nekutai o tie obj.
marker kooto-to coat and or court katta. bought ENG: The judge bought a coat and a tie.
Figure 1 indicates translation process through bilingual dictionary and the statistics for cooccurrence of each pair of expressions in both Japanese and English necessary to translate tile sentence 1. The Japanese word 'qmoto" has two equivalent English translations: '(over)coat' and '(tennis) court'. We cannot decide which is eligible
1The statistics for co-occurrence of expressions shown in the figures are given provisionally for understanding. AcrEs DE COLlNG-92, NANT .ES, 23-28 AObq" 1992 5 2 7 with only logical restrictions on tile direct object of the Japanese verb "kau", because we can buy both 'coLt' and 'court' tile sentence "Tenisu-.kooto o kau" :: 'To buy a tennis court' is also quite acceptable. In this case, the statistics for co-occurrence in the target language English text denotes that the most frequent pair is 'court-judge', because the word ~COllr|' also lneans a qaw court'. Then using only statistical data on the targct language text misleads a wrong expression 'court' ms the. C<luivalent translation of "kooto", and the exanlple senteach may Im translated into 'The judge bought a court and a tic.'.
Tile second examph! is this sentence~: JAP: Kotori no
bird of kago-ni cage dative or marker basket booru-o bowl obj, or market ball EN(;: I put a bowl filled with water
in the bird cage. mizu o water obj.
marker ireta filled ella. put
q'ranslation process of this sentence and the statistics for co-occurrence are shown in Figure 2. Because the pair of 'basket' and 'ball' co-occurs most frequently in the target language, tile senteuce nlay be translated into 'I put a ball filled with water in the bird basket.'. 3 Equivalent "Ih'anslation Selection by Statistical Data on Dual Corpora of Source and Tat-get Languages
Now we propose a new method to provide rea sonable criteria for selecting a suitable equivalent tralMation or meaning using the simple statistical data extracted from source language text in addition to tile one from target language text. These source and target language texts don't have to be translations of each other. The proposed method gives us a way to select tile expression with the highest frequency of the target language text that keeps high frequency of the source language text at tim same time, so it overcomes the difficulty of the method using the frequency data on the target language text only, because it does not select the expression with the highest frequency of only the target language text.
~The subject phrase "watashi-wa" = T is omitted in this sentence. PRec. O1~ COLING-92, NANTES, AUG. 23-28, 1992
The method using only statistical data on the target language text may mislead a wrong equivalent translation, because in general each target language xpression corresponds to nmre than one source language expression.
The equivalent ranslation selection with statistics for co-occurrence in the target language text when a source language xpression S, has n equivalent translations in target language T,i(i = 1 •.. r~) is shown ~s this: S. Sb where Sk Tkl (i = 1... n) SCO(Ei,Ej) T,,i --~ SCO(T,i,Tbi) T6j source language xpression n target language eqnivalent translations of Sk statistics for co-occurrence of two exprt~sions EI,Ej The method using only statistical data on the target language text selects T,i which maximizes the statistics for co-occurrence in the target language text 3 as the equivalent translation of S,, where the partner of the co-occurrence Tbj plays the part of the basis for the equivalent ranslation selection. The bigg~t problem of this method is that Tbj which depends both b and j is selected by only statistical data on the targct language text.
Our new method provides reasonable criteria for selecting the basis for the equivalent ranslation selection using the statistical data on the source language text. First the source language expression Sb which maximizes the statistics for co-occurrence in the source language text 4 is selected, then the equivalent translation T,i which maximizes the statistics for co-occurrence in the target language text 5 is selected. The dependency relation in the source language is reflected in the translated text through this method. We call this method for equivalent translation and meaning selection DMAX Criteria (Doable Maximize Criteria based on Dual Corpora). ~,.2 Double Maximum Criteria based on
Dual Corpora
The algorithm of this method is summarized as follows: 1. Prepare the source and target language texts
(the target language text needs not to be a
translated text of the source language text). ~T.il maxb,15 SCO(T.~, Tbi) 4 Sb\[ maxl, SCO(S., Sb) bT. d max,5 SCO(T.I, Tbj) AcrEs DE COLING-92, NANTES, 23-28 AOOT 1992 5 2 8 2. Accumulate the statistics for co-occurrence of
every expression in both texts. 3. When it source language expression Sa has
n equivalent translations in target language
T~i(i = 1...n)
(a) Select S~l maxb sco(s., sb) (b) Select T,, I maxl,j SCO(T,I, Tbj) 3.3 Operation Example
Figure 1 3 show operation examples. Figurc 1 and 2 are examples of Japanesc~Engllsh translation. In Figure 1, with only statlstic,t\] data on the target language text, ~court' may be chosen as an equivah;nt translation of "kooto" because tim pair of 'eonrt-judge' co-occurs most frequeatly in the target language. However with DMAX Crite rio, the equivalent ranslation of "kooto" is selected correctly. • The expression which co-occurs with "kooto"
most frequently in the source language is • • The pair of the equivalent translation of "kooto" and the one of "nekutai" which cooccurs most frequently in the target language is 'coat tie'. As a result, "kooto" is translated into 'coat'.
It is the same ms shown in Figure 2. A pair of 'basket ball' co-occurs most frequently in the target language. But using DMAX Criteria, giving attention first to the most frequent pair in the source language text, "kotori-kago" cart gain the correct equivalent ranslation 'cage'. Next, a pair of "mizu-booru" decides 'bowl' as an equivalent translation of "booru'. Finally, correct trauslation can be acquired in this way.
Figure 3 shows the translation proct.~s and the statistics for co-occurrence of another English~ Japanese tr~tnslation example. ~NG: The coiling of the court was
cleaned quite well. # JAP: Saibansho no tenjoo~wa
court of ceiling subj. marker kireini sonjl-sareteita. quite well be cleaned
In this case, the English words 'court' and 'clean' have two meanings respectively. 'court' saibansho a room or building in which law cases
can be heard and judged kooto (a part of) an area specially prepared and
marked for various ball games, such as tennis PRec. OF COL1NG-92, NANaT.S, AUG. 23-28, 1992 'clean' souji-surn to clean rooms kuriiningu-surn to clean clothes with chemicals
instead of water A pair of "kooto-knriining'u" co-occurs most frequently in the target language, so the sentence may be translated into "Kooto -no ten joe-ha kireinl kuriiningu-sareteita.". But using DMAX Criteria, 'ceiling' is selected as a basis for the equivalent translation selection of 'court', and "saibansho" is selected as an equivalent ranslation of 'court' by the comparison between statistics for co-occurrence on the pairs of "tenjoo-saibansho" and "tenjoo kooto'.
Interpretation
In language understanding systems or machine translation systems throngh semantic expressions, one suitable meaning must be selected out of the ones described in a dictionary according to an entry word. However in conventional systems the meaning selection mechanism isn't robust and cannot select the most suitable meaning only by logical restrictions described in the dictionaries. We presented a new method for the equivalent translation selection in the former chapter using statistical data on source language and target language through bilingual dictionary. To apply this method to meaning selection, it is necessary to calculate statistical data on the pairs of each meaning in advance, but there is no means of calculating them automatically.
We have already devdoped an interlinguabased machine translation system whose interlingun named PIVOT doesn't depend on arty particular natural language \[Muraki 86, ichiyama 89, Okumura 91\]. In its dictionary, a.s illustrated in Figure 4., expressions in the source language are mapped onto some interlingua vocabularies (CONCEPTUAL-PRIMITIVE:CP), which are next mapped onto some equivalent ranslations. Then we propose a new method of computing linguistic statistics for occurrence of meanings automatically using this format of dictionary.
Suppose linguistic statistics on the pairs of expressions in both source and target language texts have already been calculated. In case of translation, when an expression Si occurs in the source language text, an equivalent ranslation Tij k is decided through the passage of Si :=*'Cij ~Tij~, and as a result, CPCIj is also selected from the CPs corresponding to the expression Si. Therefore, the linguistic statistics on the pairs of CPs or meanings Acrgs DE CoLING-92, NANT~, 23-28 Aot~r 1992 5 2 9 is nothing but coupling linguistic statistics on the pairs of corresponding exl)ressions in the target language text. Thus, the linguistic statistics ~Z on the pairs of the meaning expressions in the dictionary can be obtained as the sum of the linguistic statistics w on the pairs of target language oxpressious according to the following equation. fi(C.,,Cb,,) = ~wm(Ta.,v,T~,,,q)
p,q This linguistic statistics can be added to the dictionary in advance I and we c~n select the meaning in the same way as equivalent ranslation selection. 5 Conclusion
We lu'oposed a new method DMAX Criteria (Double Maximize Criteria based on Dual Corpora) in this paper. It can select a suitable equivalent translation or meaning using the statistical data extracted from both source and target language corpora even when linguistic restrictions described in the dictionary or grammar cannot. The depen dency relation in the source language is reflected in the translated text through bilingual dictionary. Moreover, the nmthod has the following features: 1. It utilizes linguistic statistics as context infer
motion in addition to logical restrictions effec-
tive for ambiguity resolution. 2. The source of the linguistic statistics is the dual corpora of source and target languages, not the bilingual corpora (the target language text doesn't have to be the translation of the source language text). 3. The linguistic statistics can be coml)uted send-
automatically in advance. 4. Tim linguistic statistics on the pairs of mean-
ing expressions are computed from the lin-
guistic statistics in source and target language
texts with the interlingua-based bilingmd dic-
tlonary to resolve ambiguity in meaning inter-.
pretation.
Based on this method, we have carried out an experinmnt on a limited-scale translation system, attd confirmed eti~ctiveness of the method. We are preparing further experiments on a large-scMe dual corpora with PIVOT interlingua dictionary. Their result will he reported on another paper. 6 Acknowledgments
The authors wish to thank Mr. Masse WATARI for his continuous encouragement. The authors also thank the members of Media Technology Lal> oratory for their good suggestkms. PRec. OF COLING-92, NA~rES, AUG. 23-28, 1992 ~eferences \[Brown 90\] P.F.Brown et al. "A Statistical Ap-
proach to Machine Translation", Computa-
tional Linguistics, Vol.16, No.2, 1990 \[Doi 92\] S.Doi and K.Muraki "Robust Translation
and Meaning Interpretation Mechanism based
on Examples in Dictionary", Prec. of 44th
Annua\] Conference oflPSJ , 1P-2, 1992 (in
Japanese) \[Ichiyama 89\] S.Ichiyama "Multi-lingual Machine
Translation System," O~ce Equipment and
Products, 18-131, pp.46-48, August 1989 \[Muraki 86\] K.Murakl "VENUS: Two-phase Ma-
chine Translation System," Future Genera-
tions Computer Systems, 2, pp.117-119, 1986 \[Muraki 91\] K.Muraki and S.Dol "Translation
Ambiguity Resolution by using Text Corpora
of Source and Target Languages", Proc. of 5th
An~ua/ Conference of JSAI, 11-7, 1991 (in
Japanese) \[Nagao 84\] M.Nagao "A Framework of a Mechani-
cal Translation between Japanese and English
by Analogy Principle", ArtificiM and Human
Intelligence, ed. A.Elithorn and R.Banerji,
North-Holland, 1984 \[Nomiyama 91\] H.Nomiyama "Lexieal Selection
Mechanism Using Target Language Knowl-
edge and Its Learning Ability", IPSJ-WG ,
NL86 8 , 1991 (in Japanese) \[Okumura 91\] A.Okumura, K.Muraki and S.Aka-
mine "Multi-lingual Sentence Generation from
the PIVOT interlingua," Prec. of MT SUM-
MIT III, pp.67-71, July 1991 \[Sat, o 90\] S.Sato and M.Nagao "Toward Memory-
based Translation", COLING-90, 1990 \[Sumita90\] E.Sumita, H.Iida and H.Kohyama
"Example-based Approach in Machine Trans-
lation", hffoJapan'90, 1990 ~tatistics for co-occurrence ~ in source language text f nekutai ~ ~, 50 \ saibankan Yi" .. Baibansho J . . . . . . . Figure 1 -sono
'The judge bought a coat and a tie.' AcrEs DE COLING-92, NANTES, 23-28 AOt'rr 1992 5 3 0 PROC. OF COLING-92, NANTES, AUG. 23-28, 1992 statistics for co-occurrence in target language text .-tle 50 ~-coat I ; ~ ,' ,' , ~ . . . . . saibankan-wa kooto-to nekutai-o katta." bilingual dictionary nekutai--t|e _~- coat ~.salbankan--Judge ..... ¢.saibansho .... court" ,, t ~ , ~ ; ' ; . • statistics for co-occurrence, ~, ~ ~' I bilingual dictionary fstatistics ;. for eo-occurrcllCe . , , J m source language text ~ ; ,,in target langoage text , / _~oeorl ~-----~,---~r~oeori .bird ~ ~ bird "~ ,,
50 " ~ ,'~-e ".L____ ____N. catle ~ ! 4,0 l booru ' I ~ b o o 3. i =izu ~ = i z , i r u ~ u $ jba,, ~ --'-bowl ~water ........... ....... e , ................................ 60 be,, ~ ~v----"~--~ bOWl ~ w a t e r ' 1 , I , ! \] Figure 2 "Kotori-no kago-ni mizu-o ireta booru-o oita."
'I put a bowl filled with water in the bird cage.' $ s ;statistics . for co-occurrence, I in source language text ~ l , I t bihngoal . . . . dictionary ,' , ,, I statistics for co-uccurrence ~ in target langmige text celllnn -. @celllng--een3oo court clean coat Figure 3 ~,~eon3oo ; ' • -, ~ ,--7 kooeo ""q- ~\]~0 / 30 ! ;/20 .. A t -~ , -~ s ° u J : L '-~"d~|l ~ ~ "~,~ - kuriinin~" ~u ~-~ ~" ' ~" N ~-----~-~, , Crourt<:~kooeo 20 ~ ~ , I ', $ ~sou~i d~rclean<z~ , , Kurllnln .", ~.o ~, '~ J ....... ~o.-~-coat kooeo • 'The ceiling of the court was cleaned quite well.' "Saibansho no tenjoo-wa kireini souji-sareteita." '2~ i :~ • i X : , x. : source language expressions interlingua target language expressions
~'--~.~..~_. • ~.......... • Figure 4 Bilingual dictionary of the interlingua-based translation system • AETES DECOLING-92. NANTES, 23-28 AOt~" 1992 5 3 1 I'ROC. OV COLING-92, NANIT, S. AtJc;. 23-28. 1992