In optical character ecognition and coni.inuous speech recognition of a natural language, it has been diflicult to detect error characters which are wrongly deleted and inserted. \]n <>rder to judge three types of the errors, which are characters wrongly substituted, deleted or inserted in a Japanese "bunsetsu" and an l';nglish word, and to correct these errors, this paper proposes new methods using rn-th order Markov chain model for Japanese "l~anjikana" characters and Fmglish alphabets, assuming that Markov l)robability of a correct chain of syllables or "kanji-kana" characters is greater than that of erroneous chains. From the results of the experiments, it is concluded that the methods is usefld for detecting as well as correcting these errors in Japanese "bunsetsu" and English words. Key words: Markov model, error detection, error correction, bunsetsu, substitution, deletion, insertion
In order to improve the man-machine interface with computers, the <tevelopment of input devices such as optical cha.racter tea<lets (OCR) or speech recognition devices are expected, llowew;r, it is not easy to input Japanese sentences J)y these devices, because. they are written by many kinds of characters, especially thousands of "kanji" characters. The sentences input through an OCR. or a speech recognition device usuMly contain erroneous character strings.
The techniques of natural language processing are expected to find and correct these errors. tIowever, since current technologies of natural language analysis have been developed for correct sentences, they cannot directly be applied to these problems. Up to now, statistical approaches have been made to this problem.
Markov mo<lels are considered to be one of" machine learning models, sinfilar to neural networks a.nd fuzzy models. They have been applied to character chains of natural lang,,a~ges (e.g.,l);nglish)\[l\],\[2\], a.nd to phoneme cha.ins in continuous peech. reco~gnition . \[ 3 1~1. . \[41 ¢ ' 2nd-orde.r Markov model nt bunsets',l is known to be useful to correct errors in "kanjikana." "/m nsetsu" \[(;\],to choose a correct syllable chain from Japa.nese syllable "bunsetsu" candidates \[7\], and to re(!nce the ambiguities in translation processing of non-segmented "kana." sentences into "kanji-kana" sentences \[8\].
The erroneous characters can be classilied Ul,O three types, lhe hrst is w~ongly recognized chal;aclers instead of correct (haracters. The second and the third are wrongly inserted and deleted (skipped) characters respectively. Markov chain mode.Is above mentioned were restricted to tind and correct the first type of errors\[5\],\[6\]. No method has been proposed for correcting errors of the second and the. third types. 'Phe. rea.son might be considered to be I.he di\[ticulties of finding the error location and distinguishing between deletion and insertion er I'ors.
On the other hand, contextual algorithm utilizing ,,-g,'atn letl.er statistics (e.g.\[.()\]) a,,d a dictionary look-ul) algorithm\[10\] have been discussed to detect a.nd correct erroneous characters in English sentences, which is segmented into words.
This paper proposes new methods, which are able to be applied to a nor>segmented chains or" characters, to judge three types of the errors, which are characters wrongly subst.ituted, deleted a.nd inserted in a Japanese "bunsetsu", and to correct these errors in Japanese "kanji-l<ana" chains using m-th of der Markov chain model. The methods are based on the idea about the relation between the types of errors and the length of a chain in which the wdnes of Markov joint probability remain small, l,'urthermore, this method is applied to detect and correct errors in segmented English words•
Experiments were conducted for the case of 2nd-order and 3rd-order Markov model, and they were applied to Japanese and English newspaper arhcles. Relevance Factor 1 and "Reeall Factor" R for erroneous characters detected and corrected by this method were experimentally evaluated using statistical data for 70 issues of a daily Japanese newspaper and 5 issues of a daily English newspaper.
Method of Error Detection and Error Correction using
2,1 Basic Definitions
In this paper, two types of natural language's sentences are discussed. One is a Japanese sentence, which is non-segmented sentence and the other is an English sentence, which is segmented into words.
A Japanese sentence can be separated into syntactic units called '%unsetsu", where a "bunsetsu" is composed of one {~" m ( lependent word" and ~ sequence of n (greater than equal to 0) "dependent words".
A "bunsetsu" is a chain of Japanese "kanjikana" characters or an English word is a chain of alphabets, and are represented by 3' = sl s2...s,~, where s~ is a "kanji-kana" character or an alphabet. In particular, a chain, 7 , is called a "J-bunsetsu" when all of its elements are "kanji-kana" characters, and is called a "\[iJword" when all of its elements are English alphabets. The set of eorre, ct .lapanese "bunsetsu" or English words is represented by Pc.
Three types of erroneous "J-bunsetsu" or
E-word are dehned as follows:
First, a chain ce = N,¢~... s\[Zlg;.., s,;~ is called a "(i,k)-Erroneous J-bunsetsu or Eword Wrongly Substituted " ( (i, k)-EWS) if a subehain fl = tltu... Ih is wrongly substituted at the location i of ce, that is 3 7 Cre, -y = ~(o11/< Here ~(Ollf3 donotes substitution of a subchain fl at, the location i in a chain c~ , that is, d01i/ &,"',tk ~-- siq~-,.
Next, a chain c~ = &g.,... si~_lgi ... s;,~ is called a "(i,k)-Erroneous J-bunsetsu or I';word Wrongly Deleted" ((i,~)-~WD) if a subchMn fl = t~t=...tk is wrongly deleted at the location i of a, that is ~7 ~ l'c, "y = c~ (1) << ft. Here {,(0 << fl denotes insertion of a subchain fl at the location i in a chain c, , that is, a (0 << fl -- s't.sS"' Si~-lltl2 "" "lk,qi''" S~n.
Finally, a chain cr = .¢t • "" s\[-lgi"" si-(k-1 s;+k"' s;,~ is also called "(i, k)-Erroneous Jbunsetsu or F~word Wrongly Inserted" ( (i,k)-EWI) if a sul)chain /3 = tlt~...tk is wrongly inserted at the location i of % that is 37 E Pc', 7 = d;) >> ft. tIere c~ (1) >> fl denotes deletion of a subchain f3 at the location i in a chain c~ , that is, at0 >> fl = .¢~'2'"siZlsi+~"'sZ, and tl = gl,'",tk = Si+k-1. The set of (i,k)-EWS, (i, k')-EWD and (i, k)-EWI are represented by P(~)s', P~) and 17(1 ~) respectively. In this paper, all inputs "bunsetsn" or all inputs words to computers are assumed to belong to one of l-'c , p(k)s ,P~) and 1'(1 k).
Next, the meaning of detecting and correcting errors are define.d in the Nllowing. The words, "error detection problem", means the problem how to detect the location i of error in if, and "error correction problem" means the problem how to replace an erroneous "d-bunsetsu" or an "E-word" ~v by a correct "bunsetsu" or an English word 7, where
s ,aEP , oreeEP aud'rcic.
"Relevance Factor" p(D) and "Recall l)'actor" R (D) for tile "error detection problem" is defined as follows: \]): p(D) ' '~ " t,. 2 at ( the number of "J-bunsetsu" or \],-word the location i and length k of error ill I '(k) .s' , "n p(k) or \]7,5k) is correctly detected )/ q ,, he total number . of J-lmnsetsu . or ,, Bword detected as erroneous ,l-bunsetsn or "E-word"). (2): {{ ~ R. (D) " ~_ ( the ,mmber of "a-bunsetsu" or l~-word that the location, • . and length k of error in P!s. ~), ~ p(k) D or p~k) is correctly detected ) / ( the number of all "3-bunsetsu" or "Ewo,d" in t,,e set ) o,, :' )p,,epared in advance ).
"Releva.nce factor" p(C) and "Recall factor" R (c) for the, "error correction problem" is also similarly defined. Here p}D) denotes the "Relevance Factor" for tile "error detection problem" of \] ~(k) ~' , and R (c) D denotes the "Recall Factor" for the "error correction problem" of p(k) respectively. D
We introduce the following assumption according to the experiences.
Assumption Each Markov probability for erroneous chains of "ka@-kana" characters or English alphabets is small compared to that of correct chains. ! According to this assumption, the. procedure. of detecting the location i .~nd the length k of error chains arc detined as follows:
Pwcedure 1 ( Method of detecting the location and the length of chain wrongly substituted in p(.k) and substituted or in.qerted in
sPi~\[(l the subchMn of lelx~th k which satisfy the followin~ conditions..\['his chain is iudge~t to be wrongly inserted at the location ~. (1) P(Xh I Sh-,~ "'' Z,,_t) > 5", r,,r I,,--i- I
orh.=i+k+mand (2) r(xs I xs-,,... ,'%.<) < 'r, ro~. vj su,:H
that i < j < i+ k + m- 1, where P(Xj I Xj-v,...Xj-+)is ,~+-th order Markov chain probability which denotes prol> ability of occurrence of sueeessiw+ character Xj when string Xj .... • • • Xj-t has occurred, mtd X,, denotes a space symbol if u < 0. And T denotes a critical v',dne of m-th order Markov probability used for detecting errors.
ri'his procedure detects that k characters a.rv. wrongly substituted or inserte(l at the. location i, if m-th order Markov probability for,cha.in remMn smMler vMue than critical wdue 1' just (k+m) times fi'om the location i to i+k+m- 1.
l?or an example, the change o~ the val ~(; oI 2hal-order Markov probability for each eharac.ter of the erroneous chain \[,!~,e) or l'~ '2) is shown in l~ig. 1. In this ex~tmph{, \[wo charaet,ers are wrongly substituted or inserted. According to the previous assumption, 2nd-order Markov probability for erroneous~:!tain remain smaller value than eriticM value l just four tinms. S~ Sz $3 S,, Ss S~ Sv Sa O0000C) O0
I _ _ 1 ' l' (S3ISISI) > T
L__X__J I' (S~\[$2S3) < T
L----X_---~ l' (S~ISJS~) < T 'O; Enoneous chalacter \[_ X j l' (S6\[SISs) < T X : \] ecation o\[chatacler which has tile ~ - - J I' (S~\[q~";6~ <T
value n\[ Ma~kov probability smaller than '\[' L ...... J T: Clitical value of Markov probability 1' (S81,S:g, S7 ) >T Fig.1. Change ofthewflueof2nd-ordcrMarkovprobabilltlcs
for each character ofthe erroneous rain,t: inch~dhu: ........................... Wt'ongly substltutcd or inserted chalactcrs lln case to (h+LecL errors in Markov model, it is able to presunw.d Lhal; a sul)chM,L # of length '2 is wrongly inserted at I,h(,. location i of erroneous ('\]tah~ (~, if 2n(l<n'der Mar\]coy prol)al)iltry for erroneous chain ~v remMn smalhw than .'/' just four times from location i.
l'roee, dure 2 ( Method of detecting the location of chain wrongly deleted in 1.'52 ~) ) Find the, subchain of length k which Satisfy the following conditious. 'Fhis clmin is judged t,o /)e wrongly deleted ~t the. location i. (~) r(x,, I~ ,,-.,,, " x,~_,) > I, to,. h = i-l
or h.=i+k+'m, and (2) e'(X# I Xj ..... -..Xj_,) < '/', for V# such
that i<j <i-t-'m-- 1, whe.re 7' denotes a critical value of 'n>th order Markov i.'obMfility used for detecting errors. I
If m-th order Mal'kOv probM)ilities R)r chain remMn smaller than the critical wdue 7' just m tinms from the location i Lo i + m-1, it is judged that some characters are wrongly (teleted at the location i. l\[owever note that length k of characters wrongly deleted at the location i, can not be, de.termined by this procedure, the length k is determined by the proc(,dure 4 shown in Sec. 2.3.
Table \] shows that the relation of times that Markov prolm.bilities remain slnaller than 7' in the cases of Ist- aml 2nd-order Markov models. li'rroneous (:hains (:an I)e classified into the following two eases: on('. is a case of the eh;.mmters wrongly substituted or inserted, the other is a class of the eha.racl.ers wrongly deleted. Table I' q'he mnnber of times that Markov l)rol)al)ility of the erroneous chains remain a
smaller than T i),ypii tsi-i,,.,ti;, M~;\]~i;v \[~iil:oi:;t(i,: M,~,HiSv /2 i -7'!~ ) , IJp I' ' ) - ;(~Y D I "
i'~ ':) ,'5 ~y ~,3 t~,re,~ .i,nes " (k+\])-iiiiiiiS-
one times
¢me ti rues
one times two ti.,e~ ~ " tl.'ee times (/c-k:i 5 thnes -
:ro.r iime~ ......
(k-l-2)-iimes -
two times . . . . . two times two times
t~,rol i3~,el-
- ro"r-tim;s -(k'fl~27Lirne, s .... . P~ 2) using 2rid-order
However, this method can not distinguish the erroneous characters wrongly substituted, from the characters wrongly inserted in the former c~e~ and can not determine the length k for the type of 1?~ ), because the Markov probability of any erroneous chmns in Pl) ~) remMns small value just the same times for length k. These problems can be solved by the procedt/re 3 and 4 shown in Sec.2.3.
In this paper, the effect to detect errors for cases of length k = 1, 2 is evaluated. 2.3 The Method of Error Correction us-
ing 2rid-Order Markov Model
The procedure of replacing erroneous chains by correct chains using Markov model is presented as follows:
Procedure 3 ( Method of correcting the chains in r(2 or )
"bunsetsu" or words ~ = s-lg2 • "" s(l& • • • si+h-lSi+h or o: = ,¢ts~:~ ''" si"--tg~"" s~+~-ts¢+k"" s~ denotes a " (i, k)-EWS" and a "(i, k)-EWI" and a subchain fl = tit=.., tk is assumed to be wrongly substituted or inserted at the location i of cY respectively. Then the erroneous chain ae can be replaced by the following correct chain "y in I'c if condition (1) is satisfied. l I ~- &,"',lk +--- s~-I or 7=c~(;) >> fl ~_ 8182 ^ ,,, Si_tSi+ ^ ^ k . . . S,n ^ : and 11 = sl, ^ ,, ,, l h = si+k-1 P(xs I > r for Vj such thati+k<j < i+k + m-1. By comparing Markov probability for correct chains in two cases above, choose a correct chain which has the great Markov probability. |
Procedure 4 ( Method of correcting the erroneous chains in P~) )
A chain ~ = &&...scLtgl...s;,~ denotes a "(, )i k-EWD", and. asubcham' .. o=ltl,>...lk.~s assumed to be wrongly delete(t)at the location i of c~. Then the erroneous chain c~ can be replaced by the following correct chain 7 in I'e if condition (1) is satisfied. 3' = oe(O << fl ~qt ~42 • ' • Si~.t~l\[ 2 • • • ~kS-i • • • Sin. (1) P(X~ I x~_,~.., xj_,) > r, rot Vj such thati+k<j <i+k+m-1. |
An example of correcting the erroneous chain, two characters of which are wrongly substituted (P(~) ), is shown in Fig. 2. If Markov probabilities do not remain smaller than critical value T, then it is judged that these erroneous chains have been corrected. SI $2 0 O $1 O (:Critical w~lue of Markov probability ~ su\[)st\[tulio\[1 CI'I'OI'S $1 $2 Sa S~1 So~ Sa Sz $8 O O O@@ O O O I I P(XellX2Xa) > T I ! P(X~,elXaXe,) > T I P(XGIX,~IX~) > T I
P(XTIX~2Xo) > T Choose the candidate of "bunsetsu",which has a great Markov probability in two cases Fi9.2 Procedure for correcting an erroneous
string using error detection 3 Experimental Results 3.1 Experimental Conditions \]. The number of "bunsetsu" for 70 issues
of a daily Japanese newspaper: 283,96:~
~bnnsets\[l" 2. The number of words for 5 issues ofadaily
English newspaper: 155,,159 wor(Is 3. Type of errors and the numt)(,r of "bnn-
se{sll" :
8(10 "bunsetsu" are prepared for each of l:!,!), rS? ) l'i and l'? (a) The average length of "bunsetsu"
composed of "kanji-kana" character
chmns: 6 characters
(b) The twera,ge length of alphabets
composed of correct English words
chains : 7 characters 4. Markov model of Japanese "kanji-l{ana"
characters :2nd-order Markov Model 5. Markov models of Fnglish alphabets:
2nd- and 3rd-order Markov models Sa $4 Ss So Sz SB O Q® O 0 0
(1) Correction of
insertion errors $2 Sa So $7 So O O O 0 0 \] P(X6lX2Xa) ) \]I
J P(XTIXaX~O>T / (2) Correction of 3.2 Experimental Results and Discussion
The accuracy of error detection ~md error correction depends on the critical va,lue 7' of Markov proba, bilities. "Rehwance Factor" P and "ReaM1 Factor" R, for e;tch method were obtained by changing the wdue of T. \[1\] The Relation between P and R of Detectmg Erroneous ChMn Using \])eteetion Proee~ dure
'Phe relation between P and R for the location of erroneous "k,~nji-kan£' chains det, ecl.ed in p(t). s' , P(~)s, .,P(t)., ..\]'(~), 1'~ t), ;rod l'~ ~) using Procedure 1 a,Ild 2, are. S\]lOWll ill Fig. 3, ;t.lI{{ those for erroneous Mp}utbets chains ;u:e shown in Fig. 4.
From these figures, the following results are obtained : 1. The maximum wdue of P and R of detectins erroneous characters wrongly inserted or substituted, is greater than that of erroneous characters wrongly deleted'. (a) In the case of "J-bunsetsu" : r}") = 07- 09%, n.~ '') = l~fl ) = 100%, r~(~)) " t) = \]{/') = 88- 94%, //') = (b) In the case of 'q~word": P}') = 38-49%, R5 ") = ,{p) = 94- ,as{~, 4'/) = P~/') = 42 - s8%, n!,P ) = 2. Compsred with the, se maximM wdues, it iS shown tha% the Irla, xilnuin va, hle o\[ i)ro(\] uct of P and R for "k~nji-kmu~" %unsetsu" ix 35%--60% greater than that of English words. \[2\] The Relation between l) and .R of Cha.ins Corrected Using Correction l'rocedure
The relation betwee, n 1" ~nd IC of ",lbunsetsn" corrected using Procedure 3 and 4 for p(t) -p(2) pO),(2) F~l.), P~ 2) of ".\]-bunsetsu" are shown ill Fig. 5.
From this tigure, the following results ~l'e obtained :
The maximum wduc of P and \]Z of correctins erroneous etum~eters wrongly inserted or substituted, using 2nd-order M~u'kov model, is greater thcnn that of erroneous cluu'acters wrongly deleted. r} ~> = 92- 98%, n,? '~ = ~):~ -9r% p~O) : r,a - 8~r0, .4; ,) = 4(;- <.)% PI. c) 69 - 9,t%, R (c) {P2- 88% ~}7 - ,a~)% 57-58% 88 -%%
:\]9 - 42% b o r Fig.3. Experireenlal resuRs for detecting a location of an erroneous
"kanji-kana" string using the error detection procedure !
____ l .... I_ _ _ t _ _ l L 0 2 0 ,I 0 610 Recall lacier \[%\] Fig.4. Experienlal resulls Ior delecling a localion of an erroneous
I£ngli."Ji words using Ihe error doleclion procedure Z °'a' ;i ~ o °'~ : I:ig.5. Expelimenlal results (or correcting an erfolleous
"kanji-kana" siring using error correction procedure # *w .... 7¢
^o ^e o ^ 0--0 : Fo BI o u o--I : F t~ I
D--KJ : I'1 °)
W--Mr : l'n ~1
A--Z~ : Fs (n}
~k--,& : Fs (a Recall lacier l~l~ A "o ,o o ^ 6 u ~n • • o r I 7-----r ----7 l 1-¢¢o • q o o a,0,,'/'\[
'e~ o--o : I" ~'} o ,,,~,t~ \[-\]-D : 1", ('~ N N FI R) A~-A : l's (q .~-..& : l-s I:) u, n • cl ~l=j , L, , O--O : Po 0) m • PO p) ~--\[~ : P a Iq ~.--~. : i,s p)
\[3\] The Combinatorial Effect to Correct Erroneous Lnghsh Words Using the Spell Checker and the Correction Procedure by Marker Model
The experimental results of detecting errors in English words using Ispell ( Interactive Spell checker ) is shown in Table 2. l?rom the results, it is seen that Ispell cart almost perfectly detect erroneous words in U~, I'~) and P s. using dictionary, but it cannot perfectly cur rect erroneous words, because it can output the correct candidates for erroneous words in p~), r(~), pO)s, but can not output the correct candidates for erroneous words in F~ ~), P(~) ~1) 1 F~ ~). It is necessary todetect the locationoferroneous alphabets in words to detect MI these errors. However, it should be noted that \[spell can not detect the location of erroneous alphabets in words.
In order to detect and correct erroneous "Eword" more effectively, the method to combine Ispell and the procedure (in see. 2.3 ) using Markov model is expected• The combinatorial method is denoted in the following way: (1) At first, erroneous "E-words" are detected by Ispell, but the locations of erroneous alphabets in words can not be detected by it. (2) Next decide the correct candidates words by procedure 3 and 4. (3) Finally, ls!)ell again checks if these candidates are correct words. The experimental results using this method is shown in Fig. 6(2nd-order) and in Fig. 7( 3rd-order ). From the results, it is seen that this combinatorial method of Ispell and the procedure by 3rd-order Markov model to very useflfl to detect and correct all errors in English words.
It takes about 10 milli-seeonds and 6 seconds in average to detect and to correct erroneous "bunsetsu" . Examples of "bunsetsu" and the output results of error de\]cotion and error correction using Mm!kov model, are shown in Fig. 8. r i) (e r 01 l" i U) r,,~
Table 2 The capability of error detection, usipg Ispcll
Able Io dclecl Unable It) dclcct Widl corrcct candidate Wltht}ut carrcct candM;dc
7 ({. 0% 17. 5%
0% 7~, 5%
B2. 0% 18. 0%
0% i00. 0%
80. 5% 18. 5%
4. 0% g6, 0%
G, 5% 20. 5%
0%
0%
1. 0%
0% o o . . . , , , , , . . . , • o o o , A ~ Q o o o o o .o " o D--O : r 1,1 l--Ill : rl In Z~--A : rs I') ~-~ : I~S Ct)
Rank 20 Fig.6. Exprimental resuR lot correcting erroneous English words
using Ispell and error correction procedure in case of
2rd-order Marker rnodel M--Ill : rl I~} ~.--g3~ : \]'S Ill ~k--~. : rs p) o ~'o . . . . Flank ~ .~ Fig.7. Exprimental result rot correcting erroneous English words
using Ispell and error correction precedtgo in case of
3rd-order Marker model ! \[1'; .......... inlmt "b ..... tsu" :-: I, Y ~'7--/~-~ \] • O.tput result (co.ccl bunsetsu) of er,or corrcclion : -: b.~ ov'411 * (a) Case of an erroneous ylluble "bunselsu" for \]@1 Erroneous hq)ut "bunselsu" : l'l..J,l~.\] • Output result (crror lx)shion) of error detcclion : first character • Oulpul tcsull (cogccl bunsctsu) of error coefcclion : ~I~..~.. A~,~,,.'~ (b) Case of an crroccorls "kanji-ka\],,'~" "bu,rso st," for FD (' \]:\]g.8. E×amp\]cs of cn'oncous "buaselsu" and the resulls
of relot dctcctlon and error correclion 4 Conclusion
This l),~per proposed the methods to .ittdge three type of errors mM correct these errors, which are characters wrongly substituted, inserted ~nd deleted in the .l~panese "ka.njikmt,~" chains and English words using m-th order Marker model.
The effects of the methods were experimentally ev;dnated for the case of 2nd- and 3rdorder M~rkov chain. ~'rom the exI)erimental results, the following conclusions have been obt;dned:
llowever they are not so usefltl for detecting and correcting of eharactells, wrongly deleted in "k,~I\ji-kana" "bunsetsu". 1: hen, m(>re .flicient rrmthods are expected for this type of errors. References
\[1\] T.Araki,J.MurM~;mfi and S.Ilcehm:a "l';f-
feet of Reducing Ambiguity of I/eeog-
nition Candidates in J~panese thmset.su
Unit by 2nd-Order Mm'kov Model of Syl-
lables", Information Processing .S'ociely of
Japan, Vol.30, No.4, pp.4(;7-,t77 (l!)8!)) \[2\] S.Ikehar~ ;rod S.Shira.i "./al)anese (?ha, r-
atter Error Detection by Word Analysis
mid Correction Candidate l'~xtra.ction t>y
2nd-Order Mm:kov Model ", h@,'mation
Processing Sociely of Japan, Vol.25, No.2,
pp.298-305 (:1984) \[3\] F.Jelinek "Contimmus Speech Recogni-
tion by Statistical Methods", Pro< of lhe
IEEE, Vol.64, No.4, pp.532-556 (197(;) \[4:\] 'P. Kurita ~md T.Aiz~w~ "A Method for
Correcting Errors on Japanese Word \]n-
put and Its Application to Si)oken WoM
Recognition with Large Voenlml+u:y", /n-
formation Processing Society of Japan,
Vol.25, No.5, pp.831-841 (1984) \[5\] J.Murakami,T.Araki and S.\]keharn "The
Elfect of 'Prigr~m Model in .hq)anese
Speech ILecognition", 7'he Institute of l';leeironies, lnfln'raation and Com)'nunicalion Engineers, Vol.J75-1)-ll, No.l, pp.l 1-20 (1992) \[G\] Y. Ooyama. ~tn(l Y. Miya.za.ki "Natura.l
Languag{: lq'ocessing in ;~ JN)a.nese.-text-
to-speech System ", Information Fro-
('essin(.l ,¢ociety of Jal)an , Vo1.27, No.111,
p p. 1053-10(; 1 ( 1 !) 86) \[7\] ,}.l,.l)eterso)t "Coml)uter Progra.ms for
l)et.ecthtg and Correcting Spelling l~r-
rots", Comm., A(;M, Vol. 23, No. :12, pl ).
(;7(;-(;87 (1980) \[8\] l,.l{.lLa.biner,S.l';.l~evinson a.nd M.M.
,qo)tdai "On the Al:>l>lic++tion of VeC-
(r()~ uQuantization arl|(\] Ilidden Ma.rlmv
Models t(> Sl)eaker-indepetl<lent , lsol;~ted
Word Ilecognition", Bell ,5"ystern Techni-
cal .\]o.urnal, Vol.62, No.4, pp.1075-1105 \[9\] I';.M.l-(i.qeman +rod A.l/.. l\[a.nson "A (\[~on-
textttM l)osi.processing System for I';rror
Correction Using I\]ina.ry n-C,r~m", II','EI';
Trans. G'ompul., Vol. (?-22/, No. 5, pp.
480-<I!)3 (11974) \[10\] C.l'3.,qlmtmon "Ma.(.hematical Theory of
Commurfication", tlell £'yslem %'~chnical
.\]ournal, Vol.27, i)1).379-423, (;23-656, Oc-
tober (l 9'18) \[1 it\] C.f,;.Shamt(m "l)redi(:l.ion and Entropy of
Printed lgnglish", Hell Syslem "Technical
Journal, Vet. 30, i)i).50-6,1, January (1951 ) ?93