The overall structure of PRONOUNCE is as shown in Fig. 1. The lexical database consists of "approx­imately 20,000 words based on Webster's Pocket Dic­tionary" in which text and phonemes have been auto­matically aligned. Dcdina and Nusbaum acknowledge the crude nature of their alignment procedure, saying it "was carried out by a simple Lisp program that only uses knowledge about which phonemes are consonants and which arc vowels."

An input siring is matched in turn against all ortho­graphic entries in the lexicon. The process starts with the input string and the current dictionary entry left-aligned. Information about matching letter substrings

INPUT (spelling pattern)

Substring matching alignment

Build pronunciation lattice

Decision function

OUTPUT (pronunciation)

- and their corresponding phoneme substrings in the dictionary entry under consideration - is entered into a pronunciation lattice as detailed below. The shorter of the two strings is then shifted right by one letter and the process repeated. This continues until the two are right-aligned, i.e. the number of right shifts is equal to the difference in length between the two strings. The process is repeated for all words in the dictionary.

A node of the lattice represents a matched letter, L, , at some position, i, in the input, as illustrated in Fig. 2. The node is labelled with its position index i and with the phoneme which corresponds to L, in the matched substring, 1),,, say, for the mth matched sub­string. An arc is placed from node i to node j if there is a matched substring starting with L, and ending with Lr The arc is labelled with the phonemes in­termediate between /',,„ and Pj,„ in the phoneme part of the matched substring. Note that the empty string labels arcs corresponding to bigrams: the two symbols of the bigram label the nodes at cither end. Addition­ally, arcs arc labelled with a "frequency" count which is incremented by one each time that substring (with that pronunciation) is matched during the pass through the lexicon. Finally, there is a Start node at position 0 and an End node at position one greater than the length of the input string.

Sullivan and Damper employ a more principled align­ment procedure based on the Lawrence and Kaye (1986) algorithm. By pre-computing mappings and their statistics, they implemented a considerably more 'implicit' form of PbA: there is no explicit matching of thé input string with lexical entries. Their pronun­ciation lattice differs, with nodes representing junc­tures between symbols and arcs representing letter-phoneme mappings. They also examine different ways of numerically ranking candidates, taking into account probabilities estimates for the letter-phoneme map­pings used in the assembled pronunciation.

Given the improved alignment and candidate-ranking methods, better performance than Dedina and Nusbaum might be expected. On the contrary, Sullivan and Damper's best result on the full set of 131 pseudo-words from Glushko (1979) (plus another 5 words -see section 5.1) is only 70.6% (1993, p. 449). This is an error rate of almost 30%, as compared to Dcdina and Nusbaum's 9% on the smaller test set of size 70. Differences in test-set size and between British and American English, the transcription standards of the phoneticians, and the lexicons employed seem insuffi­cient to explain this.

To examine any impact that the specific lexical data­base might have on performance, we have used two in this work: the 20,009 words of Webster's Pocket Dictionary and the 16,280 words of the Teacher's Word Book (TWB) (Thorndike and Lorge, 1944). In both cases, letters and phonemes have previously been hand-aligned for the purposes of training back-propagation networks. The Webster's database is that used by Sejnowski and Rosenberg (1987) to train and test NETtalk. The TWB database is that used by Mc-Culloch, Bedworth and Bridle (1987) for NETspeak.

The re-implementation was programmed in C on a Hewlett-Packard 712/80 workstation running HP-UX. A 'direct' version scores candidates using Dedina and Nusbaum's method with its two prioritised heurist­ics: we call this model D&N. Two other methods for scoring have also been implemented. In one, we re­place the second (maximum sum) heuristic with the maximum product of the arc frequencies: we call this model PROD. (It still selects primarily on the basis of shortest path length.) We have also implemented a version which uses a single heuristic. This takes the product along each possible path from Start to End of the mapping probabilities for that arc. These arc computed using Method 1 {a priori version) of Sulli­van and Damper (1993, pp. 446-447). For all paths corresponding to the same pronunciation, these values arc summed to give an overall score for that pronun­ciation. Wc call this the MP model. The final product score is not a proper probability for the assembled pro­nunciation, since the scores do not sum to one over all the candidates.

The 'best' pronunciation is found by depth-first search of the lattice, implemented as a prcorder tree traversal. For the D&N and PROD models, paths were pruned when their length exceeded the shortest found so far for that input, leading to a useful reduction in run times. A similarly motivated pruning was carried out for the MP model. If any product fell below a threshold during traversal, its corresponding path was discarded. The threshold used was e times the maximum product score found so far, with e set by at 10~. While this may have led to the pruning of a path contributing to the 'best' pronunciation, its contribution would be very small. Again, this gave a very significant improvement in run times for the testing of lexical words (section 5.2 below) but was unnecessary for the testing of pseudo-words.

Pronunciations have been obtained for:

• the 70 pseudowords from Glushko ( 1979) used by Dedina and Nusbaum to lest PRONOUNCE. The 'correct' pronunciation for these strings is taken to be that given by Dedina and Nusbaum (1991, pp. 61-62). We refer to this test set as D&N 70.

• the full set of 131 pseudowords from Glushko plus two others (goot, pome) plus two lexical

words (cat and play) plus the pseudohomophone kwik, as used by Sullivan (1992). The 'correct' pronunciations are those read aloud by Sullivan's 20 non-phonetician subjects, and transcribed by him as British Received Pronunciation. We refer to this test set as Sulll36. Our expectation is that the error rate will be relatively high for this test set, partly because of its larger size but more importantly because the subjects' dialect of Eng­lish is British RP rather that general American, i.e. there is a very significant inconsistency with the lexical databases.

The output has been scored on words correct and also on symbol score (i.e. phonemes correct) using the Lcvenshtein (1966) string-edit distance as shown in Table 1.

Our best comparison with Dedina and Nusbaum (D&N 70 test set, D&N model, Webster's database) gives a figure of 77.1% words correct. This is enorm­ously poorer than their approximately 91 % words cor­rect - yet the implementation, reference pronunci­ations and test set are (as far as we can tell) identical. The only relevant difference is that the Webster's data­base is automatically-aligned in their work and hand-aligned in ours. The clear expectation, given the crude nature of their alignment, is that they should have ex­perienced a higher error rate, not a dramatically lower one. Overall, this result accords far more closely with Sullivan and Damper (1993) whose best word score for automatic alignment (and using smaller databases but a larger test set) was just over 70%.

The re-implementation made 16 errors under the above conditions. Dcdina and Nusbaum's claim of 9% words correct amounts to just 6 errors, 3 of which arc the same as ours. The commonest problem is vowel substitution. It is possible to discount a very few errors as essentially trivial, reducing the error rate marginally to some 20%. We conclude, therefore, that Dedina and Nusbaum's reported error rate of 9% is unattainable.

In our opinion, a major deficiency of the simple shortest-path length heuristic is that the output can be­come unreasonably sensitive to rare or unique pronun­ciations. For instance, mone receives the strange pro­nunciation /mgni/ by analogy with anemone. Also, the pseudoword shead receives the bizarre, vowel-less pronunciation /J___AI (where '_' denotes the null phoneme) when using the D&N model and the TWB database. As illustrated in Fig. 2 earlier, this turns out to be a result of matching the unique but long (arc frequency 1) in conjunction with the very com­mon mapping sh —>■ /]'_/ as in she and shed (arc fre­quency 174) which swamps the overall score of 175. The same bizarre pronunciation does not occur with the PROD model. In this case, the path through the

Table 1 : Results for PbA of pseudowords with both dictionaries. See text for further specification.

(lei, 3) node has a product score of 12 x 30 = 360 for the pronunciation /Jed/ which considerably exceeds the score of 174 for/fd/.

Replacing the arc-sum heuristic of the D&N model by arc-product as in the PROD model leads to a considerable increase in performance, e.g. from 77.1 % words correct to 82.9% for the D&N 70 test set with Webster's database. In turn, the MP model per­forms better than PROD in all cases.

For the Sull 136 test set, our expectation of poorer performance (because of the larger test set and incon­sistency between of dialect between the target pro­nunciations and the lexical databases) is borne out for Webster's dictionary. For TWB, however, the perform­ance difference between test sets is less consistent.

The primary ability of a text-to-spcech system must be to produce correct pronunciations for lexical words (rather than pseudowords) which just happen to be ab­sent from the system's dictionary. Accordingly, we have tested the PbA implementations by removing each word in turn from its relevant database, and ob­taining a pronunciation by analogy with the remainder. In these tests, the transcription standard employed by the compilers of the dictionary becomes its own ref­erence and problems of transcription inconsistencies between input strings and lexical entries are avoided.

Results for the testing of lexical words are shown in Table 2. Again there are consistent performance differences with the 'standard' D&N model worst and the mapping probability (MP) model best. All models perform better with the TWB database than with Web­ster's, probably simply because of its smaller size.

For some lexical words, no pronunciation at all was produced because there was no complete path from Start to End in the lattice. This occurred for 92 of the TWB words and 117 of the Webster's words irre­spective of the scoring model. This is a serious short­coming: a PbA system should always produce a best-attempt pronunciation, even if it cannot produce the correct one. Sometimes, this failure is a consequence of the form of pronunciation lattice in which nodes are used to represent the 'end-points' of mappings. One of the inputs for which no pronunciation was found is anecdote, whose (partial) lattice is shown in Fig. 3. There is in fact no arc in the complete lattice between nodes (Ikl, 4) and (Id/, 5) because there is no cd -> /kd/ mapping anywhere in either dictionary. Nor is there an ecd or cdo trigram - with or without the right end-point phonemes - which could possibly bridge the gap. This problem is entirely avoided with the Sullivan and Damper style of lattice, because the shortest-length arc corresponds to a single-symbol mapping rather than to a bigram (which may be unique). Thus, there will always be a 'default' single-symbol mapping corres­ponding to the commonest pronunciation of the let­ter. This is not to say that Sullivan and Damper's sys­tem will necessarily produce the correct output here: it almost certainly will not because of the rarity of the c -> Ikl mapping in the -d context.

Another input which foils to produce a pronunci­ation is aardvark. The problem here is not that there is no aa bigram in the dictionary (which is found in words such as bazaar), but that it only appears to­wards the end of other words. Dedina and Nusbaum's strategy of performing substring matching only over a restricted range (the number of matching comparisons is equal to the difference in length between the input string and lexical entry) is at the root of this problem.