Genetically related languages often exhibit recur­rent sound correspondences (henceforth referred to simply as correspondences) in words with similar meaning. For example, t:d, 6:t, n:n, and other known correspondences between English and Latin are demonstrated by the word pairs in Table 1. Word pairs that contain such correspondences are called cognates, because they originate from the same protoform in the ancestor language. Correspon­dences in cognates are preserved over time thanks to the regularity of sound changes, which normally ap­ply to sounds in a given phonological context across all words in the language.

The determination ofcorrespondences is the prin­cipal step of the comparative method of language reconstruction. Not only does it provide evidence for the relatedness of languages, but it also makes it possible to distinguish cognates from loan words and chance resemblances. However, because man­ual determination of correspondences is an ex­tremely time-consuming process, it has yet to be ac­complished for many proposed language families. A system able to perform this task automatically

Table 1: Examples of English-Latin cognates exhibiting correspondences. The corresponding phonemes shown in boldface originate from a sin­gle proto-phoneme.

from unprocessed bilingual wordlists could be of great assistance to historical linguists. The Recon­struction Engine (Lowe and Mazaudon, 1994), a set of programs designed to be an aid in language re­construction, requires a set of correspondences to be provided beforehand.

The determination of correspondences is closely related to another task that has been much stud­ied in computational linguistics, the identification of cognates. Cognates have been employed for sentence and word alignment in bitexts (Simard et al., 1992; Melamed, 1999), improving statisti­cal machine translation models (Al-Onaizan et al., 1999), and inducing translation lexicons (Koehn and Knight, 2001). Some of the proposed cognate identification algorithms implicitly determine and employ correspondences (Tiedemann, 1999; Mann and Yarowsky, 2001).

Although it may not be immediately apparent, there is a strong similarity between the task of matching phonetic segments in a pair of cognate words, and the task of matching words in two sen­tences that are mutual translations (Figure 1). The consistency with which a word in one language is translated into a word in another language is mir­rored by the consistency of sound correspondences. The former is due to the semantic relation of syn­onymy, while the latter follows from the principle of the regularity of sound change. Thus, as already asserted by Guy (1994), it should be possible to use similar techniques for both tasks.

The primary objective of the method proposed in this paper is the automatic determination of corre­spondences in bilingual wordlists, such as the one in Table 1. The method exploits the idea of relat­ing correspondences in bilingual wordlists to trans-lational equivalence associations in bitexts through the employment of models developed in the con­text of statistical machine translation, The second task addressed in this paper is the identification of cognates on the basis of the discovered correspon­dences. The experiments to be described in Sec­tion 6 show that the method is capable of determin­ing correspondences in bilingual wordlists in which less than 30% of the pairs are cognates, and out­performs comparable algorithms on cognate identi­fication. Although the experiments focus on bilin­gual wordlists, the approach presented in this paper could potentially be applied to other bitext-related tasks.

In a schematic description of the comparative method, the two steps that precede the determi­nation of correspondences are the identification of cognate pairs (Kondrak, 2001), and their phonetic alignment (Kondrak, 2000). Indeed, if a compre­hensive set of correctly aligned cognate pairs is available, the correspondences could be extracted by simply following the alignment links. Unfortu­nately, in order to make reliable judgments of cog­nation, it is necessary to know in advance what the correspondences are. Historical linguists solve this apparent circularity by guessing a small number of likely cognates and refining the set of correspon­dences and cognates in an iterative fashion.

Guy (1994) outlines an algorithm for identifying cognates in bilingual wordlists which is based on correspondences. The algorithm estimates the prob­ability of phoneme correspondences by employing a variant of the % statistic on a contingency ta­ble, which indicates how often two phonemes co-occur in words of the same meaning. The prob­abilities are then converted into the estimates of cognation by means of some experimentation-based heuristics. The paper does not contain any eval­uation on authentic language data, but Guy's pro­gram COGNATE, which implements the algorithm, is publicly available. An experimental evaluation of COGNATE is described in Section 6.

Oakes (2000) describes a set of programs that together perform several steps of the comparative method, from the determination of correspondences in wordlists to the actual reconstruction ofthe proto-forms. Word pairs are considered cognate if their edit distance is below a certain threshold. The edit operations cover a number of sound-change cate­gories. Sound correspondences are deemed to be regular if they are found to occur more than once in the data. The paper describes experimental results of running the programs on a set of wordlists rep­resenting four Indonesian languages, and compares those to the reconstructions found in the linguistic literature. Section 6 contains an evaluation of one of the programs in the set, JAKARTA, on the cog­nate identification task.

In statistical machine translation, a translation model approximates the probability that two sen­tences are mutual translations by computing the product of the probabilities that each word in the target sentence is a translation of some source lan­guage word. A model oftranslation equivalence that determines the word translation probabilities can be induced from bitexts. The difficulty lies in the fact that the mapping, or alignment, of words between two parts of a bitext is not known in advance.

Algorithms for word alignment in bitexts aim at discovering word pairs that are mutual translations. A straightforward approach is to estimate the likeli­hood that words are mutual translations by comput­ing a similarity function based on a co-occurrence statistic, such as mutual information, Dice coeffi­cient, or the % test. The underlying assumption is that the association scores for different word pairs are independent of each other.

Melamed (2000) shows that the assumption ofin-dependence leads to invalid word associations, and proposes an algorithm for inducing models oftrans-lational equivalence that outperform the models that are based solely on co-occurrence counts. His mod­els employ the one-to-one assumption, which for­malizes the observation that most words in bitexts are translated to a single word in the correspond­ing sentence. The algorithm, which is related to the expectation-maximization (EM) algorithm, iter-atively re-estimates the likelihood scores which rep­resent the probability that two word types are mu­tual translations. In the first step, the scores are initialized according to the G statistic (Dunning, 1993). Next, the likelihood scores are used to in­duce a set of one-to-one links between word tokens in the bitext. The links are determined by a greedy competitive linking algorithm, which proceeds to link pairs that have the highest likelihood scores. After the linking is completed, the link counts are used to re-estimate the likelihood scores, which in turn are applied to find a new set of links. The process is repeated until the translation model con­verges to the desired degree.

Melamed presents three translation-model esti­mation methods. Method A re-estimates the like­lihood scores as the logarithm of the probability of jointly generating the pair of words u and v:

where links u v denotes the number of links in­duced between u and v. Note that the co-occurrence counts of u and vare not used for the re-estimation, In Method B, an explicit noise model with auxil­iary parameters A+ and X~ is constructed in order to improve the estimation of likelihood scores. A+ is a probability that a link is induced between two co-occurring words that are mutual translations, while At is a probability that a link is induced between two co-occurring words that are not mutual trans­lations. Ideally, A+ should be close to one and At should be close to zero. The actual values of the two parameters are calculated by the maximum likeli­hood estimation. Let cooc u v be the number of co-occurrences of u and v.Thescore function is defined as:

where B k n p denotes the probability of k being generated from a binomial distribution with param­eters n and p.

In Method C, bitext tokens are divided into classes, such as content words, function words, punctuation, etc., with the aim of producing more accurate translation models. The auxiliary parame­ters are estimated separately for each class.

Thanks to its generality and symmetry, Melamed's parameter estimation process can be adapted to the problem ofdetermining correspondences. The main idea is to induce a model of sound correspondence in a bilingual wordlist, in the same way as one in­duces a model of translational equivalence among words in a parallel corpus. After the model has con­verged, phoneme pairs with the highest likelihood scores represent the most likely correspondences.

While there are strong similarities between the task of estimating translational equivalence of words and the task of determining recurrent corre­spondences of sounds, a number of important modi­fications to Melamed's original algorithm are neces­sary in order to make it applicable to the latter task. The modifications include the method of finding a good alignment, the handling of null links, and the method of computing the alignment score.

For the task at hand, I employ a different method of aligning the segments in two corresponding se­quences. In sentence translation, the alignment links frequently cross and it is not unusual for two words in different parts of sentences to correspond. In contrast, the processes that lead to link inter­section in diachronic phonology, such as metathe­sis, are quite sporadic. The introduction of the no-crossing-links constraint on alignments not only leads to a dramatic reduction of the search space, but also makes it possible to replace the approximate competitive-linking algorithm of Melamed with a variant of the well-known dynamic programming algorithm (Wagner and Fischer, 1974; Kondrak, 2000), which computes the optimal alignment be­tween two strings in polynomial time.

Null links in statistical machine translation are induced for words on one side of the bitext that have no clear counterparts on the other side of the bitext. Melamed's algorithm explicitly calculates the likelihood scores of null links for every word type occurring in a bitext. In diachronic phonol­ogy, phonological processes that lead to insertion or deletion of segments usually operate on individ­ual words rather than on particular sounds across the language. Therefore, I model insertion and deletion by employing a constant indel penalty for unlinked segments.

The alignment score between two words is com­puted by summing the number of induced links, and applying an indel penalty for each unlinked seg­ment, with the exception of the segments beyond the rightmost link. The exception reflects the relative instability of word endings in the course of linguis­tic evolution. In order to avoid inducing links that are unlikely to represent recurrent sound correspon­dences, only pairs whose likelihood scores exceed a set threshold are linked. All correspondences above the threshold are considered to be equally valid. In the cases where more than one best alignment is found, each link is assigned a weight that is its av­erage over the entire set of best alignments (for ex­ample, a link present in only one of two competing alignments receives the weight of 0.5).

The method described above has been implemented as a C++ program, named CORDI, which will soon be made publicly available. The program takes as input a bilingual wordlist and produces an ordered list of correspondences. A model for a 200-pair list usually converges after 3-5 iterations, which takes only a few seconds on a Sparc workstation. The user can choose between methods A, B, and C, de­scribed in Section 3, and an additional Method D. In Method C, phonemes are divided into two classes: non-syllabic (consonants and glides), and syllabic (vowels); links between phonemes belonging to dif­ferent classes are not induced. Method D differs from Method C in that the syllabic phonemes do not participate in any links.

Adjustable parameters include the indel penalty ratio d and the minimum-strength correspondence threshold t. The parameter d fixes the ratio be­tween the negative indel weight and the positive weight assigned to every induced link. (A lower ratio causes the program to be more adventurous in positing sparse links.) The parameter t controls the tradeoff between reliability and the number of links. In Method A, the value of t is the minimum number of phoneme links that have to be induced for the correspondence to be valid. In methods B, C, and D, the value of t implies a likelihood score threshold of t ■ log which is a score achieved by a pair of phonemes that have t links out of t co­occurrences. In the experiments reported in Sec­tion 6, d was set to 0 15, and t was set to 1 (suf­ficient to reject all non-recurring correspondences). In Method D, where the lack of vowel links causes the linking constraints to be weaker, a higher value of t 3 was used. These parameter values were op­timized on the development set described below.

I have presented a novel approach to the determi­nation of correspondences in bilingual wordlists. The results of experiments indicate that the ap­proach is robust enough to handle a substantial amount of noise that is introduced by unrelated word pairs. CORDI does well even when the number of non-cognate pairs is more than double the number of cognate pairs. When tested on the cognate-identiication task, CORDI achieves sub­stantially higher precision than comparable pro­grams. The correspondences are explicitly posited, which means that, unlike in some statistical ap­proaches, they can be veriied by examining indi­vidual cognate pairs. In contrast with approaches that assume a rigid alignment based on the syllabic structure, the models presented here can link phonemes in any word position.

Currently, I am working on the incorporation of complex correspondences into the cognate identifi­cation algorithm by employing Melamed's (1997) algorithm for discovering non-compositional com­pounds in parallel data. Such an extension would overcome the limitation of the one-to-one model, in which links are induced only between individual phonemes. Other possible extensions include taking into account the phonological context of correspon­dences, combining the correspondence-based ap­proach with phonetic-based approaches, and iden­tifying correspondences and cognates directly in dictionary-type data.

The results presented here prove that the tech­niques developed in the context of statistical ma­chine translation can be successfully applied to a problem in diachronic phonology. The transfer of methods and insights should also be possible in the other direction.

Thanks to Graeme Hirst, Radford Neal, and Suzanne Stevenson for helpful comments, to Michael Oakes for assistance with JAKARTA, and to Gemma Enriquez for helping with the experimen­tal evaluation ofCOGNATE. This research was sup­ported by the Natural Sciences and Engineering Re­search Council ofCanada.