treebank, maximum likelihood estimation can be applied to learn the probability values in the model.

More recently, new machine learning methods have been developed or ex­tended to handle models of grammatical structure. Notably, conditional estimation (Ratnaparkhi, Roukos, and Ward 1994; Johnson et al. 1999; Lafferty, McCallum, and Pereira 2001), maximum margin estimation (Taskar et al. 2004), and unsupervised con-trastive estimation (Smith and Eisner 2005) have been applied to structured models. Weighted grammars learned in this way differ in two important ways from traditional, generative models. First, the weights can be any positive value; they need not sum to one. Second, features can "overlap," and it can be difficult to design a generative model that uses such features. The benefits of new features and discriminative training methods are widely documented and recognized.

This article focuses specifically on the first of these differences. It compares the expressive power of weighted context-free grammars (WCFGs), where each rule is associated with a positive weight, to that of the corresponding PCFGs, that is, with the same rules but where the weights of the rules expanding a nonterminal must sum to one.

One might expect that because normalization removes one or more degrees of free­dom, unnormalized models should be more expressive than normalized, probabilistic models. Perhaps counterintuitively, previous work has shown that the classes of proba­bility distributions defined by WCFGs and PCFGs are the same (Abney, McAllester, and Pereira 1999; Chi 1999).

However, this result does not completely settle the question about the expressive power of WCFGs and PCFGs. As we show herein, a WCFG can define a conditional distribution from strings to trees even if it does not define a probability distribution over trees. Because these conditional distributions are what are used in classification tasks and related tasks such as parsing, we need to know the relationship between the classes of conditional distributions defined by WCFGs and PCFGs. In fact we extend the results of Chi and of Abney et al., and show that WCFGs and PCFGs both define the same class of conditional distribution. Moreover, we present an algorithm for converting an arbitrary WCFG that defines a conditional distribution over trees given strings but possibly without a finite partition function into a PCFG with the same rules as the WCFG and that defines the same conditional distribution over trees given strings.

This means that maximum conditional likelihood WCFGs are non-identifiable, be­cause there are an infinite number of rule weights all of which maximize the conditional likelihood.

Unless otherwise specified, we assume a fixed underlying context-free grammar G. Let fi(G) be the set of (finite) trees that G generates. For any t g 0(G), the score se(x)of t is defined as follows:

where f (X — a; t) is the number of times X — a is used in the derivation of the tree t. The partition function Z(G) is the sum of the scores of the trees in fi(G).

Because we have imposed no constraints on G, the partition function need not equal one; indeed, as we show subsequently the partition function need not even exist. If Z(G) is finite then we say that the WCFG is convergent, and we can define a Gibbs probability distribution over Q(G) by dividing by Z(G):

A probabilistic CFG, or PCFG, is a WCFG in which the sum of the weights of the rules expanding each nonterminal is one:

It is easy to show that if Ge is a PCFG then Z(G) < 1. A tight PCFG is a PCFG Gefor which Z(G) = 1. Necessary conditions and sufficient conditions for a PCFG to be tight are given in several places, including Booth and Thompson (1973) and Wetherell (1980).

We now describe the results of Chi (1999) and Abney, McAllester, and Pereira (1999). Let G = {Ge} denote the set of the WCFGs based on the CFG G (i.e., the WCFGs in G all have the same underlying grammar G but differ in their rule weight vectors G). Let GZ<oo be the subset of G for which the partition function Z(G) is finite, and let GZ=oc = G\ GZ<oo be the subset of G with an infinite partition function. Further let GPCFGdenote the set of PCFGs based on G. In general, Gpcfg is a proper subset of GZ<o ,that is, every PCFG is also a WCFG, but because there are weight vectors G that don't obey Equation 2, not all WCFGs are PCFGs.

However, this does not mean that WCFGs are more expressive than PCFGs. As noted above, the WCFGs GZ<oc, define Gibbs distributions. Again, for a fixed G,let PZ<o be the probability distributions over the trees Q(G) defined by the WCFGs GZ<oc, and let PPCFG be the probability distributions defined by the PCFGs GPCFG. Chi (Propo­sition 4) and Abney, McAllester, and Pereira (Lemma 5) showed that PZ<o = PPCFG, namely, that every WCFG probability distribution is in fact generated by some PCFG. There is no "Vz=o" because there is no finite normalizing term Z(G) for such WCFGs.

where Zx(G) is the sum of scores for all parses of x. Note that Equation (4) will be ill-defined when Zx(G) diverges. Because Zx(G) is constant for a given x, solving Equa­tion (3) is equivalent to choosing t to maximize Pe (t | x).

We turn now to classes of these conditional distribution families. Let CZ<00 (CPCFG) be the class of conditional distribution families that can be expressed by grammars in Gz«x> (GPCFG, respectively). It should be clear that, because Vz«00 = PPCFG, Cz«00 = CPCFGsince a conditional family is derived by normalizing a joint distribution by its marginals.

We now define another subset of G.Let GZn<00 contain every WCFG Ge = (G, G) such that, for all n > 0, (Note that, to be fully rigorous, we should quantify n in GZn<0, writing "GynZn(e)«x>." We use the abbreviated form to keep the notation crisp.) For any Ge £ GZn<0,italso follows that, for any x £ L(G), Zx(G) < oo; the converse holds as well.

It follows that any WCFG in GZn«00 can be used to construct a conditional dis­tribution of trees given the sentence, for any sentence x £ L(G). To do so, we only need to normalize se (t)by Zx(G) (Equation (4)). Let GZn=00 contain the WCFGs where some Zn(G) diverge; this is a subset of GZ=0. To see that GZ=0 n GZn<00 = 0, consider Example 1.

Example 1

This grammar produces binary structures over strings in a+. Every such tree receives score 1. Because there are infinitely many trees, Z(G) diverges. But for any fixed string an, the number of parse trees is finite. This grammar defines a uniform conditional distribution over all binary trees, given the string.

For a grammar Ge to be in Gzn<0, it is sufficient that, for every nonterminal X £ N, the sum of scores of all cyclic derivations X =>+ X be finite. Conservatively, this can be forced by eliminating epsilon rules and unary rules or cycles altogether, or by requiring the sum of cyclic derivations for every nonterminal X to sum to strictly less than one. Example 2 gives a grammar in GZn=0 with a unary cyclic derivation that does not "dampen." Example 2

For any given an, there are infinitely many equally weighted parse trees, so even the set of trees for an cannot be normalized into a distribution (Zn(G) = oo). Generally speaking, if there exists a string x £ L(G) such that the set of trees that derive x is not 2 Here, full rigor would require quantification of n, writing "G^nzn(e)=0." finite (i.e., there is no finite bound on the number of derivations for strings in L(G); the grammar in Example 2 is a simple example), then Gzn<oo and Gz<oo are separable.

For a given CFG G, a conditional distribution over trees given strings is a function X* — (0(G) — [0,1]). Our notation for the set of conditional distributions that can be expressed by Gzn<oo is CZn<o. Note that there is no "CZn=OQ" because an infinite Zn(G) implies an infinite Z(x) for some sentence x and therefore an ill-formed conditional fam­ily. Indeed, it is difficult to imagine a scenario in computational linguistics in which non-dampening cyclic derivations (WCFGs in GZn=oc) are desirable, because no linguistic explanations depend crucially on arbitrary lengthening of cyclic derivations.

We now state our main theorem.

Theorem 1

For a given CFG G, Czn<o = Cz<o. Proof

Suppose we are given weights G for G such that Ge € GZn<o0. We will show that the sequence Z1(G),Z2(G),... is bounded by an exponential function of n, then describe a transformation on G resulting in a new grammar Ge> that is in Gz<oo and defines the same family of conditional distributions (i.e., Vt € 0(G),Vx € L(G), Pe (t | x) = Pe' (t I x)).

First we prove that for all n > 1 there exists some c such that Zn (G) < cn .Given Ge, we construct Ge in CNF that preserves the total score for any x € L(G). The existence of Ge was demonstrated by Goodman (1998, Section 2.6), who gives an algorithm for constructing the value-preserving weighted grammar Ge from Ge.

Note that G = (N, S, X, R), containing possibly more nonterminals and rules than G. The set of (finite) trees 0 (G) is different from 0(G); the new trees must be binary and may include new nonterminals.

Next, collapse the nonterminals in N into one nonterminal, S. The resulting gram­mar is Ge = (({S}, S, X, R), G). R contains the rule S — SS and rules of the form S — a for a € X. The weights of these rules are

The grammar Ge will allow every tree allowed by Ge (modulo labels on nonterminal nodes, which are now all S). It may allow some additional trees. The score of a tree under Ge will be at least as great as the sum of scores of all structurally equivalent trees under Ge, because p and u are defined to be large enough to absorb all such scores. It follows that, for all x € L(G):

3 We are grateful to an anonymous reviewer for pointing this out, and an even stronger point: for a given G, G and Gzn<o have a nonempty set-difference if and only if G has infinite ambiguity (some x € L(G) has infinitely many parse trees).

A Mealy HMM defines a probability distribution over pairs (x, n), where x is a length-n sequence (x\,x2,xn)e Sn and tt = (n0,n\,n2,nn)e Nn+1 is a state (or nonterminal) path. The distribution is given by n0 is assumed, for simplicity, to be constant and known; we also assume that every state transition emits a symbol (no e arcs), an assumption made in typical tagging and chunking applications of HMMs. We can convert a Mealy HMM to a PCFG by including, for every tuple (x, n, (x e S and n, $ e N) such that p(x, n | ^) > 0, the rule n — x ^, with the same probability as the corresponding HMM transition. For every n such that p(STOP I n), we include the rule n — e, with probability p(STOP | n).

A Moore HMM factors the distribution p(x, n | ^)into p(x | n) • p(n | ^). A Moore HMM can be converted to a PCFG by adding a new nonterminal n for every state n and including the rules — n (with probability p(n | ^)) and n — x n (with probability p(x | n)). Stop probabilities are added as in the Mealy case. For a fixed number of states, Moore HMMs are less probabilistically expressive than Mealy HMMs, though we can convert between the two with a change in the number of states.

We consider Mealy HMMs primarily from here on. If we wish to define the distri­bution over paths given words, we conditionalize

This is how scores are assigned when selecting the best path given a sequence.

For a grammar G that is right-linear, we can therefore talk about the set of HMM (right-linear) grammars GHMM, the set of probability distributions PHMM defined by those grammars, and CHMM, the set of conditional distributions over state paths (trees) that they define.