semantics, and have dreams of a grand unified head dependency-driven Theory of Everything that that will one day make probabilistic and deterministic components work together to yield Meaning. How come we don't get respect?

The answer, I'm afraid, is that we haven't yet managed to form ourselves into a dis­cipline, within which those important truths are respected and held beyond reasonable doubt, in whose name we stand united in the eyes of the world. Instead, our history has been sectarian, with dominant factions seeking to suppress proposals that diverge too much from the current orthodoxy, until they too are overthrown.

This tendency is very evident in computer science peer reviewof proposals to funding agencies, which are notably harsher than in other sciences, and which have the effect of making the agencies assume a shortage of good science in the field, so that they reduce funding accordingly. Worse still, when we have theoretical disagreements, we play them out in public. (The split between connectionists and the rest of AI/NLP is a case in point. It is sad to note that similar internal dissension seems to have played a role in the adverse conclusions reached in both the ALPAC and Lighthill reports.)

The physicists don't do this. When they reviewproposals, they rate them excel­lent at around twice the rate in CS, even if it means not getting funded themselves in the current round. The funding agencies conclude that there is a surplus of good science there, and seek or allocate further funding for the next round. When there are disagreements—as there seem to be currently about string theory—they are argued out behind closed doors, until a consensus can be reached and be presented to the general public. The biologists behaved similarly over a recent disagreement concerning methods for sequencing the genome.

One measure of our own fisiparous tendencies is that, while from the 1950s to the 1980s, the information theoreticians and statistical modelers among us used to make common cause with the linguists, we have subsequently drifted apart. We have come to believe that the linguists have forgotten Zipf's law, which says that most of the variance in language behavior can be captured by a small part of the system.

The linguists, on the other hand, think that it is we who have forgotten Zipf's law, which also says that most of the information about the language system as a whole is in the Long Tail.

It is we who are at fault here, because the machine learning techniques that we rely on are actually very bad at inducing systems for which the crucial information is in rare events—like physics, for example.

There is a grave danger here for the future of our putative discipline. One day, either because of the demise of Moore's law, or simply because we have done all the easy stuff, the Long Tail will come back to haunt us. For example, consider the current state of our former nemesis MT.

English, using documents of the United Nations and the European Union key sources.

And howa newtranslation Ouch said that although the quality would not be complete That was a good in the previous translation mechanism, and that the correct translation mostly might be good enough for some tasks. He stated that more data be fed by the results were better.

... He commended Miles Osborne Professor at the University of Edinburgh, who died last year at work in the company's efforts to Google, but he pointed out that the software will not prevail over people skilled in translations as they do in the game of chess and should use software to understand and not to complete documents.

The first thing to notice is that this is really very good. It is quite clear what the story is, and you can even guess that what Franz Och actually said in the English reference text was: "The more data we feed into the system, the better it gets." It even seems to know that "Google" can be a verb.

However, it exhibits all the problems to which we have always known MT is heir. Both pronouns "he" in the last paragraph will be understood as referring to Franz, whereas in the reference text it is Miles Osborne who does the commending and the pointing out. Moreover, the alarming rumor of the latter's death has been greatly exag­gerated by the English language-model: The reference text says he "spent a sabbatical last year working on the Google project." The human Arabic translation says much the same, but the Arabic words for spent and died are homographs, and the newswire-based model favors the latter.

And of course, our friends the Construction Grammarians will gleefully point out that the system makes a hash of the unbounded dependency in Franz's use of what they call the "MORE-MORE" construction.

We can specifically probe the disability with respect to other kinds of unbound­ed dependencies, using back-translation on artificially generated (but in-domain) examples:

This is the bank that bought the company. "This is the bank that bought the company." This is the company that the bank bought. "*This is the company that bought the bank." This is the bank that wants to buy the company. "This is the bank, which wants to buy the company." This is the company which the bank wants to buy. "*This is the company that wants to buy the bank."

Comparing the examples that are translated correctly and those (labeled *) that are not, the generalization is already clear: even a 5-gram model can only handle root subject relative clauses. Object relatives are beyond the horizon. (These effects are robust under variation of the content words.) Here are some more challenging embedded examples that confirm the diagnosis:

This is the company that said the bank bought bonds. "This is the company that said the bank bought the bonds." This is the company that the bank said bought bonds. "*This is the company that said the bank bought the bonds." These are the bonds that the company said that the bank bought. "*These are the bonds that the bank bought the company." 4. Who Cares?

What does this tell us? Nothing that we shouldn't have already known. We knew that n-gram models and FSTs weren't going to handle long-range dependencies, because Chomsky told us so. That's one of the Big Truths of computational linguistics.

Our situation is in fact rather like that of the physicists. We have one theory for talking about phenomena on a large scale, just as they have the General Theory of Relativity, and another theory for talking about the very small scale, just as they have Quantum Theory. Like the physicists, we have difficulty in reconciling those theoretical levels. Like them, some of us think it's fine to have two theories, whereas others of us think it's intolerable.

The former kind of computational linguist will point out that long-range depen­dencies are sparse. (There are around 20K *T* empty categories in around 16K of the roughly 40K sentences in the Penn Treebank, of which around 6K seem to be non-subject, non-sentential wh-traces of some kind.) Worrying about them isn't going to significantly impact overall parser dependency recovery rates, much less n-gram-precision-based BLEU scores. By the time we have fed enough data into the system to make it knowthat spending a sabbatical at Google is more likely than dying there, and Moore's lawhas made the machines exponentially bigger and faster, and fancier algorithms allowus to deal with bigger n-grams, maybe this problem will go away.

Maybe. These are certainly good reasons for the IR and SMT researchers to keep working at the large scale, giving the world these amazing search engines and transla­tion aids that give human beings vastly increased access to other cultures. This is our discipline's equivalent in street-credibility terms of delivering atomic energy and robots on Mars.

Nevertheless, to the other kind of computational linguist, it sounds depressingly like getting better and better at recalling what is already well-known, and understand­ing what has often been said before.

They point out that, in the long run, finite state methods alone may simply not work. Accuracy in most areas (WER in ASR, BLEU score in SMT, Eval-b for parsers) is at best linear in the logarithm of the amount of training data. Even optimistic extrapolation of current learning curves suggests truly incredible amounts of data will be needed (Lamel, Gauvain, and Adda 2002; Moore 2003; Knight and Koehn 2004; Brants et al. 2007).

Moreover, the more of the local stuff we get right, the more users will come to trust the software, and hence the more noticeable long range dependencies will become, and the more upset people will get if they are deceived by a wrong analysis.

What this should tell us is that the Long Tail is not mocked. Long-range depen­dencies of the kind investigated above are semantically crucial. Ignoring them disrupts all the other dependencies in those examples. (They are also more frequent in genres like questions.) So we need to remember—and above all, teach our students—what our discipline tells us the problem is, even when it's not doing much for our BLEU score. In this connection, it is encouraging to see that many of the MT papers in the 2007 ACL explicitly invoked syntax-level representations.

But we need to do more than this. To get respect, and avoid the risk of yet another AI/NLP Winter, we will need to pull ourselves together as a discipline, lift our eyes above our rivalries, and take a longer view of where we are going (and where we have come from) than seems to be the current style. This will probably require a gradual move to a more considered and authoritative style of publication, with journal articles taking the place of hastily written and reviewed conference papers, as another author of this column recently suggested.

It will mean speaking with one voice, as the physicists and biologists do, and supporting a diversity of views that transcends fashion and funding, wherever the science is good. It also means telling the public in honest terms howto think about what we do, what can be relied on and what the really hard problems are, in good times and bad. This should not be too difficult if we keep reminding them and ourselves of the following:

Human knowledge is expressed in language. So computational linguistics is very important.